XRDC 2018 Session Scheduler

View, browse and sort the ever-growing list of XRDC 2018 sessions by pass type, track, and format. With XRDC 2018’s Session Scheduler, you can build your schedule in advance and access it during the show. Sessions do fill up so please arrive early to sessions that you would like to attend. Adding a session to your schedule does not guarantee you a seat.

Powering a Cognitive Assistant for the Blind Using AR

Yang Liu (PhD student, Caltech)

Markus Meister (Professor and Executive Officer, Caltech)

Pass Type: XRDC Pass - Get your pass now!

Track: Innovation

Format: Session

Vault Recording: TBD

Audience Level: All

Augmented reality can extend your cognitive abilities. Things that you cannot perceive can be brought within reach of your senses in real time. On that premise, speakers Yang Liu and Markus Meister from Caltech will present a cognitive assistant for blind persons based on the Microsoft HoloLens. The system identifies objects in the environment and gives them virtual voices, communicating maximally distilled knowledge in a way that is intuitive and natural to the human user. By interacting with these voices, the blind user gains all kinds of new abilities: from obstacle avoidance to formation and recall of spatial memories. With this technology, blind people can navigate an unfamiliar multi-story building on their first attempt, without any training. Liu and Meister will discuss the design principle behind the technology and argue that with the same principles, AR can power intelligent assistants that boost cognitive powers for everyone.


The experimental evidence demonstrates the efficacy of an AR powered system as an intelligent cognitive assistant, opening new opportunities for developers and users. They will benefit from learning a useful design principle stemmed from brain science for intelligent cognitive assistants and systematic ways to test them.

Intended Audience

The talk aims to inspire a general audience including consumers, XR content developers, AR hardware developers, and healthcare professionals.