How to Use Voice Interaction in Augmented Reality

by Frank Spillers and Ditte Hvas Mortensen | | 14 min read

The user experience with augmented reality applications is at its best when it’s multimodal. As much as possible, users should be able to use gaze, gesture and voice in a seamless blend to create an interaction with the digital world that’s as natural as interaction with the physical world -but with all the added action possibilities that the digital overlay affords. That means that in order to create awesome augmented reality user experiences, you need to be able to design user interaction for all modalities. In this video, we’ll go through what you need to consider when designing voice interaction for augmented reality. Voice interactions have become ubiquitous and are also part of augmented reality, so you should know how to design for them!

Show Hide video transcript
  1. Transcript loading...

The Takeaway

Using voice interaction with augmented reality can really benefit the user by making the interaction fast and effortless, but it also has limitations; so, make sure you use it where it’s most appropriate – public places and fine-grained control are not the best match for voice interaction. When you design voice commands, make sure they are brief and simple, so they’re easy for users to utter and remember. To ensure that your entire user base can use your design, check that it can understand people with different accents. Testing your app with different types of users will help in broadening its scope to deliver pleasing and impressive experiences every time.

References and Where to Learn More

You can see Microsoft’s guidelines for voice design for HoloLens here.

Hero Image: Copyright holder: Jisc. Copyright terms and license: CC BY-NC-ND


Open Access - Link to us!

We believe in Open Access and the democratization of knowledge. Unfortunately, world class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.

If you want this to change, , link to us, or join us to help us democratize design knowledge!