Man interacting with AR interface

How to Design Gesture Interactions for Virtual and Augmented Reality

by Frank Spillers and Ditte Hvas Mortensen | | 20 min read
618 shares

Gesture interactions have become an easy and convenient form of interaction on touch devices largely because we have developed conventions for how gestures should be performed. In virtual and augmented reality, gestures are also a very natural way to interact with digital artifacts and menus, but we don’t have as clear conventions for how to design gesture-based interaction in AR and VR. Gestures and other hand movements are a natural part of how we interact with the world around us in everyday life, so it can seem like translating those gestures into interaction with the digital world would be straightforward. In reality, gestures are a complex combination of physical laws and cultural conventions, so you have to be careful if you want to design gestures that are easy to learn and perform, without fatiguing the user. In this video, we present guidelines for how to design gesture interaction in virtual and augmented reality.

Show Hide video transcript
  1. Transcript loading…

The Takeaway

Whenever you design gesture-based interaction for virtual and augmented reality, you need to make gestures as simple as possible. Users should be able to learn how to interact using gestures without having to learn a completely new language.

In order for gesture-based interaction to be successful, it’s crucial that you learn enough about the physicality of arm movements to know when gesture interaction becomes too tiresome. While big arm movements can look really cool, few users will be able to keep up that style of motion for long. You should also consider which affordances your users expect from different digital objects. If a digital object is similar to a physical object, users will expect to be able to perform the same kinds of actions with it. Finally, being aware of the social nature of gestures, both in the digital and physical world, is critical. Certain gestures are inappropriate or potentially embarrassing to perform in front of other people.

References and Where to learn more

You can read Christophe Tauziet’s article on designing for hands in VR here: Designing for Hands in VR

Hero Image: Copyright holder: youflavio. Copyright terms and license: CC BY-SA 2.0

Topics in This Article

618 shares

Open Access—Link to us!

We believe in Open Access and the democratization of knowledge. Unfortunately, world-class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.

If you want this to change, , link to us, or join us to help us democratize design knowledge!

Share Knowledge, Get Respect!

Share on:

or copy link

Cite according to academic standards

Simply copy and paste the text below into your bibliographic reference list, onto your blog, or anywhere else. You can also just hyperlink to this article.

Spillers, F. and Mortensen, D. (2024, July 3). How to Design Gesture Interactions for Virtual and Augmented Reality. Interaction Design Foundation - IxDF.

New to UX Design? We're Giving You a Free eBook!

The Basics of User Experience Design

Download our free ebook “The Basics of User Experience Design” to learn about core concepts of UX design.

In 9 chapters, we’ll cover: conducting user interviews, design thinking, interaction design, mobile UX design, usability, UX research, and many more!

A valid email address is required.
315,444 designers enjoy our newsletter—sure you don’t want to receive it?

New to UX Design? We're Giving You a Free eBook!

The Basics of User Experience Design

Download our free ebook “The Basics of User Experience Design” to learn about core concepts of UX design.

In 9 chapters, we’ll cover: conducting user interviews, design thinking, interaction design, mobile UX design, usability, UX research, and many more!

A valid email address is required.
315,444 designers enjoy our newsletter—sure you don’t want to receive it?