Extended reality (XR) is an umbrella term for any technology that alters reality by adding digital elements to the physical or real-world environment to any extent and includes but is not limited to, augmented reality (AR), mixed reality (MR) and virtual reality (VR).
Any new technology that blends the physical and virtual worlds will also be categorized as XR. The “X” in XR stands for any variable—any letter of the alphabet—that may be used in the future for such technologies.
The term XR includes AR, MR, VR, and any technology that blends the physical and the digital world.
Therefore, the term extended reality does not refer to any specific technology; it includes any existing or new technologies that may be created in the future that alter reality, either by blending the digital and the physical world or by creating an entirely virtual environment.
UX design for XR experiences vastly differs from traditional UX design. UX for XR is not screen-bound and needs to consider 3D spaces and the safety and physical comfort of the user. Although there are not a set of standardized guidelines yet, there are some proposed frameworks to set the bases for UX designers. The study Exploring User Experience Guidelines for Designing HMD Extended Reality Applications by the University of Cagliari (Vi, 2022) proposes the following guidelines:
Organize the spatial environment to improve efficiency.
Create flexible interactions and environments.
Prioritize users’ comfort.
Do not overwhelm the user.
Design around hardware capabilities and limitations.
Use cues to help users through their experience.
Create a compelling XR experience.
Build upon real-world knowledge.
Allow users to feel in control of the experience.
Allow for trial and error.
In the book UX for XR, Cornel Hillmann suggests that object-oriented UX methodology (OOUX) might be an excellent approach to solving design problems in the XR world. A typical UX process starts with user research and user flows and progresses to wireframes and mockups; therefore, it usually defines flows, interactions and features before defining the objects. OOUX first breaks down a design problem's complexity by focusing on the core content—first the objects—and then assigning actions to these objects. For example, using the OOUX design process, you would first focus on the button and then assign an action. This approach typically has four phases:
Discover objects.
Define objects.
Establish relationships.
Force rank objects.
However, more studies and research are needed to establish common standards and best practices for UX for XR.
Learn more about AR and VR in our UX Design for Augmented Reality course and our UX Design for Virtual Reality course.
Watch the How to Innovate with XR Master Class with Associate Professor at the University of Michigan Michael Nebeling.
Watch the How To Craft Immersive Experiences in XR with Senior Product Designer at Magic Leap Mia Guo.
Watch the How To Influence Behavior Through Virtual Reality Narratives on-demand Master Class by VR pioneer Mel Slater.
To learn more about UX design guidelines in XR, read this paper: Vi, S. (2022). Exploring User Experience Guidelines for Designing HMD Extended Reality Applications.
Read the book UX for XR by Cornel Hillmann for inspiring insights about UX for XR.
Here's a revised list of highly cited scientific research within XR:
User Experience and Interaction: Alexandrovsky, D., Putze, S., Schwind, V., Mekler, E. D., Smeddinck, J. D., Kahl, D., Krüger, A., & Malaka, R. (2021). Evaluating user experiences in mixed reality. Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3411763.3441337
Education and Training: Garzón, J., & Acevedo, J. (2019). Augmented Reality in Education: A meta-analysis and research synthesis. This study evaluates AR's effectiveness in education through a comprehensive review and meta-analysis.
Healthcare: Powers, M.B., & Emmelkamp, P.M.G. (2008). Virtual reality exposure therapy for anxiety disorders: A meta-analysis. This research highlights VR's effectiveness in treating anxiety disorders, showcasing its potential in healthcare.
Cornel Hillmann’s book UX for XR: User Experience Design and Strategies for Immersive Technologies, provides an in-depth examination of UX design for VR and beyond.
Read Jason Jerald’s comprehensive book on VR, The VR Book: Human-Centered Design for Virtual Reality.
Extended Reality (XR) is an umbrella term that encompasses Virtual Reality (VR), Augmented Reality (AR), and Mixed Reality (MR), covering the full spectrum of real-and-virtual combined environments. Here's a concise differentiation:
VR immerses users in a fully virtual environment, disconnected from the real world.
AR overlays digital content onto the real world without creating a new environment.
MR combines elements of both AR and VR, allowing real and virtual elements to interact in real-time.
XR represents the entire spectrum, from fully real to fully virtual environments, including VR, AR, and MR technologies.
XR technologies offer innovative ways to interact with digital content, enhancing fields from education to healthcare with immersive experiences.
Read more about AR-MR-VR differences in the article Beyond AR vs. VR: What is the Difference between AR vs. MR vs. VR vs. XR?
Learn how to design your own VR experiences with our course: UX Design for Virtual Reality.
Learn how to design your AR experiences with our UX Design for Augmented Reality course.
Cornel Hillmann’s book UX for XR: User Experience Design and Strategies for Immersive Technologies, provides an in-depth examination of UX design for VR and beyond.
Extended Reality (XR) significantly impacts design, and has applications in:
Product Design: XR enables designers to visualize and interact with 3D models in real-world contexts, enhancing prototyping and user testing.
Interior Design: Through AR and VR, clients can experience spatial designs virtually, making informed decisions about layouts, furniture, and decor.
Educational Content: Designers create immersive learning experiences, facilitating engaging and interactive educational environments.
User Experience (UX) Design: XR provides tools for testing and refining user interfaces in virtual environments, ensuring intuitive and user-friendly interactions.
Architectural Visualization: Architects use XR to create walk-throughs of proposed buildings, allowing stakeholders to explore designs before construction.
XR technologies empower designers across various fields to innovate, create more engaging products, and improve the design process.
Cornel Hillmann’s book UX for XR: User Experience Design and Strategies for Immersive Technologies, provides an in-depth examination of UX design for VR and beyond.
Learn how to design your own VR experiences with our course: UX Design for Virtual Reality.
Learn how to design your AR experiences with our UX Design for Augmented Reality course.
UX design for Extended Reality (XR) differs from traditional UX design primarily due to its immersive nature and the need for spatial interaction design. While traditional UX focuses on 2D interfaces and user flows on screens, XR UX design requires creating 3D environments where users can move and interact with virtual elements as if they were physical objects. This involves considerations for user movement, depth perception, and interaction with both virtual and augmented elements, ensuring a seamless and intuitive experience. Additionally, XR UX design must account for new user inputs, such as gaze, gesture, and voice, moving beyond the click-and-type interfaces common in traditional UX design. The challenge lies in crafting experiences that are not only usable and accessible but also comfortable and engaging over extended periods, addressing the unique constraints and opportunities presented by VR, AR, and MR technologies.
Cornel Hillmann’s book UX for XR: User Experience Design and Strategies for Immersive Technologies, provides an in-depth examination of UX design for VR and beyond.
The key principles of UX design in Extended Reality (XR) center around immersion, interaction, and user comfort. Firstly, the design of immersive experiences requires designers to build convincing, engaging environments that leverage spatial audio and 3D visuals. Secondly, interaction design must be intuitive and utilize natural gestures, eye tracking, and voice commands to allow users to interact with virtual elements seamlessly. Lastly, user comfort is paramount, designers need to address potential issues like motion sickness through stable frame rates, ergonomic interface placement, and ensure a user's physical safety by considering their real-world surroundings. These principles ensure XR experiences are not only immersive and interactive but also accessible and enjoyable for extended periods and foster a deep sense of presence and engagement within virtual or augmented spaces.
Cornel Hillmann’s book UX for XR: User Experience Design and Strategies for Immersive Technologies, provides an in-depth examination of UX design for VR and beyond.
Learn how to design your own VR experiences with our course: UX Design for Virtual Reality.
Learn how to design your AR experiences with our UX Design for Augmented Reality course.
Common UX challenges in Extended Reality (XR) include user disorientation, motion sickness (cybersickness), and ensuring intuitive interaction. To overcome these, designers should focus on consistent and clear navigational cues to help orient users within virtual environments. The prevention of motion sickness requires careful attention to movement mechanics, such as limiting the speed of movement and avoiding sudden shifts that can disorient users. For intuitive interaction, designers should leverage familiar gestures and interactions from the real world to help users adapt more quickly to XR interfaces. Additionally, designers should provide users with control over their experience, such as the ability to adjust settings for comfort or accessibility and ensure a more inclusive design.
Cornel Hillmann’s book UX for XR: User Experience Design and Strategies for Immersive Technologies, provides an in-depth examination of UX design for VR and beyond.
Learn how to design your own VR experiences with our course: UX Design for Virtual Reality.
Learn how to design your AR experiences with our UX Design for Augmented Reality course.
User interface design in Extended Reality (XR) diverges significantly from traditional digital interfaces due to its immersive and three-dimensional nature. In XR, designers must account for spatial interactions, where the user can move around and interact with elements in a 3D space, unlike the flat, 2D screens of traditional interfaces. This requires a profound understanding of depth, scale, and user ergonomics to ensure interfaces are accessible and comfortable. Additionally, XR interfaces often utilize gesture, voice, and gaze-based inputs rather than mouse clicks or touchscreen taps, demanding a more intuitive design approach that leverages natural human behaviors. Creating effective XR UIs means prioritizing clarity, simplicity, and ease of navigation in a fully immersive environment, challenging designers to think beyond the screen and consider the user's entire physical space as part of the interface.
Cornel Hillmann’s book UX for XR: User Experience Design and Strategies for Immersive Technologies, provides an in-depth examination of UX design for VR and beyond.
Learn how to design your own VR experiences with our course: UX Design for Virtual Reality.
Learn how to design your AR experiences with our UX Design for Augmented Reality course.
Emerging trends in XR design focus on enhancing realism, accessibility, and user immersion. Advances in haptic feedback technologies are enabling more tactile interactions, which allows users to feel virtual objects for a more immersive experience. There's also a push towards the creation of more accessible XR environments through adaptive interfaces and voice controls, making XR more inclusive for users with different abilities.
Additionally, the integration of AI and machine learning is improving personalization and interactivity, tailoring experiences to individual user preferences and behaviors. Social XR is also gaining traction, with platforms developing shared virtual spaces for collaboration and socializing, bridging geographical gaps. What are the ethical considerations in designing for XR?
Cornel Hillmann’s book UX for XR: User Experience Design and Strategies for Immersive Technologies, provides an in-depth examination of UX design for VR and beyond.
Here’s the entire UX literature on Extended Reality (XR) by the Interaction Design Foundation, collated in one place:
We believe in Open Access and the democratization of knowledge. Unfortunately, world-class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.
If you want this to change, , link to us, or join us to help us democratize design knowledge!
