Mixed Reality (MR)

Your constantly-updated definition of Mixed Reality (MR) and collection of videos and articles
76 shares

What is Mixed Reality (MR)?

Mixed reality (MR) refers to the blending of the physical world with the digital world. It allows the superposition and interaction between digital elements and the real-world environment to varying degrees. MR experiences can fall anywhere between the ends of the virtuality continuum. 

In MR experiences, the user is not bound to a screen and can interact with both the digital and the physical elements.

In the video below, you can see how digital objects interact with physical objects in an MR experience.

© Microsoft HoloLens, Fair-Use (link)


This video shows how MR experiences blend the physical and digital worlds. As you can see, the same experience would be different if the user was in a different place. The MR experience adapts to the user’s physical environment. Therefore, MR technology needs to get data from the physical environment to be able to construct the digital elements accordingly. MR requires advanced input methods and environmental perception.

MR includes any reality-altering technology and is not limited to augmented reality (AR) and virtual reality (VR).

AR vs. MR vs. VR. In AR, we see a digital robot superposed to a person. In MR, we see a digital robot shaking hands with a person. In VR, we see a fully digital environment where the robot and the avatar of the person are dancing.

In MR experiences the user can interact with both digital and physical elements. MR differs from AR—where digital and physical elements don’t interact— and VR—where the physical or real world is completely blocked out.

© Christian Briggs and the Interaction Design Foundation   

What is the Difference between AR and MR?

The main difference between AR and MR is that in AR experiences digital elements are overlaid on the physical world in real time but there is no interaction between them. AR technology allows the superposition of a digital layer on top of the physical world. Instead, in MR experiences digital elements are not only superposed upon the real-world environment but also interact with it.  

What is the Difference between VR and MR?

The main difference between VR and MR is that in VR experiences the physical world is completely blocked out. Instead, MR experiences blend the digital and the physical world to any degree. Therefore, VR technology completely ignores the environment which the user is in, whereas MR experiences process the environment which the user is in and include it in the experience. Similarly, in a VR experience, the user only interacts with the virtual environment, whereas in an MR experience, the user interacts with both virtual and physical elements. 

Learn More about Mixed Reality

Watch the How To Influence Behavior Through Virtual Reality Narratives on-demand Master Class by VR pioneer Mel Slater.

Learn how to design your own VR experiences with our course, UX Design for Virtual Reality and how to create your AR experiences with our UX Design for Augmented Reality course.

Questions related to Mixed Reality (MR)

What are some highly cited scientific research about mixed reality?

Highly cited scientific research in mixed reality (MR) often explores its applications, challenges, and theoretical foundations. Notable works include:

  • Speicher, M., Hall, B. D., & Nebeling, M. (2019). What is mixed reality? Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3290605.3300767 

  • Rokhsaritalemi, S., Sadeghi-Niaraki, A., & Choi, S.-M. (2020). A review on Mixed reality: Current trends, challenges and prospects. Applied Sciences, 10(2), 636. https://doi.org/10.3390/app10020636 

  • Billinghurst, M., & Kato, H. (1999). Collaborative mixed reality. Mixed Reality, 261–284. https://doi.org/10.1007/978-3-642-87512-0_15 r

What are some recommended books that cover mixed reality well?

Some books that offer different perspectives about mixed reality are:

  • "Virtual and Augmented Reality in Education, Art, and Museums" by Giuliana Guazzaroni and Anitha S. Pillai (Editors): This book delves into how AR and VR technologies transform education, arts, and museums, making these fields more accessible and engaging.

  • "Virtual Reality (The MIT Press Essential Knowledge series)" by Samuel Greengard: Provides a comprehensive overview of AR, VR, and MR, exploring current developments, applications across industries, and their potential future impact.

  • "Research Handbook on the Law of Virtual and Augmented Reality" by Woodrow Barfield and Marc J. Blitz: Addresses the legal challenges and considerations of immersive technologies, including data protection, intellectual property rights, and the real-world implications of virtual actions.

How does Mixed Reality differ from Augmented Reality (AR) and Virtual Reality (VR)?

Mixed Reality (MR) merges the real world with digital elements, allowing for interaction with both, unlike Augmented Reality (AR), which only overlays digital data onto the real world without interactive capabilities. Virtual Reality (VR) immerses you in a wholly digital environment, cutting off the physical world. MR's interactive blend offers innovative possibilities for design and development, making it distinct from AR's simple overlay and VR's full immersion.

Read more about AR-MR-VR differences in the article Beyond AR vs. VR: What is the Difference between AR vs. MR vs. VR vs. XR?

Learn how to design your own VR experiences with our course, UX Design for Virtual Reality and how to create your AR experiences with our UX Design for Augmented Reality course.


Which industries benefit most from Mixed Reality applications?

Industries that benefit most from Mixed Reality (MR) applications include healthcare, education, manufacturing, and retail. In healthcare, MR aids in complex surgeries by overlaying crucial patient data onto the surgeon's field of view, enhancing precision. Education leverages MR for immersive learning experiences, making complex subjects more accessible and engaging. Manufacturing uses MR for training, maintenance, and assembly, allowing workers to interact with 3D machinery models for better understanding and efficiency. Retail offers customers a unique way to visualize products in their own space before purchasing, improving satisfaction and reducing returns.

Learn how to design your own VR experiences with our course, UX Design for Virtual Reality and how to create your AR experiences with our UX Design for Augmented Reality course.


What are the design principles for Mixed Reality?

Design for Mixed Reality (MR) involves principles that ensure immersive and intuitive experiences. These include:

  • Spatial awareness: MR applications should understand and respect the user's physical environment and integrate digital content seamlessly into the real world.

  • User comfort: Designers should have ergonomics in mind. This includes minimizing motion sickness through stable and coherent movement and ensuring that interfaces do not overwhelm or fatigue users.

  • Interactivity: MR experiences should offer meaningful interaction and allow users to manipulate virtual objects in a natural and responsive way.

  • Contextual relevance: Digital content should be relevant to the user's physical surroundings and enhance their real-world environment without distraction or out of place.

  • Visual clarity: Ensure virtual elements are easily distinguishable from the real world, with clear visual cues and interfaces guiding user interaction.

  • Adaptability: MR designs should adapt to different user environments and preferences, providing personalized experiences that cater to a wide range of users and use cases.

Learn how to design your own VR experiences with our course, UX Design for Virtual Reality and how to create your AR experiences with our UX Design for Augmented Reality course.


How do UI/UX design approaches differ in MR compared to traditional digital design?

UI/UX design for Mixed Reality (MR) diverges from traditional digital design by focusing on spatial interaction, physical ergonomics, and immersive experiences. In MR, designers must integrate digital elements into the 3D space and make interactions feel natural and intuitive. This involves the consideration of spatial dynamics, user comfort, and visually clear and contextually relevant interfaces that facilitate seamless interaction between the user and the virtual environment.

Learn how to design your own VR experiences with our course, UX Design for Virtual Reality and how to create your AR experiences with our UX Design for Augmented Reality course.


How can designers ensure usability and accessibility in MR applications?

Designers can ensure usability and accessibility in Mixed Reality (MR) applications with these key strategies:

  • Inclusive design: Consider the diverse needs and abilities of users. Include options for different interaction modes (e.g., voice commands, gestures) to accommodate various physical abilities.

  • Ergonomics: Design interactions that minimize physical strain. Remember the comfort of movements and the ergonomic placement of virtual elements to prevent fatigue.

  • Straightforward navigation: Implement intuitive navigation systems. Use spatial cues and easy-to-understand instructions to guide users through MR environments without confusion.

  • Adaptive interfaces: Create interfaces that adapt to individual user needs, including adjustable text sizes, contrast settings, and audio descriptions to cater to users with visual or hearing impairments.

  • Feedback and assistance: Offer immediate and clear feedback on user actions. Provide easily accessible help and support within the MR experience to assist users in learning and troubleshooting.

  • Test with diverse users: Conduct usability testing with diverse users, including those with disabilities. This helps identify and address accessibility barriers in MR applications.

Read the paper Towards the Era of Mixed Reality: Accessibility Meets Three Waves of HCI.

Learn how to design your own VR experiences with our course, UX Design for Virtual Reality and how to create your AR experiences with our UX Design for Augmented Reality course.


What role does artificial intelligence (AI) play in MR experiences?

AI enhances Mixed Reality (MR) and makes it more interactive, personalized, and intelligent. It powers contextual interactions and allows MR systems to understand and respond to user commands and gestures. AI personalizes experiences and adapts to individual user preferences. Through computer vision, AI improves the realism of digital content integration, making MR experiences more immersive. 

Additionally, AI supports accessibility by offering features like real-time translation and speech recognition, and it enriches MR applications with predictive analytics, anticipating user needs for a seamless experience.

Learn how to design your own VR experiences with our course, UX Design for Virtual Reality and how to create your AR experiences with our UX Design for Augmented Reality course.


How vital are 3D modeling skills in MR design?

3D modeling skills are helpful in Mixed Reality (MR) design. They are essential to design immersive environments and realistic objects that enhance user interaction and engagement. These skills enable the creation of detailed, high-quality visuals crucial for convincing MR experiences. 

They are also very useful for prototyping and to explore different ideas within the MR space.

Learn how to design your own VR experiences with our course, UX Design for Virtual Reality and how to create your AR experiences with our UX Design for Augmented Reality course.


How do you conduct user research for MR projects?

To conduct user research for Mixed Reality (MR) projects, focus on understanding user needs through interviews and surveys. Prototype testing is crucial; develop MR prototypes and observe how users interact with them to identify usability issues. Include ethnographic research to see how users might use MR in real-world contexts. 

Ensure accessibility by involving users with diverse abilities. Utilize iterative feedback loops, incorporating user feedback into design refinements. Advanced methods like physiological measurements can also provide insights into user engagement and immersion.

Learn how to design your own VR experiences with our course, UX Design for Virtual Reality and how to create your AR experiences with our UX Design for Augmented Reality course.


Literature on Mixed Reality (MR)

Here’s the entire UX literature on Mixed Reality (MR) by the Interaction Design Foundation, collated in one place:

Learn more about Mixed Reality (MR)

Take a deep dive into Mixed Reality (MR) with our course UX Design for Virtual Reality .

Virtual reality is a multidimensional universe that invites you to bring stories to life, transform digital interactions, educate with impact and create user-centric and unforgettable experiences. This course equips you with the skills and knowledge to embrace the possibilities and navigate the challenges of virtual reality.

UX Design for Virtual Reality is taught by UX expert Frank Spillers, CEO and founder of the renowned UX consultancy Experience Dynamics. Frank is an expert in the field of VR and AR, and has 22 years of UX experience with Fortune 500 clients including Nike, Intel, Microsoft, HP, and Capital One.

In UX Design for Virtual Reality, you’ll learn how to create your own successful VR experience through UX design. Informed by technological developments, UX design principles and VR best practices, explore the entire VR design process, from concept to implementation. Apply your newfound skills and knowledge immediately though practical and enjoyable exercises.  

In lesson 1, you’ll immerse yourself in the origins and future potential of VR and you’ll learn how the core principles of UX design apply to VR. 

In lesson 2, you’ll learn about user research methods, custom-tailored for the intricacies of VR.

In lesson 3, you’ll investigate immersion and presence and explore narrative, motion and sounds as design tools. 

In lesson 4, you’ll delve into interface and interaction design to create your own user-friendly, compelling and comfortable VR experiences.

In lesson 5, you’ll gain insights into prototyping, testing, implementing VR experiences, and conducting thorough evaluations.

After each lesson you’ll have the chance to put what you’ve learned into practice with a practical portfolio exercise. Once you’ve completed the course, you’ll have a case study to add to your UX portfolio. This case study will be pivotal in your transition from 2D designer to 3D designer. 

All open-source articles on Mixed Reality (MR)

Please check the value and try again.

Open Access—Link to us!

We believe in Open Access and the democratization of knowledge. Unfortunately, world-class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.

If you want this to change, , link to us, or join us to help us democratize design knowledge!