Virtuality Continuum

Your constantly-updated definition of Virtuality Continuum and collection of videos and articles

77 shares

What is Virtuality Continuum?

The virtuality continuum represents the full spectrum of technological possibilities between the entirely physical world or real environment and the fully digital world or virtual environment. It includes all current technologies that alter reality with computer-generated graphics as well as those yet to be developed.

Virtuality continuum, from left to right: real environment, augmented reality, augmented virtuality and virtual environment. Mixed reality covers all the continuum except the ends.

In a continuum, adjacent parts are almost indistinguishable, but the extremes are very different. Therefore, the exact limits of the various terms are not a hundred percent clear. The term mixed reality covers any environment where the real and virtual objects are combined within a single display. According to this framework, mixed reality covers most of the continuum except for the endpoints. The researchers Paul Milgram and Fumio Kishino first introduced the virtuality continuum or reality-virtuality continuum concept in 1994.

What are the Components of the Virtuality Continuum?

The virtuality continuum, as initially proposed by Milgram and Kishino in 1994, considered only visual displays. Therefore, the different sections within the continuum only take into account the visual aspect of the blending between the physical and digital worlds. This continuum does not take into account sound, smell, haptics or taste.

The virtuality continuum is broken down into four categories:

Real environment: consists solely of real or physical objects. The real environment represents the left end of the virtuality continuum.
Augmented reality: the real world is augmented with digital elements.
Augmented virtuality: the virtual world is augmented by the inclusion of real or physical objects.
Virtual environment: consists solely of digital objects. The virtual environment represents the right end of the virtuality continuum.

It is important not to confuse the virtuality continuum components with the different extended reality (XR) technologies. The virtuality continuum is a theoretical framework. The different sections of the continuum define how many real elements vs. digital elements are displayed, starting from the left end—the real environment—where 100% of what is displayed are real or physical objects and 0% are digital elements versus the right end—the virtual environment—where 100% of the objects displayed are digital and 0% are physical objects.

Limitations of Milgram and Kishino’s Virtuality Continuum

Some researchers have stated that the virtual environment, which is considered the right end of this continuum, should be included within the mixed reality definition. They argue that a fully immersive digital environment is unreachable only considering the visual display. Even if the user only sees a digital environment, they would still have the real-world environment physical constraints; for instance, the user won’t be able to move freely if there is a physical wall in front of them, even if there is no wall in the virtual world they are immersed in. Also, they would be able to taste food or smell a flower.

Many revised versions of Milgram and Kishino’s virtuality continuum have been developed to include the notion of a user and to include all senses. However, there is no new universally accepted standard yet.

Learn More about the Virtuality Continuum

Learn how to design your own XR experiences with our course: How to Design for Augmented and Virtual Reality.

Watch the How To Influence Behavior Through Virtual Reality Narratives on-demand Master Class by VR pioneer Mel Slater.

To see new revisions of the virtuality continuum, read this paper: Skarbez, R., Smith, M., & Whitton, M. (2021). Revisiting Milgram and Kishino's Reality-Virtuality Continuum. Frontiers In Virtual Reality, 2. doi: 10.3389/frvir.2021.647997

Questions related to Virtuality Continuum

How does the Virtuality Continuum relate to AR, VR, and MR?

The Virtuality Continuum spans from the real world to fully immersive virtual environments, placing AR, VR, and MR at various points along this spectrum. AR adds digital elements to the real world, sitting near reality. VR immerses users in a completely virtual environment positioned at the other end. MR merges real and virtual worlds for interactive experiences located in the middle. This concept is pivotal for designers to choose the right technology for desired immersive experiences.

Learn more about AR and VR in our UX Design for Augmented Reality course and our UX Design for Virtual Reality course.

To see new revisions of the virtuality continuum, read this paper: Skarbez, R., Smith, M., & Whitton, M. (2021). Revisiting Milgram and Kishino's Reality-Virtuality Continuum. Frontiers In Virtual Reality, 2. doi: 10.3389/frvir.2021.647997

Why is the Virtuality Continuum important in design?

The Virtuality Continuum is vital in design to guide the choice of technology—AR, VR, or MR—based on the level of user immersion desired. It enables designers to create seamless transitions between real and virtual worlds and enhance user experiences across various applications.

This spectrum helps designers understand innovation possibilities within each technology's constraints.

Learn more about AR and VR in ourUX Design for Augmented Reality course and our UX Design for Virtual Reality course.

How has the concept of the Virtuality Continuum evolved over time?

Since its introduction in 1994, the Virtuality Continuum has evolved from a concept to distinguish between real and virtual environments to a more nuanced framework incorporating augmented reality (AR), virtual reality (VR), and mixed reality (MR). This evolution mirrors technological advancements and enables richer integrations of the digital and physical worlds. Today, it guides the creation of immersive experiences across various fields.

How does the Virtuality Continuum influence UX/UI design?

The Virtuality Continuum profoundly impacts UX/UI design. It demands versatility to design user interactions that range from the integration of digital elements into the physical world (AR) to entirely virtual spaces (VR). Designers must embrace various interaction modalities, including gesture and voice controls, to ensure intuitive user experiences.

The continuum also emphasizes the importance of spatial design, user comfort, and safety, with a need to address challenges like motion sickness in VR (cybersickness) and distraction in AR.

Learn more about AR and VR in our UX Design for Augmented Reality course and our UX Design for Virtual Reality course.

What design principles apply across the Virtuality Continuum?

Design principles that apply across the Virtuality Continuum include:

User-centered design: The user's needs and preferences are at the center of the design process, regardless of the level of virtuality. This involves the consideration of user comfort, accessibility, and intuitive interaction.
Consistency: Consistent design cues, interactions, and navigation across different levels of immersion help users adapt and understand how to interact with the environment, whether it's AR, VR, or somewhere in between.
Clarity: Information and interactions should be clear and easily understandable to prevent confusion, especially as environments become more complex in entirely virtual spaces.
Feedback: Immediate and meaningful feedback for user actions is essential for successful interactive experiences. This helps users understand the impact of their actions within the environment.
Immersive and engaging: Designers should leverage the unique capabilities of AR, VR, and MR to create compelling content.
Safety and comfort: Designers should ensure users' physical safety and comfort, which is paramount, especially as experiences become more immersive. This includes designing to prevent motion sickness and incorporating features that allow users to customize their experience for comfort.

These design principles apply across the Virtuality Continuum.

Learn more about AR and VR in our UX Design for Augmented Reality course and our UX Design for Virtual Reality course.

How does interaction design differ across the Virtuality Continuum?

Interaction design varies significantly across the Virtuality Continuum. Augmented reality (AR) focuses on seamlessly integrating digital elements with the physical world. In mixed reality (MR), the design challenge intensifies and requires spatial considerations for users to interact with virtual objects in real environments. In virtual reality (VR), interaction design transcends physical constraints, allows for innovative interaction paradigms, and demands intuitive and comfortable user experiences.

Across these stages, the core difference lies in how designers bridge the gap between digital content and user environment and must ensure interactions remain natural and engaging regardless of the medium.

Learn more about AR and VR in our UX Design for Augmented Reality course and our UX Design for Virtual Reality course.

What technologies enable experiences along the Virtuality Continuum?

Technologies that enable experiences along the Virtuality Continuum include:

Augmented Reality (AR): Utilizes devices like smartphones, tablets, and AR glasses to overlay digital content onto the real world. Key technologies include GPS for location-based content, cameras and sensors for environment mapping, and AR software development kits (SDKs) like ARKit and ARCore.
Mixed Reality (MR): Combines elements of both the real world and virtual reality, allowing for interaction with and manipulation of virtual objects in real space. MR relies on advanced spatial computing, depth sensing, and holographic display technology, exemplified by devices like the Microsoft HoloLens.
Virtual Reality (VR): Offers fully immersive experiences by transporting users to completely virtual environments. VR headsets like Oculus Rift, HTC Vive, and PlayStation VR are central to this technology, alongside motion tracking sensors and hand controllers for navigation and interaction.
Haptic feedback: Provides tactile responses to interactions with virtual objects, enhancing realism and immersion. Technologies include wearable gloves, suits, and controllers equipped with vibration motors or force feedback mechanisms.
3D audio: Enhances spatial awareness and immersion by simulating sound sources in three-dimensional space, which is crucial for both AR and VR experiences.
Eye tracking: Allows for more intuitive interactions and interface navigation by detecting the user's gaze direction, increasingly integrated into VR headsets.
Gesture recognition: Enables users to interact with virtual environments using natural movements, using cameras and sensors to track hand and body motions.

These technologies work together to create a seamless spectrum of experiences across the Virtuality Continuum.

Learn more about AR and VR in ourUX Design for Augmented Reality course and our UX Design for Virtual Reality course.

What are the limitations of current technology in realizing the Virtuality Continuum?

Several technological limitations currently hinder the realization of the Virtuality Continuum. Hardware discomfort, bulkiness, high latency, and limited interaction fidelity detract from immersive experiences. Additionally, the creation of realistic 3D content is resource-intensive, which poses challenges for widespread adoption.

The design of intuitive interfaces that cater to a broad user base across VR, AR, and MR platforms remains complex, with accessibility for users with disabilities also needing further attention. Moreover, privacy and security concerns regarding the extensive data collected by XR technologies need addressing to gain user trust.

Learn more about AR and VR in our UX Design for Augmented Reality course and our UX Design for Virtual Reality course.

How might the Virtuality Continuum evolve with advancements in technology?

The Virtuality Continuum will evolve significantly as technology advances, leading to more immersive, intuitive, and personalized experiences. Here’s how:

Increased realism: Graphics and processing power advancements will render virtual environments and augmented overlays indistinguishable from the real world.
Enhanced sensory integration: Beyond visual and auditory enhancements, future technologies will integrate tactile feedback, smell, and taste into the Virtuality Continuum.
Seamless AR Integration: As AR technology becomes more sophisticated, digital overlays on the real world will become more seamless and interactive, making AR elements an indistinguishable part of our daily environments.
AI and machine learning: With further advancements in AI, virtual environments will become more responsive and personalized, adapting in real-time to user behaviors and preferences for more engaging experiences.
Brain-Computer Interfaces (BCIs): The integration of BCIs will enable users to control virtual and augmented environments through thought, making interactions more intuitive and eliminating the need for physical controllers.
Ubiquitous and accessible XR: As hardware becomes more lightweight, powerful, and affordable, XR technologies will become more accessible to a broader audience, integrating into various aspects of daily life and multiple industries.
Social and collaborative experiences: Advancements will enable more complex, shared virtual spaces where users can interact with each other in real time across the globe, further blurring the lines between physical and virtual social interactions.

Learn more about AR and VR in our UX Design for Augmented Reality course and our UX Design for Virtual Reality course.

How is AI integrated into Virtuality Continuum experiences?

Artificial Intelligence (AI) significantly enhances the Virtuality Continuum and may offer personalized and interactive experiences in VR, AR, and MR. It brings virtual environments to life with dynamic interactions and responsive objects that mimic real-world behavior.

Through natural language processing and gesture recognition, AI facilitates intuitive communication and control, making experiences more accessible and natural. AI's predictive analytics personalize content and interfaces to individual user preferences, improving engagement.

Additionally, AI-driven accessibility features ensure that these immersive experiences can be tailored to users with diverse needs, making virtuality more inclusive. The integration of AI thus deepens immersion, enriches user interaction, and broadens accessibility across the continuum.

Learn more about AR and VR in ourUX Design for Augmented Reality course and our UX Design for Virtual Reality course.

Literature on Virtuality Continuum

Here’s the entire UX literature on Virtuality Continuum by the Interaction Design Foundation, collated in one place:

Beyond AR vs. VR: What is the Difference between AR vs. MR vs. VR vs. XR?

There is a fair amount of confusion about the differences between the terms virtual reality (VR), augmented reality (AR), mixed reality (MR), and extended reality (XR). While most people have stuck with AR and VR to describe the different technologies, these terms are not enough to fully comprehend the extent of these technologies and harness their potential. That’s why we’re going to explore them in detail now.

According to research by MarketsandMarkets, the market size of all extended reality technologies combined is expected to reach USD 125.2 billion by 2026, at a compound annual growth rate (CAGR) of 30.6% between 2021 and 2026. As a UX designer, you have the chance to contribute to this technology revolution and help define how to take user experience to the next level.

The first step is to define each technology clearly. Once you understand the possibilities each one offers, you’ll be able to adapt your UX design knowledge to these new environments.

Let’s start by looking at the word they all have in common: “reality.” What does “reality” mean in the context of AR, MR, VR and XR?

What is Reality?

Have you ever been in an argument with someone over a specific color? Did it seem dark blue to you and black to the other person? To some extent, the way we process information and construct reality is unique to each one of us—our genetics, previous experiences, etc.—and shapes how we perceive the world.

Humans tend to confuse reality with the physical world and struggle to understand why virtual reality feels so real even when they know it is not.

“We have had people literally run out of the VR room, even though they know that what they are witnessing is not real”
— Mel Slater, Distinguished Scientist and VR pioneer, Institute of Neurosciences of the University of Barcelona

To understand these technologies, you need to understand reality as a construct that each of us makes based on what we perceive from our senses, whether what we perceive comes from the digital or the physical world. The same thing happens when we watch movies. Do you remember the last time you cried or laughed while watching a movie? Were you aware that what you were watching was not real? Yes. Did it make you cry or laugh anyway? If the movie was good, absolutely! Even though you knew you were watching fiction, it felt real at the time. Extended reality technologies add another layer to this phenomenon, which makes the experiences feel even more intense.

“It’s really important to understand we’re not seeing reality. We’re seeing a story that’s being created for us.”
— Patrick Cavanagh, Research Professor, Dartmouth College

When you wear a VR headset, you can feel present in a fully digital environment. The digital information that you perceive through your senses overpowers your reasoning “this is not real.” For your body, at that moment, it is real.

And for all these technologies to work, they all need to feel real. The difference between them is how much they rely on physical or digital elements.

What is the Virtuality Continuum?

The virtuality continuum—continuum being the critical word—contains the full spectrum of possibilities between the entirely physical world or real environment and the fully digital world or virtual environment. In a continuum, adjacent parts are almost indistinguishable, but the extremes are very different. The researchers Paul Milgram and Fumio Kishino first introduced the virtuality continuum or reality-virtuality continuum concept in 1994.

Representation of the virtuality continuum (adapted from Milgram; Kishino, 1994).

The virtuality continuum is a theoretical framework that can help you visualize and understand the differences between the various technologies that exist today and those that are yet to be invented. For example, you can simplify the concept and think of a spectrum of immersion, where one end is low immersion and the other end is high immersion. Then, you can situate the technologies according to the degree of immersion they provide.

Representation of current XR technologies according to the spectrum of immersion.

Sometimes the exact limits of the various technologies are not a hundred percent clear, and they may overlap. However, the critical part is that you understand the region they occupy in the spectrum. This exercise will help you clarify the differences between all these technologies.

What is Extended Reality (XR)?

Extended reality (XR) is an umbrella term that encompasses any sort of technology that alters reality by adding digital elements to the physical or real-world environment by any extent, blurring the line between the physical and the digital world.

XR includes AR, MR, VR, and any technology—even those that have yet to be developed—situated at any point of the virtuality continuum.

The term XR includes AR, MR, VR, and any technology that blends the physical and the digital world.

Bear in mind that XR technologies keep evolving, and their full potential remains to be seen. There are still many things to discover about how users can better interact with them and achieve the best results. For a UX designer specializing in XR experiences, the difficulty lies in the lack of defined standards. An excellent place to start is to keep experiences simple and push the envelope a little bit at a time. In the case of new technologies, usability testing will be your best friend.

What is Augmented Reality (AR)?

Augmented reality (AR) is a technology that allows the superposition of digital elements into the real-world environment. In the AR experience, you can see a composite view of physical or real-world elements and digital elements. While some AR experiences may offer a certain degree of interaction between physical and virtual elements, typically, there is limited to no direct interaction between the digital and physical world components.

AR experiences are close to the physical world end of the virtuality continuum. The ability to overlay digital objects onto the physical world is revolutionizing many industries such as gaming, education, healthcare, and manufacturing. For example, have you ever been to the doctor for a blood test, and the nurse couldn’t find your vein? It can be excruciating. What if AR technology could help with that?

AccuVein uses projection-based AR technology and a laser-based scanner to convert the heat signature of a patient’s veins into an image superimposed on the skin, making the veins easier for clinicians to locate. This AR technology increases the likelihood of a successful first-time injection by 350%.

© AccuVein, Fair-Use (link)

As a UX designer specializing in AR experiences, you have the chance to go beyond the rectangular square of a screen and contribute to a revolution that will continue to change the way we interact with digital products. However, you’ll need to be aware of additional parameters to deliver a good UX that you may not be used to, like physical constraints and safety issues.

Get your free template for “AR Usability Plan Example”

Secure form

Name Please provide your name.

We respect your privacy

Please provide a valid email address.

Get free UX design learning material every week

310,382 designers enjoy our newsletter—sure you don’t want to receive it?

What is Mixed Reality (MR)?

Mixed reality (MR) is a technology that allows not only the superposition of digital elements into the real-world environment but also their interaction. In the MR experience, the user can see and interact with both the digital elements and the physical ones. Therefore, MR experiences get input from the environment and will change according to it.

In MR experiences the user can interact with both digital and physical elements. MR differs from AR—where digital and physical elements don’t interact— and VR—where the physical or real world is completely blocked out.

MR experiences cover the center of the virtuality continuum.

The concept of mixed reality can be a bit difficult to grasp; watch the video below to see an example of an MR experience.

This video shows how MR experiences blend the physical and digital worlds.

As a UX designer specializing in MR, you’ll have to master all the possibilities that MR technologies have to offer. Immersive experiences add a new layer to the user experience and require you to learn continuously and stay up to date to deliver an excellent user experience.

What is Virtual Reality (VR)?

Virtual reality (VR) is a technology that allows the creation of a fully-immersive digital environment. In VR experiences, the physical or real-world environment is entirely blocked out.

VR experiences are located at the fully virtual extreme of the virtuality continuum.

Many people struggle with the fact that VR experiences generate true emotional responses even if we know it’s “fake.” Keep in mind that humans construct reality from the information they receive from their senses, and this is why, even if we are aware that we have a fully digital experience, our bodies respond in the same way. Typically, VR takes advantage of the visual and auditory systems. However, there is an even greater sense of presence and immersion if you add other senses. Have you heard about the “Walk the Plank” challenge? It is a VR experience where you enter an elevator and get out at the top of a skyscraper. Then you’re asked to “walk a plank.” Just with auditory and visual stimulation, many people (especially those who suffer from a fear of heights) can’t do it, even though they are aware that what they see and hear is not real. However, if you add a physical plank to the setup and thus activate the sense of touch, even fewer people will be able to do it.

In this video, you can see how football players try the “Walk the Plank” VR experience with a physical plank and struggle to complete the experience, even though they are aware that the heights are not “real.”

The more coherent information we get through our senses, the more immersive the experience. As a UX designer, you’ll have to consider the different dimensions of the experience and the particularities of VR technology to create the best possible experience for your user.

Get your free template for “Storyboard for VR”

Secure form

Name Please provide your name.

We respect your privacy

Please provide a valid email address.

Get free UX design learning material every week

310,382 designers enjoy our newsletter—sure you don’t want to receive it?

The Take Away

There are many technologies that alter our perception of reality by adding digital elements to the physical world to a greater or lesser extent. All these technologies can be situated at some point of the virtuality continuum according to the extent that the digital elements block the real-world environment and their interactive capabilities. If you have the image of the virtuality continuum in your head, you’ll be able to clearly see the difference between each type of technology.

The most important terms to learn to harness the potential of these new technologies are:

Augmented reality (AR): a view of the real world—physical world—with an overlay of digital elements.
Mixed Reality (MR): a view of the real world—physical world—with an overlay of digital elements where physical and digital elements can interact.
Virtual reality (VR): a fully-immersive digital environment.
Extended reality (XR): an umbrella term that covers all these different technologies, including AR, MR, and VR.

As all these technologies keep evolving and new ones emerge, UX designers need to keep learning and understanding the characteristics of each one to deliver the best possible user experience. If you can clearly distinguish AR, MR and VR and any other XR technology, you’ll be able to design experiences that take advantage of each technology and give an outstanding boost to your UX career!

References and Where to Learn More

Learn how to design your own XR experiences with our course: How to Design for Augmented and Virtual Reality.

Watch the How To Influence Behavior Through Virtual Reality Narratives on-demand Master Class by VR pioneer Mel Slater.

Vi, S. (2022). Exploring User Experience Guidelines for Designing HMD Extended Reality Applications.

The way you see colour depends on what language you speak. (2022). Aina Casaponsa, Panos Athanasopoulos.

UX for XR by Cornel Hillmann.

Learn more about Virtuality Continuum

Take a deep dive into Virtuality Continuum with our course UX Design for Virtual Reality .

Virtual reality is a multidimensional universe that invites you to bring stories to life, transform digital interactions, educate with impact and create user-centric and unforgettable experiences. This course equips you with the skills and knowledge to embrace the possibilities and navigate the challenges of virtual reality.

UX Design for Virtual Reality is taught by UX expert Frank Spillers, CEO and founder of the renowned UX consultancy Experience Dynamics. Frank is an expert in the field of VR and AR, and has 22 years of UX experience with Fortune 500 clients including Nike, Intel, Microsoft, HP, and Capital One.

In UX Design for Virtual Reality, you’ll learn how to create your own successful VR experience through UX design. Informed by technological developments, UX design principles and VR best practices, explore the entire VR design process, from concept to implementation. Apply your newfound skills and knowledge immediately though practical and enjoyable exercises.

In lesson 1, you’ll immerse yourself in the origins and future potential of VR and you’ll learn how the core principles of UX design apply to VR.

In lesson 2, you’ll learn about user research methods, custom-tailored for the intricacies of VR.

In lesson 3, you’ll investigate immersion and presence and explore narrative, motion and sounds as design tools.

In lesson 4, you’ll delve into interface and interaction design to create your own user-friendly, compelling and comfortable VR experiences.

In lesson 5, you’ll gain insights into prototyping, testing, implementing VR experiences, and conducting thorough evaluations.

After each lesson you’ll have the chance to put what you’ve learned into practice with a practical portfolio exercise. Once you’ve completed the course, you’ll have a case study to add to your UX portfolio. This case study will be pivotal in your transition from 2D designer to 3D designer.

All open-source articles on Virtuality Continuum

Beyond AR vs. VR: What is the Difference between AR vs. MR vs. VR vs. XR?

There is a fair amount of confusion about the differences between the terms virtual reality (VR), augmented reality (AR)

901 shares
8 mths ago

Open Access—Link to us!

We believe in Open Access and the democratization of knowledge. Unfortunately, world-class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.

If you want this to change, , link to us, or join us to help us democratize design knowledge!

Virtuality Continuum

What is Virtuality Continuum?

What are the Components of the Virtuality Continuum?

Limitations of Milgram and Kishino’s Virtuality Continuum

Learn More about the Virtuality Continuum

Questions related to Virtuality Continuum

Literature on Virtuality Continuum

Learn more about Virtuality Continuum

All open-source articles on Virtuality Continuum

Beyond AR vs. VR: What is the Difference between AR vs. MR vs. VR vs. XR?

Open Access—Link to us!

Popular related searches

Filters

Privacy Settings

Security

Enhanced Functionality

Referral Program

Error Reporting

Analytics Storage

Google

Advertising Storage

Personalization Storage

Advertising Personalization

User Data

Meta Pixel

Meta CAPI

LinkedIn Insights

LinkedIn CAPI

Google Ads Tag

Share the knowledge!

Cite according to academic standards

New to UX Design? We’re Giving You a Free ebook!