Qualitative Research

Your constantly-updated definition of Qualitative Research and collection of topical content and literature

652 shares

What is Qualitative Research?

Qualitative research is the methodology researchers use to gain deep contextual understandings of users via non-numerical means and direct observations. Researchers focus on smaller user samples—e.g., in interviews—to reveal data such as user attitudes, behaviors and hidden factors: insights which guide better designs.

“There are also unknown unknowns, things we don’t know we don’t know.”

— Donald Rumsfeld, Former U.S. Secretary of Defense

See how you can use qualitative research to expose hidden truths about users and iteratively shape better products.

Qualitative Research Focuses on the “Why”

Qualitative research is a subset of user experience (UX) research and user research. By doing qualitative research, you aim to gain narrowly focused but rich information about why users feel and think the ways they do. Unlike its more statistics-oriented “counterpart”, quantitative research, qualitative research can help expose hidden truths about your users’ motivations, hopes, needs, pain points and more to help you keep your project’s focus on track throughout development. UX design professionals do qualitative research typically from early on in projects because—since the insights they reveal can alter product development dramatically—they can prevent costly design errors from arising later. Compare and contrast qualitative with quantitative research here:

	Qualitative research	Quantitative Research
You Aim to Determine	The “why” – to get behind how users approach their problems in their world	The “what”, “where” & “when” of the users’ needs & problems – to help keep your project’s focus on track during development
Methods	Loosely structured (e.g., contextual inquiries) – to learn why users behave how they do & explore their opinions	Highly structured (e.g., surveys) – to gather data about what users do & find patterns in large user groups
Number of Representative Users	Often around 5	Ideally 30+
Level of Contact with Users	More direct & less remote (e.g., usability testing to examine users’ stress levels when they use your design)	Less direct & more remote (e.g., analytics)
Statistically	You need to take great care with handling non-numerical data (e.g., opinions), as your own opinions might influence findings	Reliable – given enough test users

Regarding care with opinions, it’s easy to be subjective about qualitative data, which isn’t as comprehensively analyzable as quantitative data. That’s why design teams also apply quantitative research methods, to reinforce the “why” with the “what”.

Qualitative Research Methods You Can Use to Get Behind Your Users

You have a choice of many methods to help gain the clearest insights into your users’ world – which you might want to complement with quantitative research methods. In iterative processes such as user-centered design, you/your design team would use quantitative research to spot design problems, discover the reasons for these with qualitative research, make changes and then test your improved design on users again. The best method/s to pick will depend on the stage of your project and your objectives. Here are some:

Diary studies – You ask users to document their activities, interactions, etc. over a defined period. This empowers users to deliver context-rich information. Although such studies can be subjective—since users will inevitably be influenced by in-the-moment human issues and their emotions—they’re helpful tools to access generally authentic information.
Interviews
- Structured – You ask users specific questions and analyze their responses with other users’.
- Semi-structured – You have a more free-flowing conversation with users, but still follow a prepared script loosely.
- Ethnographic – You interview users in their own environment to appreciate how they perform tasks and view aspects of tasks.
Usability testing
- Moderated – In-person testing in, e.g., a lab.
- Unmoderated – Users complete tests remotely: e.g., through a video call.
- Guerrilla – “Down-the-hall”/“down-and-dirty” testing on a small group of random users or colleagues.
User observation – You watch users get to grips with your design and note their actions, words and reactions as they attempt to perform tasks.

Qualitative research can be more or less structured depending on the method.

Qualitative Research – How to Get Reliable Results

Some helpful points to remember are:

Participants – Select a number of test users carefully (typically around 5). Observe the finer points such as body language. Remember the difference between what they do and what they say they do.
Moderated vs. unmoderated – You can obtain the richest data from moderated studies, but these can involve considerable time and practice. You can usually conduct unmoderated studies more quickly and cheaply, but you should plan these carefully to ensure instructions are clear, etc.
Types of questions – You’ll learn far more by asking open-ended questions. Avoid leading users’ answers – ask about their experience during, say, the “search for deals” process rather than how easy it was. Try to frame questions so users respond honestly: i.e., so they don’t withhold grievances about their experience because they don’t want to seem impolite. Distorted feedback may also arise in guerrilla testing, as test users may be reluctant to sound negative or to discuss fine details if they lack time.
Location – Think how where users are might affect their performance and responses. If, for example, users’ tasks involve running or traveling on a train, select the appropriate method (e.g., diary studies for them to record aspects of their experience in the environment of a train carriage and the many factors impacting it).

Overall, no single research method can help you answer all your questions. Nevertheless, The Nielsen Norman Group advise that if you only conduct one kind of user research, you should pick qualitative usability testing, since a small sample size can yield many cost- and project-saving insights. Always treat users and their data ethically. Finally, remember the importance of complementing qualitative methods with quantitative ones: You gain insights from the former; you test those using the latter.

Learn More about Qualitative Research

Take our course on User Research to see how to get the most from qualitative research: https://www.interaction-design.org/courses/user-research-methods-and-best-practices

Read about the numerous considerations for qualitative research in this in-depth piece: https://www.invespcro.com/blog/guide-to-conducting-qualitative-usability-studies/

This blog discusses the importance of qualitative research, with tips: https://www.hotjar.com/blog/qualitative-vs-quantitative-user-research

Explore additional insights into qualitative research here: https://measuringu.com/qual-methods/

Literature on Qualitative Research

Here’s the entire UX literature on Qualitative Research by the Interaction Design Foundation, collated in one place:

Best Practices for Qualitative User Research

No matter how big or small your user research project is, if you're going to do user research, you want to make sure you do it right. You don’t want to waste your time doing research that does not truly represent your users’ opinions and behaviors. That's a waste of both your and your research participants’ time, and it’s bad for business. The first step in doing user research right is to understand best practices for the method you are using, and why best practice is, in fact, best practice. If you understand this, you can ensure that your own user research is truly relevant, you can evaluate the validity of other people’s research results, and you can explain the benefits of using a certain method to stakeholders. Here, we'll consider the differences between qualitative and quantitative research and explain what the best practices are when you do qualitative user research.

Quantitative vs. Qualitative User Research

Different user research methods have different guidelines for best practice. User research methods are often divided into quantitative and qualitative methods. In quantitative methods such as surveys, analytics, and formal experiments, the researcher seeks to measure aspects of users and user behavior in a way that can be quantified and used for statistical analysis. Best practice for quantitative methods is to collect objective measurements that are unbiased by the researcher’s hypotheses, presence, and personality. Getting a sample of users who can represent the entire user population in a statistically meaningful way often requires many research participants. Qualitative methods such as interviews, ethnographic field studies, and (to some degree) usability tests are often more exploratory and seek to get a more in-depth understanding of the individual users’ or user group’s experiences, motivations, and everyday lives. In qualitative methods, the researcher accepts that he/she is to some degree a co-creator of research results, so the results are not completely objective and reproducible. The results of qualitative research are usually not expressed numerically, but rather as themes or categories that have occurred during the research.

To illustrate the difference, let’s say you want to study a user group’s exercise habits. You can choose to study these using either quantitative or qualitative research methods. If you use a quantitative method, you could create an online survey and distribute it to a large number of participants. Participants must answer predefined questions about their exercise habits such as “How many hours per week do you exercise?”. If you do your job dutifully, the survey answers can be numerically summarized without bias from your own opinion or personal experience with exercise.

If you use a qualitative method, you could choose to do interviews with a limited number of participants, where you talk to the participants about when, where, and how they exercise. Because the interview is similar to a conversation, your results will depend on how you ask follow-up questions to the participants’ answers – and how you do that will to some extent depend on your personality and your own experience with exercise. Similarly, the results of the interview also, to some extent, depend on a subjective interpretation of what the participant has told you.

Author/Copyright holder: Kyle Cassidy. Copyright terms and licence: CC BY-SA 3.0

You can use both qualitative and quantitative measures to study user behavior like people’s exercise habits. It’s the difference, for example, between how someone typically feels when they go for a morning run vs. how long they do it for.

Here, we'll focus on how to do qualitative user research of a high quality. Thus, we'll only bring up quantitative user research methods to highlight the differences between the types of research; however, before we go any further, we should mention that combining the methods often makes sense. For instance, many usability tests combine quantitative measures of (e.g.) time and errors with qualitative methods such as observations and interviews. Sometimes, researchers use the terms ‘objective’ and ‘subjective’ to describe the data collected through quantitative and qualitative methods. Quite often, in a usability test, you might combine both types of methods in order to interpret results. The term ‘objective’ refers to indisputable observations (facts) made during an experiment.

“A painstaking course in qualitative and quantitative analysis by John Wing gave me an appreciation of the need for, and beauty of, accurate measurement.”
— Paul D. Boyer, American Nobel Prize-winning biochemist, analytical chemist, and professor of chemistry

Let’s picture a scenario to illustrate this—imagine you're testing the interface of a mobile app for the task of uploading a new photo and participants completed the task on average in 5.5 seconds and clicking on the various control options 6.8 times. The term ‘subjective’ refers to the personal opinions and sentiments of the participants, which are individual to them; for example, 90% of users perhaps felt that the clicks required to complete the task were “too many” (we see these words often come up, in fact). To get meaningful results, you need both types of data. Although 5.5 seconds appears to be a short amount of time to complete a task, if the 6.8 clicks required for completing it are “too many” according to your participants, then clearly you need to redesign your interface.

So, quantitative research methods seek to measure an objective reality that exists externally from the researcher, and we have clear guidelines showing how to ensure that the research truly is objective and valid. With qualitative research methods, you cannot measure a completely objective reality that is not in any way influenced by the researcher. Instead, you work with the research participants to explore what the world looks like from their perspectives, or you interpret your own observations of each participant’s reality. This means that it can seem unclear how to perform qualitative research that truly represents a user’s perspective rather than your own biased understanding of his or her perspective.. This can lead to two problematic misunderstandings. One is that qualitative research is of less value than quantitative research, and the other is that qualitative research does not need to be methodical and structured, because observing or talking to people does not require any real preparation or knowledge (which then confirms misunderstanding number one!). Let’s look at each misunderstanding in turn.

Is Qualitative User Research of Less Value Than Quantitative Research?

Sometimes, we think of qualitative user research as something we choose to do when we lack the time and resources to do proper quantitative research. While time and resources certainly play a factor when planning user research, qualitative research has merit in its own right.

Qualitative methods are good when you don’t have a lot of information about the topic you want to research, because they allow you to be explorative. You don’t have to know in advance exactly what questions make the most sense for your research to ask, because you can keep an open mind and change your focus based on what the participant does or says. If we think of the exercise example, a quantitative survey requires that you know enough about your participants and how they exercise to know what questions are the most relevant to ask in your questionnaire. For instance, it only makes sense to ask a participant how often he uses a fitness tracker while exercising if he does—in fact—exercise from time to time and knows what a fitness tracker is. If, on the other hand, you were doing an interview, you could change the direction of your questions if you discovered that the participant does not know what a fitness tracker is.

Author/Copyright holder: frankieleon. Copyright terms and licence: CC BY 2.0

Comparing research methods is a bit like comparing apples and oranges — they have different qualities.

Qualitative methods can give you an in-depth understanding of something in ways that are impossible to reduce to numbers. Qualitative methods can provide you with more detail about a situation or an answer because you can always follow up with new questions or go back to your participants for more detail. So, if your participants do not use fitness trackers, an interview allows you to ask them “Why not?” in a way that is open to all kinds of answers, whereas a survey requires you to list a limited number of possible reasons.

Finally, qualitative research methods do have the advantage in that they often require less time and fewer resources than quantitative research methods do; so, they're easier to fit into—e.g.—an iterative development process.

Does Qualitative User Research Have to be Methodical and Structured?

“The worst thing that contemporary qualitative research can imply is that, in this postmodern age, anything goes. The trick is to produce intelligent, disciplined work on the very edge of the abyss.”
—David Silverman, American professor and author of several books on qualitative research

We stated earlier that quantitative research seeks to collect objective measurements about an external reality without any influence or bias from the researcher and that qualitative research is not objective in the same way, because the researcher’s personality and experience will always, to some extent, influence the results. However, qualitative research should still strive for objectivity, although in a different way from quantitative research. In his classic book Interviews: An Introduction to Qualitative Research Interviewing, professor of psychology Steinar Kvale proposes three ways in which qualitative research can strive for objectivity:

Through freedom from bias
Through intersubjective knowledge
Through being adequate to the object investigated

Each of these tells us something about how we should do qualitative research so that it is methodical and structured. With that in mind, let’s look at them in more detail.

1.Objectivity Through Freedom From Bias

When we do qualitative research, we should strive to be open and unbiased, so our research represents the participants’ perspectives as much as possible. Since no one can start a user research project without any kind of expectations or previous knowledge about the topic of investigation, we need to be consciously aware of any expectations and bias that we have when we do start one; if we aren’t, our expectations and bias might end up controlling us. So to carry out unbiased research, it's likewise vital to be systematic when you analyze interviews—to ensure that you truly explore our results and don’t just search for the results that you expected to get at the beginning of the project.

In the exercise interview example, we could have an expectation that people would be more motivated to exercise if they had an exercise partner, and we might even already have a good idea for an app that could help people find an exercise partner. While good ideas are great, it’s important that we still do the interviews with an open mind about how people feel about exercise partners. It’s important that we still ask open and explorative questions and that we don’t put words in the participant’s mouth by asking leading questions. Similarly, it is important that we don’t simply search for places where participants have stated that they would like an exercise partner when we analyze the results, but that we are open and explorative about what has actually been said. If we are not open to the participant’s point of view, there is no reason to carry out the user research.

Using an extreme example, but one that shows up just how glaringly bad this practice can get, imagine yourself being asked questions by a researcher who is adamant that the world is flat (and pretend you hadn’t been ready to rebuff his view when you entered the interview room). Picture his glee, for instance, when you admit that the sidewalk outside your home is flat, that every pool table you’ve ever played on was perfectly even, and that you’ve never noticed any substantial curvature when looking at the horizon from the ground. At the same time, imagine how he stops writing and frowns when you mention that you have, however, noticed the earth’s curvature from an airplane window... or how he asks you what your eyesight is like when you try to argue the point that everything beyond the horizon isn’t visible, due to the earth’s curvature. That researcher isn’t doing qualitative research—he’s out to confirm his beliefs by selectively picking at what you’re telling him, often lifting it out of context. That’s not really all that scientific, is it?

Author/Copyright holder: St Stev. Copyright terms and licence: CC BY-NC-ND 2.0

It’s important that you don’t become a tour guide who leads your participants to where you think the important sights are, because it will prevent you from discovering anything new. Instead, you should become a co-explorer and see things from your participant’s perspective.

You can encounter the same issues when you do usability tests. If you think you already know where the problems with the interface are, it’s important to keep an open mind so that you don’t overlook problems that you did not expect or accidentally lead the user to find the same problems as you.

2.Objectivity Through Intersubjective Knowledge

Objectivity through intersubjective knowledge refers to an understanding of objectivity as something that multiple people agree on. In qualitative user research, that means ensuring that you are not the only person who interprets your data the way you do. One way of doing this is to have another person who might have a different perspective help you analyze your data. That way, you can compare your observations to see if you agree. You can also choose to share and discuss your observations openly with your team to see if they have a different interpretation of your research. Doing this requires that everyone be prepared to be open and curious about the data. Finally, you can ensure that your research participants agree with your observations and interpretations by continually sharing them with them. In the exercise interview, you could follow up answers that might be interpreted in different ways with a quick summary of what you believe the participant has said – e.g., “You said that you find it difficult to exercise in the evening and that you would rather watch TV, so is it correctly understood that you think evenings are for more relaxing activities, or is it something else?”. You could also choose to share your analysis or report with the participants to see if they have any comments or objections to your findings. Objections do not always mean that your interpretation is wrong, but they add to your understanding of your participants nonetheless.

3.Objectivity Through Being Adequate to the Object Investigated

Research is adequate to the object it investigates when the results show the real nature of the object. Both working in an unbiased way and discussing your interpretations with other people help ensure that your research reflects the nature of the topic you are investigating, but it is also important that you choose the research method that best fits your research topic. Research methods have different attributes and are good for different things, so what results you can get depend on what research method you choose. In the exercise example, the interview reflects the participants’ memories and opinions about how they exercise, but a form of observation of their exercise might reveal a truer picture of how they exercise. After all, we humans have this tendency to ‘misremember’ things, be it because we’re so used to an activity that we stop being mindful of what we do in it, or because we like to picture ourselves as being that bit more adept than we are, or because we might be prone to self-deprecation and humility dilutes the scale of our expertise.

Author/Copyright holder: jesslef. Copyright terms and licence: CC0

If you ask me about my yoga exercise, my answers might be biased by what I wish my skill level and location to be rather than an accurate reflection of what these really look like (and no, unfortunately, this is not a picture of the author doing yoga).

Similarly, we can use interviews to talk to participants about the usability of a user interface, but observing how they interact with the user interface during a usability test (or a combination of test and interview) will probably give a more accurate picture of what they find difficult.

The Take Away

Here, we have focused on the positive attributes of qualitative user research and what it means to be objective in qualitative user research. We can sum up how to be objective when engaging in such research with the following best practices:

Be open and unbiased – both when you meet research participants and when you analyze data from your research.
Be aware of your own preconceptions.
Discuss your interpretation of data with others, or have others help with the data analysis.
Confirm your interpretations with your research participants – either during or following research.
Choose the most appropriate method for the topic you are investigating.

These are best practices. They don’t represent what everyone does when doing qualitative user research, and sometimes circumstances are such that you cannot live up to all of them. But even when you can’t, you should know what to strive for so you know the limitations of your research. That way, you will enable a more scientific transfer of facts, ground your research in firmer reality, and vastly increase the likelihood that your projects end up engaging with real-world users in the way you hope.

References & Where to Learn More

Hero Image: Author/Copyright holder: Dave. Copyright terms and licence: CC BY-NC-ND 2.0

Course: User Research – Methods and Best Practices:
https://www.interaction-design.org/courses/user-research-methods-and-best-practices

Steinar Kvale (1996) InterViews: An Introduction to Qualitative Research Interviewing. SAGE Publications.

Ann Blandford. “Semi-structured qualitative studies.” https://www.interaction-design.org/literature/book/the-encyclopedia-of-human-computer-interaction-2nd-ed/semi-structured-qualitative-studies?

You can read more about the strengths and weaknesses of qualitative research here:

http://www.uxmatters.com/mt/archives/2012/09/strengths-and-weaknesses-of-quantitative-and-qualitative-research.php

If you are interested in learning more about quantitative and qualitative research, you can read:

John W. Creswell (2014) Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 4th Edition. SAGE Publications

Learn more about Qualitative Research

Take a deep dive into Qualitative Research with our course User Research – Methods and Best Practices .

How do you plan to design a product or service that your users will love, if you don't know what they want in the first place? As a user experience designer, you shouldn't leave it to chance to design something outstanding; you should make the effort to understand your users and build on that knowledge from the outset. User research is the way to do this, and it can therefore be thought of as the largest part of user experience design.

In fact, user research is often the first step of a UX design process—after all, you cannot begin to design a product or service without first understanding what your users want! As you gain the skills required, and learn about the best practices in user research, you’ll get first-hand knowledge of your users and be able to design the optimal product—one that’s truly relevant for your users and, subsequently, outperforms your competitors’.

This course will give you insights into the most essential qualitative research methods around and will teach you how to put them into practice in your design work. You’ll also have the opportunity to embark on three practical projects where you can apply what you’ve learned to carry out user research in the real world. You’ll learn details about how to plan user research projects and fit them into your own work processes in a way that maximizes the impact your research can have on your designs. On top of that, you’ll gain practice with different methods that will help you analyze the results of your research and communicate your findings to your clients and stakeholders—workshops, user journeys and personas, just to name a few!

By the end of the course, you’ll have not only a Course Certificate but also three case studies to add to your portfolio. And remember, a portfolio with engaging case studies is invaluable if you are looking to break into a career in UX design or user research!

We believe you should learn from the best, so we’ve gathered a team of experts to help teach this course alongside our own course instructors. That means you’ll meet a new instructor in each of the lessons on research methods who is an expert in their field—we hope you enjoy what they have in store for you!