It’s an important question. Can we measure the user experience and determine whether design is good? Or are we championing a lost cause without knowing it? In order to be taken seriously as UX professionals we need to be able to show that our work has meaning and that means having something tangible as evidence that we have something valuable to bring to our organizations.
Can You Measure Experience?
The simple answer must be yes. We can measure experience because we must measure it. How else do we know if our work is doing the job correctly? We need data to show that we’re heading in the right direction and we need to be able to show what we’ve done so far on our journey. Ideally, that data should be a reflection of the objectives of our business; be they financial, satisfaction, or something less obvious like safety.
The measurement process needs to provide insight into the people who use our products. It should deliver a greater level of understanding than we had before. So, for example, we might measure the bounce rate on our website. It tells us whether users are sticking around more or disappearing faster than before we made a change. Obviously, a single measurement won’t give us a perfect view of whether the design is great or not but you don’t have to use a single measurement to measure something.
Can You Measure Experience Easily?
The truthful answer to this is no. It’s not a simple process to build an accurate set of measurements for our design work. In fact the easier a measure is to make, the more likely it is that we will be measuring something irrelevant. If you go back to the early days of social media work – everything was measured in likes or follows. Marketers have wised up to the fact that these metrics are pretty meaningless now. You have to dig deeper, which requires more effort, if you want something of value.
Can You Measure Experience Accurately?
Probably not. Experiences change over time, we have all abandoned products and applications and websites that we once loved because they’re no longer relevant to us. However, that doesn’t mean that we can’t measure with some degree of accuracy either. It also doesn’t mean that we can’t change our measurements to reflect changes in the business or our users.
You don’t need to promise easy or perfect. You do need, as a UX pro, to develop some sensible metrics which at least reflect what you’re trying to do and your understanding of your users.
How Do We Measure the User Experience?
There’s no one-size fits all approach to this and what you do in your organization will be different to what we do at the Interaction Design Foundation. However, there are some common principles to consider when choosing your metrics:
Don’t Be Afraid to Ask a Single Big Question
Barometers don’t predict the weather but they do give us a decent indication as to what might be coming. Building a barometer for the user experience of your product is a simple way to see which way the wind is blowing.
People aren’t good at remembering fine detail. Most of us, would need several minutes to explain why a film or book is our favourite. If quizzed about the exact events within… we might need much longer and many of us would find it almost impossible to get it right. But all of us can explain how we feel about something when we’ve just used it.
So why not ask a group of users this single question; “How do you feel about our product?” Provide a rating scale or something similar so they can indicate that and you have a barometer. People who feel positively about your offering are more likely to buy or recommend you to others. Those who feel negatively are much less so.
Learn to Seek Objective Data for the Big Question’s Answers
The bigger the sample you draw on, the more likely it is that the big question reflects an accurate picture of how your users feel about your product. You don’t want to stop with that question – there are plenty of ways to find underpinning data to see why people feel like that. Analytics data, eye-tracking data, etc. can all give you a better understanding of why the user experience is great or why it might not be so great.
Think About Meaningful Ways to Observe the Big Question Being Answered
Think about this for a second; does a sell out at the box office when you see a film, automatically make that film a good one? Most people who sat through The Phantom Menace (the first Star Wars prequel) can tell you that it doesn’t. Sales don’t equal experience.
You might have wanted to stand outside the cinema and listen to how people talked about the film. You might have wanted to be in the cinema to listen to people boo at Jar Jar Binks (a character in the movie).
Your choice of measurements should be meaningful to the experience that you provide. Don’t rely on off-the-shelf popular techniques unless they’re truly relevant to your experience.
Take More than One Measurement
If you were to measure the experience of users of a social network that had just gone offline for an 8 hour maintenance routine – you’d find plenty of annoyed users. But would that be an accurate reflection of how they felt about the social network? Of course not.
You need to measure experience regularly and look at trends over time. Don’t fall into the trap of treating measuring as a one-off exercise. Your snapshot in time may not be remotely accurate.
Develop a Dashboard of Metrics
Once you have the right metrics, put them together as a dashboard and have an “at a glance” feel to the current user experience. This will help you communicate the value in what you do more easily to your organization and it will help you take action when the time is right.
No-one said measuring experience was easy or a perfect science. Don’t be afraid to ignore data when it doesn’t appear to be meaningful but do create data and constantly review the way you use it – to give strong indicators of the user experience as a whole.