WEBVTT

00:00:04.160 --> 00:00:06.360
Ah, well – it's a lovely day here in Tiree.

00:00:06.360 --> 00:00:08.700
I'm looking out the window again.

00:00:08.700 --> 00:00:10.900
But how do we know it's a lovely day?

00:00:10.900 --> 00:00:11.720
Well, I could

00:00:11.720 --> 00:00:15.380
– I won't turn the camera around to show you, because I'll probably never get it pointing back again.

00:00:15.380 --> 00:00:17.520
But I can tell you the Sun's shining.

00:00:17.520 --> 00:00:19.080
It's a blue sky.

00:00:19.080 --> 00:00:21.320
I could go and measure the temperature. It's probably not that warm,

00:00:21.320 --> 00:00:23.400
because it's not early in the year.

00:00:23.400 --> 00:00:27.180
But there's a number of metrics or measures I could use.

00:00:27.400 --> 00:00:29.340
Or perhaps I should go out and talk to people

00:00:29.340 --> 00:00:32.640
and see if there's people sitting out and saying how lovely it is

00:00:32.640 --> 00:00:34.980
or if they're all huddled inside.

00:00:35.400 --> 00:00:39.100
Now, for me, this sunny day seems like a good day.

00:00:39.100 --> 00:00:42.500
But last week, it was the Tiree Wave Classic.

00:00:42.500 --> 00:00:45.180
And there were people windsurfing.

00:00:45.380 --> 00:00:48.040
The best day for them was not a sunny day.

00:00:48.040 --> 00:00:51.200
It was actually quite a dull day, quite a cold day.

00:00:51.200 --> 00:00:53.240
But it was the day with the best wind.

00:00:53.240 --> 00:00:55.600
They didn't care about the Sun;
they cared about the wind.

00:00:55.600 --> 00:00:58.280
So, if I'd asked them, I might have gotten a very different answer

00:00:58.280 --> 00:01:01.120
than if I'd asked a different visitor to the island

00:01:01.120 --> 00:01:04.100
or if you'd asked me about it.

00:01:06.260 --> 00:01:10.400
And it can be almost a conflict between people within HCI.

00:01:10.400 --> 00:01:12.620
It's between those who are more *quantitative*.

00:01:12.620 --> 00:01:15.740
So, when I was talking about the sunny day, I could go and measure the temperature.

00:01:15.740 --> 00:01:17.800
I could measure the wind speed if I was a surfer

00:01:17.800 --> 00:01:19.940
– a whole lot of *numbers* about it –

00:01:20.340 --> 00:01:23.780
as opposed to those who want to take a more *qualitative* approach.

00:01:23.780 --> 00:01:26.140
So, instead of measuring the temperature,

00:01:26.140 --> 00:01:28.880
those are the people who'd want to talk to people to find out

00:01:28.880 --> 00:01:31.980
more about what *it means* to be a good day.

00:01:31.980 --> 00:01:34.520
And we could do the same for an interface. I can look at a phone and say,

00:01:34.520 --> 00:01:37.000
"Okay, how long did it take me to make a phone call?"

00:01:37.000 --> 00:01:39.360
Or I could ask somebody whether they're happy with it:

00:01:39.360 --> 00:01:41.580
What does the phone make them feel about?

00:01:41.580 --> 00:01:44.620
– different kinds of questions to ask.

00:01:45.020 --> 00:01:47.220
Also, you might ask those questions

00:01:47.220 --> 00:01:50.460
– and you can ask this in both a qualitative and quantitative way – in a sealed setting.

00:01:50.460 --> 00:01:54.440
You might take somebody into a room, give them perhaps a new interface to play with.

00:01:54.440 --> 00:01:57.900
You might – so, take the computer, give them a set of tasks to do

00:01:57.900 --> 00:01:59.920
and see how long they take to do it.

00:01:59.920 --> 00:02:02.420
Or what you might do is go out and watch

00:02:02.420 --> 00:02:05.700
people in their real lives using some piece of

00:02:05.700 --> 00:02:08.240
– it might be existing software; it might be new software,

00:02:08.240 --> 00:02:11.020
or just actually observing how they do things.

00:02:11.020 --> 00:02:13.220
There's a bit of overlap here – I should have mentioned at the beginning –

00:02:13.220 --> 00:02:17.000
between *evaluation techniques* and *empirical studies*.

00:02:17.400 --> 00:02:20.460
And you might do empirical studies very, very early on.

00:02:20.460 --> 00:02:23.620
And they share a lot of features with evaluation.

00:02:23.820 --> 00:02:27.860
They're much more likely to be wild studies. 
And there are advantages to each.

00:02:27.860 --> 00:02:30.400
In a laboratory situation, when you've brought people in,

00:02:30.400 --> 00:02:34.760
you can control what they're doing,
you can guide them in particular ways.

00:02:34.760 --> 00:02:37.840
However, that tends to make it

00:02:37.840 --> 00:02:41.000
both more – shall we say – *robust*
that you know what's going on

00:02:41.000 --> 00:02:43.220
but less about the real situation.

00:02:43.220 --> 00:02:46.180
In the real world, it's what people often call "ecologically valid"

00:02:46.180 --> 00:02:48.640
– it's about what they *really* are up to.

00:02:48.640 --> 00:02:52.920
But it is much less controlled, harder to measure – all sorts of things.

00:02:52.920 --> 00:02:54.900
Very often

00:02:54.900 --> 00:02:57.100
– I mean, it's rare or it's rarer

00:02:57.100 --> 00:03:00.520
to find more quantitative in-the-wild studies, but you can find both.

00:03:00.520 --> 00:03:04.910
You can both go out and perhaps do a measure of people outside.

00:03:04.910 --> 00:03:09.400
You might – you know – well, go out on a sunny day and see how many people are smiling.

00:03:09.400 --> 00:03:12.760
Count the number of smiling people each day and use that as your measure

00:03:12.760 --> 00:03:15.120
– a very quantitative measure that's in the wild.

00:03:15.120 --> 00:03:17.620
More often, you might in the wild just go and ask people.

00:03:17.620 --> 00:03:20.027
It's a more qualitative thing.

00:03:21.620 --> 00:03:25.140
Similarly, in the lab, you might do a quantitative thing – some sort of measurement –

00:03:25.140 --> 00:03:29.760
or you might ask something more qualitative – more open-ended.

00:03:31.760 --> 00:03:34.280
Particularly quantitative and qualitative methods,

00:03:34.280 --> 00:03:37.180
which are often seen as very, very different,

00:03:37.180 --> 00:03:40.120
and people will tend to focus on one *or* the other.

00:03:40.280 --> 00:03:43.260
*Personally*, I find that they fit together.

00:03:43.420 --> 00:03:46.200
*Quantitative* methods tend to tell me

00:03:46.200 --> 00:03:49.460
whether something happens and how
common it is to happen,

00:03:49.460 --> 00:03:53.340
whether it's something I actually expect to see in practice commonly.

00:03:53.340 --> 00:03:58.120
*Qualitative* methods – the ones which are more about asking people open-ended questions –

00:03:58.120 --> 00:04:01.540
either to both tell me *new* things that I didn't think about before,

00:04:01.540 --> 00:04:03.880
but also give me the *why* answers

00:04:03.880 --> 00:04:07.020
if I'm trying to understand *why* it is I'm seeing a phenomenon.

00:04:07.020 --> 00:04:09.160
So, the quantitative things – the measurements – say,

00:04:09.160 --> 00:04:13.440
"Yeah, there's something happening. People are finding this feature difficult."

00:04:13.740 --> 00:04:18.860
The qualitative thing helps me understand what it is about it that's difficult and helps me to solve it.

00:04:18.860 --> 00:04:21.680
So, I find they give you *complementary things*

00:04:21.680 --> 00:04:23.720
– they work together.

00:04:24.020 --> 00:04:26.780
The other thing you have to think about when choosing methods

00:04:26.780 --> 00:04:29.900
is about *what's appropriate for the particular situation*.

00:04:29.900 --> 00:04:32.020
And these things don't always work.

00:04:32.020 --> 00:04:35.320
Sometimes, you can't do an in-the-wild experiment.

00:04:35.320 --> 00:04:38.540
If it's about, for instance, systems for people in outer space,

00:04:38.540 --> 00:04:42.380
you're going to have to do it in a laboratory.
You're not going to go up there

00:04:42.380 --> 00:04:45.760
and experiment while people are flying around the planet.

00:04:45.760 --> 00:04:49.660
So, sometimes you can't do one thing or the other. 
It doesn't make sense.

00:04:49.660 --> 00:04:52.980
Similarly, with users – if you're designing
something for

00:04:52.980 --> 00:04:56.120
chief executives of Fortune 100 companies,

00:04:56.120 --> 00:05:00.420
you're not going to get 20 of them in a room and do a user study with them.

00:05:00.420 --> 00:05:04.760
That's not practical. So, you have to understand what's practical, what's reasonable

00:05:04.760 --> 00:05:07.860
and choose your methods accordingly.