WEBVTT Kind: captions Language: en 00:00:02.960 --> 00:00:05.600 *When and Why to use Analytics* 00:00:10.640 --> 00:00:15.542 Primarily, we're going to need to be using  analytics on existing solutions. 00:00:15.542 --> 00:00:21.948 So, if you're talking about *green field* – which is a brand-new solution, hasn't been built and delivered yet –  00:00:21.948 --> 00:00:26.921 versus *brown field* – which is something that's  already running but perhaps we want to improve it – 00:00:26.921 --> 00:00:30.228 then we're decidedly on the brown field side. 00:00:30.228 --> 00:00:36.214 So, we're looking at existing solutions because it's only existing solutions that can provide us with the analytics. 00:00:36.214 --> 00:00:40.721 If you haven't got an existing solution, you're going to have to use another technique. 00:00:40.721 --> 00:00:46.880 And there are obviously many other techniques, but they're not going to provide you with much in the way of *quantitative data*. 00:00:46.880 --> 00:00:53.982 We do have early-research methods, which we'll  be talking about very briefly as an alternative, 00:00:53.982 --> 00:01:00.529 but predominantly analytics for existing deployed  solutions. 00:01:00.529 --> 00:01:04.897 Having said that, then if you're looking at a rework of an existing site or app, 00:01:04.897 --> 00:01:09.758 then looking at current analytics can tell you a lot about what you might like to address; 00:01:09.758 --> 00:01:15.894 what questions you might like to raise with your team members, stakeholders, users. 00:01:15.894 --> 00:01:18.959 So, those are important considerations. 00:01:18.959 --> 00:01:25.107 A good starting point in organizations or teams with low UX maturity is analytics 00:01:25.147 --> 00:01:31.231 because analytics are easier to sell – to be honest – than qualitative methods. 00:01:31.231 --> 00:01:37.157 If you're new to an organization, if they're only just getting into user experience, 00:01:37.157 --> 00:01:42.160 then trying to persuade colleagues that they should be making important decisions on the basis of 00:01:42.160 --> 00:01:45.915 six to eight qualitative sessions, 00:01:45.915 --> 00:01:49.216 which is typically what we do in the usability lab, 00:01:49.216 --> 00:01:54.599 then you should find by comparison web analytics a much easier thing 00:01:54.599 --> 00:01:57.324 to persuade people with. 00:01:57.324 --> 00:02:01.280 And the other issue particularly relevant to qualitative methods 00:02:01.280 --> 00:02:08.321 is that quantitative methods tend to be very,  very much cheaper – certainly on the scale of data, 00:02:08.321 --> 00:02:15.120 you are often having to talk in terms of  hundreds of dollars or pounds per participant 00:02:15.120 --> 00:02:18.517 in a *qualitative* study, for various expenses; 00:02:18.517 --> 00:02:25.317 whereas a hundred dollars or pounds will get you potentially hundreds or thousands of users. 00:02:25.317 --> 00:02:29.800 And, in fact, if you're talking about platforms like Google Analytics which are free, 00:02:29.834 --> 00:02:33.800 there is no cost other than the cost of understanding and using 00:02:33.840 --> 00:02:38.871 the statistics that you get out; so, obviously  it is very attractive from a cost perspective. 00:02:41.200 --> 00:02:45.633 Some of the things that we'll be needing to  talk about as alternatives to analytics 00:02:45.633 --> 00:02:48.961 or indeed *in addition* to analytics: 00:02:48.961 --> 00:02:53.580 Analytics can often *highlight* areas that we might need to investigate, and 00:02:53.580 --> 00:03:01.591 we would then have to go and consider what alternatives we might use to get to the bottom of that particular problem. 00:03:01.591 --> 00:03:07.407 Obviously, *usability testing* because you'll need to establish *why* users are doing what they're doing. 00:03:07.407 --> 00:03:12.640 You can't know from analytics what users' motivations are. 00:03:12.640 --> 00:03:17.008 All you can know is that they went to *this* page and  then they went to *that* page. 00:03:17.008 --> 00:03:21.029 So, the way to find out if it isn't obvious when you look at the pages 00:03:21.029 --> 00:03:24.680 – like there's something wrong or broken or the text makes no sense – 00:03:24.680 --> 00:03:28.397 is to bring users in and watch them actually doing it, 00:03:28.397 --> 00:03:32.951 or even use remote sessions – watching users doing the thing that has 00:03:32.951 --> 00:03:36.578 come up as a big surprise in your analytics data. 00:03:36.578 --> 00:03:40.391 A/B testing is another relatively low-cost approach. 00:03:40.391 --> 00:03:45.186 It's – again – a *quantitative* one, so we're talking about numbers here. 00:03:45.186 --> 00:03:50.787 And A/B testing, sometimes called *multivariate testing*, is also 00:03:50.787 --> 00:03:53.986 performed using Google Tools often, 00:03:53.986 --> 00:03:56.751 but many, many other tools are available as well; 00:03:56.751 --> 00:04:00.978 and you show users different designs; 00:04:00.978 --> 00:04:06.861 and you get statistics on how people behaved and how many converted, for example. 00:04:06.861 --> 00:04:10.510 And you can then decide "Well, yes, putting that text there 00:04:10.510 --> 00:04:14.658 with this picture over here is better than the other way around." 00:04:14.658 --> 00:04:18.605 People do get carried away with this, though; you  can do this ad nauseam, 00:04:18.605 --> 00:04:22.295 to the point where you're starting to change the background color by 00:04:22.295 --> 00:04:25.841 minute shades to work out which gets you the best result. 00:04:25.841 --> 00:04:29.949 These kinds of results tend to be fairly temporary. 00:04:29.949 --> 00:04:33.664 You get a glitch and then things just settle down afterwards. 00:04:33.664 --> 00:04:38.840 So, mostly in user experience we're  interested in things which actually really 00:04:38.880 --> 00:04:44.640 change the user experience rather than getting  you temporary blips in the analytics results. 00:04:44.640 --> 00:04:48.391 And then, finally, *contextual inquiry* and *early-design  testing*: 00:04:48.391 --> 00:04:52.307 Contextual inquiry is going out and doing research in the field 00:04:52.307 --> 00:04:58.160 – so, with real users doing real things to try to find out how they operate 00:04:58.160 --> 00:05:03.008 in this particular problem domain; what's important to  them; what frustrations they have; 00:05:03.008 --> 00:05:06.190 how they expect a solution to be able to help them. 00:05:06.190 --> 00:05:11.800 And early-design testing – mostly in the web field these days 00:05:11.800 --> 00:05:14.827 but can also be done with software and mobile apps; 00:05:14.827 --> 00:05:19.799 approaches like *tree testing* which simulate a menu hierarchy. 00:05:19.799 --> 00:05:23.196 And you don't actually have to do anything other than put your menu hierarchy 00:05:23.196 --> 00:05:26.555 into a spreadsheet and upload it – it's as simple as that; 00:05:26.555 --> 00:05:30.510 and then give users tasks and see how they get on. 00:05:30.550 --> 00:05:34.252 And you can get some very interesting and  useful results from tree testing. 00:05:34.252 --> 00:05:39.876 And another early-design testing approach is *first-click testing*. 00:05:39.916 --> 00:05:43.306 So, you ask users to do something and you show them a screenshot 00:05:43.346 --> 00:05:48.771 – it doesn't have to be of an existing site; it can be just a design that you're considering – 00:05:48.771 --> 00:05:54.199 and find out where they click, and is where they click helpful to them? Or to you? 00:05:54.199 --> 00:06:00.080 So, these are examples of early-design testing – things that you can do *before* you start building 00:06:00.080 --> 00:06:05.040 a product to work out what the product should look  like or what the general shape or terminology or 00:06:05.040 --> 00:06:08.118 concepts in the product should be. 00:06:08.118 --> 00:06:11.120 And both of these can be used to find out whether you're on the right track. 00:06:11.120 --> 00:06:17.180 I have actually tested solutions for customers where users had no idea 00:06:17.180 --> 00:06:22.726 what the proposition was: "What does this site do?"; "What are they actually trying to sell me?" 00:06:22.726 --> 00:06:29.120 or "What is the purpose of it?" – and it's a bit late to be finding  that out in usability testing towards the end 00:06:29.120 --> 00:06:34.376 of a project, I have to say. And that was indeed  exactly what happened in this particular example 00:06:34.376 --> 00:06:39.562 I'm thinking of. So, doing some of these things  really early on is very important 00:06:39.562 --> 00:06:43.312 and, of course, is totally the opposite of 00:06:43.312 --> 00:06:46.907 trying to use web analytics, which can only be done when you finish. 00:06:46.907 --> 00:06:51.400 So, do bear in mind that you do need  some of these approaches to be sure that you're 00:06:51.400 --> 00:06:57.758 heading in the right direction *long before* you  start building web pages or mobile app screens. 00:07:00.800 --> 00:07:08.398 Understand your organization's *goals* for the interactive solution that you're building. 00:07:08.398 --> 00:07:11.360 Make sure that you know what they're trying to get out of it. 00:07:11.360 --> 00:07:18.028 Speak to stakeholders – stakeholders are people typically within your organization who have a vested interest in your projects. 00:07:18.028 --> 00:07:22.800 So, find out what it's supposed to be  doing; find out why they're rebuilding this site 00:07:22.800 --> 00:07:26.700 or why this mobile app is being substantially  rewritten. 00:07:26.700 --> 00:07:31.246 You need to know that; so, don't just jump in and start looking for interesting numbers. 00:07:31.246 --> 00:07:34.480 It's not necessarily going to be that useful. 00:07:34.480 --> 00:07:37.200 Do know the solutions; become familiar with them. 00:07:37.200 --> 00:07:41.358 Find out how easy it is to use them for the kinds of things 00:07:41.358 --> 00:07:45.547 which your stakeholders or others have told you are important. 00:07:45.547 --> 00:07:51.021 Understand how important journeys through the app or website work. 00:07:51.061 --> 00:07:54.707 And get familiar with the URLs – that's, I'm afraid, something that 00:07:54.707 --> 00:07:58.746 you're going to be seeing a lot of in analytics reports – 00:07:58.746 --> 00:08:02.663 the references for the individual pages or screens, 00:08:02.703 --> 00:08:06.943 and so that you'll understand, when you actually start looking at reports of user journeys, 00:08:06.983 --> 00:08:12.724 what that actually means – "What do all these URLs mean in my actual product?" 00:08:12.764 --> 00:08:16.549 So, you're going to have to do some homework on that front. 00:08:16.549 --> 00:08:20.168 You're also going to have to know the users – you need to speak to the users; 00:08:20.168 --> 00:08:24.040 find out what they think is good and bad about your solutions; 00:08:24.080 --> 00:08:27.921 find out how they think about  this problem domain and how it differs from others 00:08:27.921 --> 00:08:33.834 and what kind of solutions they know work and what  kind of problems they have with typical solutions. 00:08:33.834 --> 00:08:40.121 Also ask stakeholders and colleagues about known  issues and aspirations for current solutions. 00:08:40.121 --> 00:08:44.474 So, you know, if you're in the process of rebuilding  a site or an app, 00:08:44.474 --> 00:08:47.395 *why* – is it just slow-ish? 00:08:47.395 --> 00:08:49.850 Is it just the wrong technology? Maybe. 00:08:49.850 --> 00:08:55.932 Or are there things which were causing real problems in the previous or current version 00:08:55.932 --> 00:08:59.920 and that you're hoping to address those in the rebuild.