33. Action Research

413 shares
Download PDF

Technology and action are two elements that define what it is to be human. It is technology that has made Homo sapiens such a successful species, and it is the actions enabled by technology that will ensure that we continue to be successful à or meet our doom. No technological development has any value without action. Action Research (AR) is all about action and, at the same time, it is also rigorous research. When applied in the context of technology, Action Research is the study of how technology is applied in the real world and the practical consequences of technology-enabled action.

More broadly, Action Research is a generic name used to refer to a set of research approaches that share a few common characteristics. In Action Research the researcher typically tries to provide a service to a research "client", often an organization, and at the same time add to the body of knowledge in a particular domain. In technology-related inquiry, an Action Research study could entail the researcher introducing a new technology in an organization, and at the same time studying the effects of the technology in that organization. The organization can be an insurance company, a church, a distributed film-editing organization, or an online mutual support community of diabetics. The emphasis may be on technology design, empirical evaluation of the effects of one ore more technologies, or both. The emphasis of my discussion in this encyclopedia entry is on the empirical evaluation modality, because that is the modality that is the most closely related to Action Research. One may design a unique technology, but the technology will only become useful when it is employed in practice; Action Research is all about practice.

33.1 Action research

According to most accounts, Action Research originated independently in the U.S.A. and England in the 1940s. In the U.S.A., Action Research emerged from the work of Kurt Lewin on a variety of topics, ranging from child welfare to group dynamics. Lewin was a German-born social psychologist whom many see as the "father" of Action Research. In England, Action Research's origins are not tied to a particular individual, but to an institution: the Tavistock Institute of Human Relations in London. There Action Research was used as a research method to both understand and treat socio-psychological disorders associated with war-related experiences.

To say that the range of areas and ways in which Action Research can be conducted is vast is an understatement. Action Research can be used in many general fields of inquiry such as bilingual education, clinical psychology, sociology, and information systems. It can be conducted in ways that are aligned with most epistemologies, including the positivist, interpretivist, and critical epistemologies. Action Research can have as its unit of analysis the individual, the small group, and even the entire organization. It can be used to address issues as varied as health concerns and environmental problems, and evaluate the impact of things like engineering techniques and business methods.

One of the key characteristics that distinguishes Action Research from most other research approaches, and also constitutes one of its main appeals, is that Action Research aims at both improving the subject of the study (often called the research "client"), and generating knowledge, achieving both at the same time. While this characteristic may seem straightforward enough to easily differentiate Action Research from most other research approaches - such as experimental, survey, and case research - it is not. Action Research will be contrasted with those other research approaches in the next section.

Let us assume, for the sake of illustration, that a survey-based research project was conducted addressing the differential access to the Internet between two main income groups, one high (wealthy) and the other low (poor), in a particular city, where the reasons for the digital divide are unclear. Can that research be considered Action Research if a report based on it is used by the city's government to bridge the gap that characterizes the divide? The answer is "yes", if the research encompasses the city's actions, and possibly a follow-up survey assessment of the impact of those actions. The answer is "no", if the research ended with the analysis of the survey and the publication of the summary report.

Because of Action Research's dual goal, researchers employing it are said to have to satisfy two "masters": the subject (or subjects) of the research, and the research community. Historically, one could argue that it has been harder to satisfy the latter, especially in fields of inquiry where Action Research has not traditionally been used very often, such as in technology-related research.

33.2 Action research vis-a-vis other research approaches

The literature on empirical research methods suggests that three general approaches have accounted for most of the published empirical investigations on the effects of technologies on individuals in organizations. These three general research approaches are: experimental, survey, and case research. The paragraphs below provide a brief description of these three research approaches, and contrast them with Action Research.

33.2.1 Experimental research

This approach has its roots in the scientific practice of biologists, physicists, and medical professionals. Variables are manipulated over time, associated numeric data is collected, and causal or correlation models are tested through inferential statistical analysis procedures. The researcher has a strong control over the environment being observed. Experimental research is typically applied to test models or hypotheses. An example would be a research study in which a group of people used a technology (e.g., an online collaborative writing tool) to perform a group-based task (e.g., collaborative writing of a sales contract), and another group performed the task by interacting face-to-face. The technology group, in this case, would be called the "treatment" group, while the other would be the "control" group. Both groups would be taken randomly from a larger group of people. An example of hypothesis to be tested in this scenario could be that the technology group would write a "better" contract than the no-technology group.

33.2.2 Survey research

This approach originates from the work of economists and sociologists. The researcher typically has a considerable sample to be analyzed, which suggests the use of questionnaires with questions that are easy to be answered and that permit quantitative evaluation "a posteriori". The researcher has little or no control over the environment being observed. As with experimental research, survey research is typically applied to test models or hypotheses. An example would be a research study in which questionnaire-based data were collected from members of 300 new product development teams (i.e., teams that develop novel products such as new toothbrushes, soft drinks or car parts) in different organizations. A hypothesis to be tested in this scenario could be the degree to which a team used a particular type of technology (e.g., an electronic collaboration technology) would be correlated with the speed with which the team completed the new product development task.

33.2.3 Case research

This approach has its roots in general business studies, particularly those using what is frequently referred to as the "Harvard Method". The researcher typically studies one organization, or a small sample of organizations, in depth, but usually does not participate in the organization's day-to-day activities. Cases are analyzed either to build or validate models or theories, typically through data collection in structured and unstructured interviews. Structured interviews usually are based on a predefined set of questions; whereas unstructured interviews tend to flow more freely (a hallway conversation could be an unstructured interview). Case research is believed to be particularly useful to refute models or theories. An example would be a research study in which 3 actual business process redesign teams would be studied in-depth by a researcher, who would conduct structured interviews with the team members on a regular basis (e.g., every two weeks). The teams would be studied over a 6-month period, while they each redesigned a complex process (e.g., the process of assembling a tractor engine) in a separate organization. Each team would use a particular type of technology (e.g., an electronic collaboration technology) to a different degree. The researcher would try to understand how the technology was used by the teams, and how that use influenced the teams' outcomes.

33.2.4 Action research

The main focus of this encyclopedia entry, this approach has its origins in studies of social and workplace issues. Like in case research, the researcher typically studies one organization, or a small sample of organizations, in depth. Unlike in case research, in Action Research the researcher uses participant observation and interviews as key data collection approaches. Although typically applying very little, if any, control on the environment being studied, the researcher is expected to apply some form of "positive" intervention. Typically this will be in the form of a service to the client organization. An example would be a research study in which 3 actual new product development teams would be studied in-depth by a researcher, who would also be a member of all teams. The teams would be studied over a 6-month period, while they each developed a new product in a separate organization. The researcher would be in constant contact with the other team members, which would be used in the generation of participant observation and unstructured interview notes. Each team would use a particular type of technology (e.g., an electronic collaboration technology) to a different degree, and the researcher would try to understand how the technology was used by the teams, and how that use affected the teams' outcomes.

The discussion above is somewhat artificial, since an Action Research study may employ specific techniques that are also used in experimental, survey and case research. Those three research approaches can also be conducted in unorthodox ways, borrowing techniques and ideas from other approaches. Finally, the list of approaches is not comprehensive, and does not address all possible research dimensions; for example, ethnographic research (where the researcher goes "native", becoming part of the environment being studied) may be conducted more like case research than Action Research without being either, strictly speaking. Nevertheless, hopefully the discussion above is helpful in illustrating key characteristics that differentiate Action Research from the most orthodox forms of experimental, survey and case research.

33.3 Action research applied to HCI

No technological development has any value without action, and Action Research is all about action. Few technologies have had a stronger impact on modern society than human-computer interaction (HCI) technologies. This has been particularly true of Internet- and Web-based HCI technologies, but is not limited to those realms. Several seminal HCI technologies predate the emergence of the Internet and the Web.

HCI emerged as a distinct area of research and practice in the early 1980s, at which point it was seen as a sub-field of or specialty area in computer science. Research on HCI has flourished worldwide, especially since the 1990s. This has been motivated by a number of factors, including the development of and experimentation with a variety of HCI tools in the 1980s and 1990s (e.g., workflow coordination and group decision support systems), the emergence of the Internet in the early 1990s, and the explosion in the personal and commercial use of the Web in the mid 1990s (motivated by the development of the first Web browsers). The flourishing of HCI research has generally coincided with the increasing use of Action Research in the study of technology-related issues.

In spite of the fact that HCI research and Action Research have grown in importance together in the last 20 years or so, there is less Action Research applied to HCI inquiry than could be expected. To be sure, there are examples of HCI studies employing Action Research, including some relatively recent ones. Nevertheless, the vast majority of the research on HCI produced in the last 15 years has employed experimental research methods, followed by survey and case research methods. Action Research trails way behind, accounting for probably no more than 5 percent of the total HCI research output.

While there is no "typical" HCI Action Research study, previous research suggests key elements that are likely to be shared by most empirical HCI studies employing Action Research, particularly studies following relaxed versions of the interpretivist and positivist epistemological paradigms. These epistemological paradigms are discussed in more detail in the next section. In these paradigms, research questions or hypotheses are formalized beforehand based on theory, and are either: (a) answered in the Action Research study (research questions); or (b) supported or refuted in the Action Research study (hypotheses). The key elements that are likely to be shared by most such empirical HCI studies employing Action Research can be summarized as follows.

33.3.1 Research questions or hypotheses

These are the practice-based or theory-based research questions that guide the data collection and analysis. In place of research questions, the data collection and analysis may be guided by one or more hypotheses, but this is less common in Action Research than in other research approaches (e.g., experimental research). An example of HCI research question is the following: Does the use of a video-conferencing suite improve the quality of the outcomes generated by new product development teams whose members are geographically dispersed?

33.3.2 HCI technology

This is the technology whose impact on a research client is the main subject of the research. An example of HCI technology is a video-conferencing suite. More than one HCI technology may be studied in an HCI Action Research study.

33.3.3 Practical problems

These are the problems being faced by an individual, group or organization; which the HCI Action Research study aims at solving, at least in part. Some prefer to refer to practical problems by using a more "benign" term, namely that of "opportunities for improvement". An example of practical problem is the following: New products need to be constantly developed by geographically dispersed teams, but the transportation and lodging costs associated with bringing team members together currently prevent more than two thirds of the needed teams from being conducted.

33.3.4 Research client

This is the individual, group, or organization whose practical problem (or problems) is supposed to be solved by the HCI Action Research study. An example of a research client would be an automobile manufacturer with several factories in Europe, the U.S.A. and South America.

One of the most straightforward and efficient ways of conducting an HCI Action Research study is to collect data using the same instrument (e.g., a questionnaire) at two key points in time, namely before and after the introduction of the HCI technology, and add to that other types of data collection such as participant observation and unstructured interview notes.

The technology introduction would more often than not have the goal of solving an important practical problem being faced by the research client. Usually, it is a good idea to collect quantitative as well as qualitative data before and after the technology introduction. The quantitative data can be used in simple non-parametric comparison of means analyses, whereas the qualitative data can be used to find explanations and underlying causes for the patterns observed in the data.

In spite of its simplicity, the type of research design discussed above is relatively rare in Action Research. It is much more common to see published examples of Action Research in which only qualitative data is collected, mostly during and after the Action Research intervention (e.g., HCI technology introduction). This may be a problem, as the researcher may end up "drowning" in a "sea of data" from which multiple models can be derived. Research project planning may also be hampered; research usually is conducted on a fixed budget and within a limited timeframe.

Quite often Action Research studies are conducted through multiple iterations of what has become known as the "Action Research cycle", rather than a "one shot," non-cyclical research design. That is, quite often Action Research studies are longitudinal, as opposed to cross-sectional, although this is not always the case. Cyclic Action Research is, generally speaking, a good thing. The Action Research cycle involves the identification of practical problems, the solution of those problems, and reflection on the part of the researcher, which is then followed again by the identification and solution of problems, new reflection, and so on. Conducting multiple iterations of the Action Research cycle tends to add validity and credibility to the research findings, as repeated observations in various iterations lead to the identification of clear patterns.

The most widely referenced version of the "Action Research cycle" has been proposed by Gerald Susman and Roger Evered, in 1978 (see Figure 1). It comprises five stages: diagnosing, action planning, action taking, evaluating, and specifying learning. The diagnosing stage, where the cycle begins, involves the identification of an improvement opportunity or a general problem to be solved at the client organization. The following stage, action planning, involves the consideration of alternative courses of action to attain the improvement or solve the problem identified. The action taking stage involves the selection and implementation of one of the courses of action considered in the previous stage. The evaluating stage involves the study of the outcomes of the selected course of action. Finally, the specifying learning stage involves reviewing the outcomes of the evaluating stage and, based on this, knowledge building in the form of a model describing the situation under study. In studies that involve several iterations of the Action Research cycle, the specifying learning stage is followed by the diagnosing stage of a subsequent cycle, which can take place in the same organizational context or in a different one (e.g., a different department or company).

Susman and Evered's (1978) Action Research cycle

Author/Copyright holder: Unknown (pending investigation). Copyright terms and licence: Unknown (pending investigation). See section "Exceptions" in the copyright terms below.

Figure 33.1: Susman and Evered's (1978) Action Research cycle

33.4 Action research and epistemologies

Epistemologies can be seen as systems of concepts, rules, and criteria that find acceptance among a community of researchers as a basis for the generation of what that community of researchers sees as valid knowledge. By far the most widely subscribed epistemology among empirical HCI researchers is positivism.

Research that conforms to positivist inquiry tenets usually departs from a set of theoretical propositions or hypotheses, and aims at testing those propositions or hypotheses through the analysis of empirical data. Also, in positivist research the data is usually (although not always) of a quantitative nature. The research methods employed in positivist studies often reflect those traditionally used by natural scientists.

Experimental research is usually conducted in a positivist way. Therefore, our previous example of experimental research can be invoked to illustrate typical positivist research: a research study in which a group of people used a technology (e.g., an online collaborative writing tool) to perform a group-based task (e.g., collaborative writing of a sales contract), and another group performed the task by interacting face-to-face. Again, an example of hypothesis could be that the technology group would write a "better" contract than the no-technology group.

In the interpretivist (a.k.a. interpretive) epistemological tradition the investigator usually departs from research questions to be answered through data collection and analysis, not hypotheses. Interpretivism is frequently referred to as anti-positivism, in an epistemological sense. Often theoretical models are built based on the analysis of qualitative data, and deep reflection by the researcher on the study subjects' and the researcher's own subjective views of the world. The critical epistemology is more similar to the interpretivist than to the positivist epistemology, but here the investigator typically wants to empower the subjects of the research so that they can correct a situation that is perceived as unfair and/or oppressive.

One issue that has led to some debate among Action Research scholars in the past is whether Action Research can be conducted in ways that are consistent with different epistemologies, including the positivist epistemology. The debate has been motivated by the fact that Action Research has typically been used in research studies that do not conform very well with traditional positivist standards, and that are better aligned with what many would see as the interpretivist and critical epistemologies. In fact, one could argue that today there is resistance in scholarly Action Research circles against the notion of positivist Action Research, and that resistance can be quite strong within specific Action Research communities. Examples are the Action Research communities that conduct investigations according to two main Action Research frameworks: John Heron's cooperative inquiry, and Paulo Freire's participatory action research.

The above scenario creates a problematic situation; what one could reasonably call a vicious circle. Empirical HCI research, where the behavioral impact of HCI technologies is evaluated, has traditionally been and continues being overwhelmingly positivist in nature. There are practical reasons for this status quo: positivist research is easier to plan and conduct, and the quantitative form of the results is usually very clear-cut.

However, because of this positivist empirical HCI research tradition, researchers who try to employ Action Research to study HCI are hampered not only once but twice in their efforts. On one hand, they have to justify using Action Research in a positivist manner, which is likely to meet with opposition from Action Research scholars. On the other hand, they have to sell the notion that Action Research can be useful for HCI research, which is likely to be seen with suspicion by established HCI researchers, who often view Action Research is a "soft" investigative approach.

This is an unfortunate state of affairs, because Action Research can address a key problem with past HCI research, namely its lack of "real world appeal". In other words, since past empirical HCI research has been by and large based on laboratory experiments with students, it has been difficult for practicing managers and professionals to personally relate to and benefit from many of the findings resulting from that research. Those managers frequently see research conducted in controlled laboratory settings as leading to findings that carry little external validity.

Can Action Research be successfully employed in empirical HCI research? The answer to this question is certainly "yes", and there are several examples of that (see, e.g., Kock, 1998; Kock et al., 2000; Kock and DeLuca, 2007). Can it be done in a positivist way? Well, based on some various examples, the answer to this follow-up question also seems to be "yes" (see, e.g., Davis, 2001; DeLuca et al., 2006). The key here is perhaps to be creative so that certain characteristics of Action Research are used to add strengths to more traditional HCI inquiry.

A widely overlooked strength of Action Research comes from the observation that it exposes the researcher to significantly more data (although relatively sparse) than more focused research approaches (e.g., experimental and survey research). One could adopt Karl Popper's view that exposure to a large body of data, whose analysis does not uncover evidence that contradicts a hypothesis, is in fact "evidence" in support of the hypothesis. Given this, Action Research could be seen as quite adequate for positivist HCI inquiry. This is a modified positivist stance, which has been called post-positivist, and sometimes neo-positivist.

33.5 Future directions

The applied nature of Action Research has tremendous appeal. At the same time, it usually makes Action Research much more difficult to implement in practice. This creates difficulties for the use of Action Research in academic circles. A lot of research is done in the context of doctoral dissertations. And a good doctoral dissertation is a finished doctoral dissertation. Adopting a research approach that makes research more difficult to complete is not the best way to get a doctoral degree.

So is Action Research doomed then? Not really, due to two interesting phenomena: (a) increasingly, research is being conducted with funding from outside academia; and (b) more and more research is being done outside academia.

External funding agencies often value practical research. The reason is that their constituents are particularly interested in research whose results have very practical applications, be they organizational or community stakeholders. While this is especially true of non-government organizations, it also applies to government agencies. Examples are the National Science Foundation in the U.S.A. and the European Commission in Europe. Both have recently ramped up research funding for HCI-related studies. The European Commission tends to favor Action Research-like research more than the National Science Foundation, but the latter is catching up quickly (Kock and Antunes, 2007).

Outside academia, research practical applications is becoming widespread. Companies like Google and Microsoft employ a significant proportion of their workforce in research and development divisions. Many of those researchers focus on Action Research-like HCI research. Notable pioneers in this respect are Jonathan Grudin at Microsoft, and Craig Nevill-Manning at Google. The careers of these two technology professionals and researchers, whom I have been following, provide a glimpse of the future of Action Research-like research in HCI. I have had many discussions with Jonathan about HCI issues, especially the connection with Darwinian evolution. Craig and I were doctoral students at about the same time at the University of Waikato, in New Zealand.

33.6 Where to learn more

33.6.1 Special issues of journals

HCI researchers often identify themselves with broader research communities. One such community is that of information systems researchers. With that in mind, a couple of special issues on information systems Action Research are worth checking, as they provide exemplars of Action Research studies that can be used as a basis for HCI researchers interested in employing Action Research. The first is the special issue on Action Research in information systems published in the journal Information Technology & People in 2001 (volume 14, number 1). The second is the special issue on Action Research in information systems published in the journal MIS Quarterly in 2004 (volume 28, number 3).

The special issue published in the journal Information Technology & People in 2001 was the first special issue ever on Action Research in information systems. The issue contained six articles. Three of those are conceptual, in the sense that they are aimed at providing insights on how to conduct information systems Action Research. The other three articles are empirical, in the sense that they discuss actual information systems Action Research studies and their results. Of the empirical articles, two addressed HCI issues in the context of group support systems investigations. Those articles are: GSS and Action Research in the Hong Kong Police by Robert Davison; and Action Learning and Groupware Technologies: A Case Study in GSS Facilitation Research, by Pak Yoong, Brent Gallupe.

The special issue on Action Research in information systems published in the journal MIS Quarterly in 2004 was aimed at providing a set of exemplars of information systems Action Research studies of an empirical nature. As such, all of the six articles published in this special issue report on empirical studies that employed Action Research to investigate information systems phenomena. None of the articles seems to be aimed at squarely addressing HCI issues, although at least two of the articles address issues that are likely to be directly relevant for HCI researchers. Those articles are: Informating the Clan: Controlling Physicians Costs and Outcomes, by Rajiv Kohli and William J. Kettinger; and Small Business Growth and Internal Transparency: The Role of Information Systems, by Christopher T. Street and Darren B. Meister.

33.6.2 Online Resources

33.7 Appendix: Doctoral action research on HCI

A great deal of the research output produced every year, and published in academic journals, is the direct result of doctoral research investigations. The field of HCI is no exception to this general rule, so it is a good idea to contemplate the pros and cons of conducting doctoral research on HCI issues employing Action Research.

One of the best ways to get a doctoral degree is to test an existing theory and, based on the results, add to or refine the theory. It is not very wise to try to develop a new theory as part of one's doctoral research project. Many doctoral students are prone to think of their research projects as likely to lead to theoretical insights that will change the world in a major way. Nevertheless, it is unlikely that doctoral students' ideas will have the same impact as Darwin's theory of evolution, or Einstein's theory of relativity; which were not developed as part of Darwin's or Einstein's doctoral work, by the way.

Conducting research aimed at testing an existing theory is quite likely to lead someone's research to fall into the general epistemological category called positivist research, discussed earlier. And, as previously argued, there is nothing wrong with conducting Action Research in a positivist manner. However, one problem may arise. Traditionally, Action Research has not been seen as the best approach for the conduct of positivist inquiry. In fact, Action Research has been widely viewed as an ideal approach to create new theories grounded in action-oriented projects, particularly in organizational settings.

So, what is a doctoral student to do when contemplating using Action Research to investigate HCI issues? First, it would be advisable to have a look at recent examples of doctoral dissertations that accomplished this. Second, it is highly advisable to design the research in a positivist (or post-positivist) manner, following some of the suggestions provided earlier. Finally, the student should make sure that the doctoral dissertation committee members are receptive to the idea of Action Research being conducted in a positivist manner. After all, those committee members are ultimately the ones that will decide whether the degree is granted or not. Those who employ and/or subscribe to the Action Research approach known as "canonical Action Research" are likely to be so inclined, and others who are not can be educated based on publications discussing what is known as "canonical Action Research".

Nevertheless, a number of obstacles await those doctoral students who decide to employ Action Research to study HCI issues. Those students who opt for studying HCI effects in organizational settings, for example, will face the challenge of finding one or more organizations willing to work with them. Even when organizational support is achieved, there is the danger that the support will be withdrawn before enough research data is collected. Finally, a multitude of political issues will have to be dealt with. There may be suspicion and opposition by employees, if support is obtained from the organization's management first, without much grass-roots consultation. Dealing with such political issues is likely to ensure that the doctoral student employing Action Research will have to spend significantly more time and effort with the research project than doctoral students employing more traditional HCI research approaches (e.g., experimental research).

Yet, the enhanced credibility of the research findings, the excitement of being "part of the action", and the personal satisfaction that comes from helping improve people's lives while conducting an Action Research study, may be well worth all the extra effort. This is why I conducted my doctoral research, many years ago, using Action Research to address HCI issues.

33.8 References

Carroll, John M. (2013). Human Computer Interaction - brief intro. Retrieved 4 November 2013 from [URL to be defined - in press]

Carroll, John M. (ed.) (2003): HCI Models, Theories, and Frameworks. San Francisco, Morgan Kaufman Publishers

Davis, Erica (2001): Applying a Personalized System of Instruction to Internet-Based Training. Doctoral Dissertation. Philadelphia, USA, Temple University

DeLuca, Dorothea, Gasson, Susan and Kock, Ned (2006): Adaptations that Virtual Teams Make so that Complex Tasks Can be Performed Using Simple e-Collaboration Technologies. In International Journal of e-Collaboration, 2 (3) pp. 64-90

Kock, Ned (2004): The Three Threats of Action Research: A Discussion of Methodological Antidotes in the Context of an Information Systems Study. In Decision Support Systems, 37 (2) pp. 265-286

Kock, Ned (1998): Can Communication Medium Limitations Foster Better Group Outcomes? An Action Research Study. In Information and Management, 34 (5) pp. 295-305

Kock, Ned (ed.) (2008b): Darwinian Perspectives on Electronic Communication: Special Issue of the Journal IEEE Transactions on Professional Communication. IEEE Computer Society Press

Kock, Ned (ed.) (2006): Information Systems Action Research: An Applied View of Emerging Concepts and Methods. New York, USA, Springer

Kock, Ned (ed.) (2008a): Encyclopedia of E-Collaboration. Information Science Reference

Kock, Ned and Antunes, Pedro (2007): Government Funding of E-Collaboration Research in the European Union: A Comparison with the United States Model. In International Journal of e-Collaboration, 3 (2) pp. 36-47

Kock, Ned, Auspitz, Camile and King, Brad (2000): Using the Web to Enable Industry-University Collaboration: An Action Research Study of a Course Partnership. In Informing Science, 3 (3) pp. 157-167

33.8 Commentary by Susanne Boedker

Let me start this comment by saying that I'm in total agreement with the comments made by Bjørn Erik Munkvold in his commentary (see below). Instead of repeating his arguments I will dig a little deeper into some details in the arguments and history outlined by Kock.

Back in the beginning of the 1990s Bannon (1991) identified the move from first wave, cognitivist HCI to the second wave, which was theoretically more diverse, focusing more on the active role of users. In his writing about the move "from human factors to human actors" he pointed both towards a different way of thinking about users, as active human beings, and towards the roles that these users may play in design and research.

In the second wave, focus was on groups working with a collection of ap­plications. Theory focused on work settings and interac­tion within well-established communities of practice. Rigid guidelines, for­mal methods, and systematic testing were mostly aban­doned for proactive methods such as a variety of participa­tory design workshops, prototyping and contextual inquiries (Bødker 2008). With this picture in mind, and at this quite general level, I have difficulties recognizing the picture of the past 20 years of HCI painted by Kock: "the vast majority of the research on HCI produced in the last 15 years has employed experimental research methods followed by survey and case research methods." Part of my problem in this is perhaps my own lack of ease of making such clear distinctions.

As pointed out in Bjørn Erik Munkvold's commentary, an important source of inspiration for the second wave of HCI was a number of Scandinavian projects, applying participatory design with future users of technology (in the workplace) and action research with systems developers to improve their ways of working. Without going into the intrinsic details about the genealogy of these projects and their methodological roots, it feels safe to say that they were in turn inspired by Nordic work researchers like Thomas Mathiesen and Bjørn Gustavsen, who in turn were inspired by Kurt Lewin. When looking at Kock's historical line-up, however, these projects approached action research largely by taking an active stance against a positivistic view. This was in parts a result of significant influence from Marxist thinking, but also of influence by activists like Paulo Freire, who gets mentioned by Kock.

In Finland a parallel development happened leading to Engeström's focus on work development research (1987), where Lewin's thinking has been important together with activity theory. This development had further strong parallels in Germany. Not being a researcher in the history of ideas I will leave this discussion, except for saying that in my view it has been through this lineage that action research has had the strongest impact on HCI, in all modesty e.g. through my own work.

Kock's characteristics of the current state of affairs in HCI does not match with Bannon's characteristics of the second wave of HCI, and with the current development into the third wave (Bødker 2006, Harrison et al 2007). As a matter of fact, lab studies have played a very insignificant role in conferences like ACM CHI or Interact over the past 20 years, and it is not many lab papers I see in my everyday life as editor of ACM ToCHI and IJHCS. Given that, Kock may well be right that what gets applied are experimental research methods followed by survey and case research methods, but they are applied in much more mixed manners than suggested by Kock.  This is largely due to the quite significant impact of qualitative, interpretative, and even radical humanist methods (Burell & Morgan 1979) in second wave HCI (e.g. ethnomethodology through Suchman (1987), activity theory through Kaptelinin, Nardi, Kuutti and Bødker). Actually one could probably argue that there is too little systematic research and too little theory in contemporary HCI in general, and with this statement I strongly sympathize with Kock's attempt to more rigorously outline the methods and outcomes of different research approaches in HCI.

I am however confused by Kock's comment about HCI's current lack of real-world appeal and its accompanying difficulties for the impact of action research. Having just come back from the DIS 2010 conference where Yvonne Rogers talked about HCI's "turn to the wild" while numerous researchers talked about the role of design in HCI research, I am convinced that Kock is wrong in this analysis:

At a time when use contexts and application types are broadened, and intermixed across private and public spheres; where technology spreads from the workplace to our homes and everyday lives and culture; where new elements of human life are included in the human-computer interaction such as culture, emotion and experience, there is more than ever a need for action-based approaches and interventions in HCI. But more than ever, these need to take an active stance against positivistic thinking.

References

  • Bannon, Liam. (1991) From Human Factors to Human Actors: The role of psychology and human-computer interaction studies in systems design. In: Greenbaum, Joan & Kyng, Morten. (Eds.) (1991) Design at Work. Cooperative Design of Computer Systems. Lawrence Erlbaum Associates, pp. 25-44.
  • Burrell, G., & Morgan, G. Sociological Paradigms and Organizational Analysis, Heinemann, 1979, 1-37
  • Bødker, Susanne. (1991). Through the Interface. A Human Activity Approach to User Interface Design. Mahwah, NJ: Lawrence Erlbaum Associates, Inc.
  • Bødker, Susanne. (2006). When second wave HCI meets third wave challenges. ACM International Conference Proceeding Series; Vol. 189. Proceedings of the 4th NordiCHI, ACM, pp. 1 – 8.
  • Engeström, Y. (1987). Learning by expansion. Helsinki: Orienta Konsultit.
  • Steve Harrison, Deborah Tatar, and Phoebe Sengers. "The Three Paradigms of HCI." alt.chi 2007.
  • Kaptelinin, Victor. & Nardi, Bonnie. (2006) Acting with technology, Cambridge, MA: MIT press.
  • Kuutti, K., The Concept of Activity as a Basic Unit for CSCW Research., in Proceedings of the 2nd ECSCW., L.J. Bannon, M. Robinson, and K. Schmidt, Editors. 1991, Kluwer: Amsterdam. p. 249-264.
  • Munkvold, Bjørn Erik [commentary]
  • Suchman, Lucy. (1987). Plans and Situated Actions, Cambridge UK: Cambridge University Press.

33.9 Commentary by Bjørn Erik Munkvold

Ned Kock's chapter offers an insightful introduction to the concept of Action Research and how this could be related to the field of HCI. The chapter reflects Ned's broad experience with this type of research, and will be useful to anyone looking for an overview of the characteristics and challenges of applying this research approach in the context of HCI. The chapter also points to sources for learning more about the topic. In this brief commentary I will address some issues from the chapter that I think could be elaborated more, to bring out the key characteristics of Action Research as well as some nuances in our understanding of this.

Ned's point of departure for the chapter is that to understand Action Research requires being able to differentiate this from other types of research. As presented in the chapter, a focus on action and practice are key elements of action research projects. However, by themselves these elements are necessary but not sufficient foci for a study to be classified as action research. For example, much of the research on IT-enabled change in organizations (conducted as case studies or surveys) focuses on changes to practice through technology-related actions. Yet, this research will typically not be classified as action research, as long as the researcher role is merely that of an observer. Thus, the role of the researcher(s) is a key distinguishing element in Action Research, in that an 'action researcher' would normally be closely involved in specifying and bringing about the targeted change. This is done in close collaboration with the 'client', bringing me to the next point of the collaborative nature of Action Research. As specified in the Action Research cycle of Susman and Evered (1978) in the chapter, the steps in the process are conducted in collaboration between the research and the client. Even in the final stage of specifying learning, while the researcher may be responsible also this step would normally involve mutual learning between the researcher and clients/practitioners. The focus of joint collaboration in the research process is thus a key distinguishing feature of Action Research, emphasized in the early literature defining this research (e.g. Rapoport, 1970; Susman and Evered, 1978). This does not go against what is presented in the chapter, but intends to emphasize more the important role of the researcher as 'change agent' and the collaborative model of action research.

Related to the application of action research in HCI, the chapter raises an issue of a possible mismatch between the positivist epistemology of HCI and the non-positivist nature of action research. The argument made is that since action research could also be done in a positivist manner, it should be 'acceptable' for the HCI community. My view on this issue is somewhat different. First, as discussed by Carol (2009), the HCI community today is very broad and from my European/Scandinavian perspective I do not experience a similar strong dominance of the positivist perspective. The part of the HCI community focusing on participatory design here serves as an example. Second, I belong to those who regard the nature of Action Research to be based on an interpretivist perspective, finding it difficult to reconcile the engaged role of the action researcher with a positivist epistemology. This of course does not preclude combining qualitative and quantitative methods, as advocated in the chapter. Thus, rather than giving Action Research a positivist framing, I would argue that this research approach should be made attractive to the HCI community on its merits of increased relevance through building and evaluating IT artifacts in close interaction with practice.

An increasing focus on action research could also be seen as a response to Van de Ven's (2007) call for engaged scholarship, defined as "a participative form of research for obtaining the different perspectives of key stakeholders (researchers, users, clients, sponsors, and practitioners) in studying complex problems" (p. 9). Action Research has also been focused lately as part of the increasing interest in design science (Iivary, 2007), and the debate on similarities and differences between these two approaches. A recent contribution is the Action Design Research method suggested by Sein et al. (2010), that combines the design science focus of building innovative IT artifacts with the Action Research focus of learning from the intervention when applying the artifact for solving a problem in practice.

The chapter appendix addresses an important issue about the challenges of conducting doctoral research based on Action Research. The potential obstacles discussed resonate well with experiences discussed by Jesper Simonsen from his action research projects in the Scandinavian context (Simonsen, 2009). He suggests that some of these challenges of Action Research can be mitigated by only assigning the PhD student with specific parts of the Action Research project, and having the supervisor and fellow senior researchers co-participate in the project. In any case, Action Research projects will often be undertaken by a research team rather than a single researcher.

As pointed out in the chapter, Action Research offers a great potential for HCI research. This is exemplified by the strong tradition of Action Research in the Scandinavian IS community over the years (Mathiassen and Nielsen, 2008). In addition to improving business practices through IT, Action Research also holds promise as a basis for research on how to develop and apply technology to bring about positive change in important societal areas such as healthcare and environment.

References

  • Carroll, John M. (2009). Encyclopedia entry on Human Computer Interaction (HCI). Retrieved 13 July 2010 from Interaction-Design.org: https://www.interaction-design.org/encyclopedia/human_computer_interaction_hci.html
  • Iivari, Juhani (2007): A paradigmatic analysis of information systems as a design science. In Scandinavian Journal of Information Systems, 19 (2), pp. 39-64.
  • Mathiassen, Lars and Nielsen, Peter A. (2008): Engaged Scholarship in IS Research: The Scandinavian Case. In Scandinavian Journal of Information Systems, 20 (2), pp. 3-20.
  • Rapoport, Robert N. (1970): Three dilemmas of action research. Human Relations, 23, pp. 499-513.
  • Sein, Maung K., Henfridsson, Ola, Purao, Sandeep, Rossi, Matti and Lindgren, Rikard (2010): Action Design Research. Forthcoming in MIS Quarterly
  • Simonsen, Jesper (2010): A Concern for Engaged Scholarship: The Challenges for Action Research Projects. In Scandinavian Journal of Information Systems, 21 (2), pp. 111-128.
  • Susman, Gerald I and Evered, Roger D. (1978): An assessment of the scientific merits of action research. Administrative Sciences Quarterly, 23, pp. 582-603.
  • Van de Ven, Andrew H. (2007): Engaged Scholarship: A Guide for Organizational and Social Research, Oxford, Oxford University Press.

33.10 Commentary by Richard Baskerville

Ned Kock's article, "Action Research: Its Nature and Relationship to Human-Computer Interaction", which appears in the HCI Encyclopedia of Interaction-Design.org, provides an excellent and stimulating introduction and overview into Action Research for the HCI researcher. There is little with which experts may disagree, except over the lacunae that are normally driven by space limitations.

Our moniker includes the word "research" and this part of the term leads scholars to regard the technique in the context of other research approaches, as Kock does in his comparison with experiments, surveys, etc.  But action research's origins are more humble. It originated primarily as a means for researching a practical problem, usually something along the lines of, "How do we get our organizational members to stop behaving badly?" Action research is not only for academic research or even for research-and-development. It is also a means by which an organization can undertake self-therapy to make itself better. Such non-academic action research may not spill out top publications, but it can cure an organization's ills. Little wonder it is a useful approach for consultants seeking to help those in need of organizational development (Baskerville & Wood-Harper, 1998).

In one sense, action research is such a painfully simple idea that the many layers of epistemology, methodology, and infrastructure might seem superfluous.  Why bother? We bother because this rigor is necessary to those researchers (such as the academics) that plan to capture the general knowledge gained in the action research and introduce this into the scientific literature. It is made necessary if we are to produce more than just the practical solution, but also the "credentialed" knowledge that can be proven valid and reliable. Kock is on-target with his pluralist discussion of action research epistemology. After all, philosophy is not an attribute of the research method; it is an attribute of the philosopher who uses it (Baskerville, 1991).

In his appendix on Doctoral Action Research, Kock introduces us to some of the issues confronting student use of the approach. For me, these issues spring from the limited control that an action researcher can exercise over his/her own research process. Most action research, and especially HCI action research, is sociological in nature. When the context is social, "control" is problematic. Most commonly, action research projects unfold under the shared control by researchers and subjects. Under such shared decision-making, theories that guide the project may change, plans for action may change, and participants may change, all beyond the unilateral control of the researcher. Doctoral researchers (and their directors) have to understand that action research is often problem-centric rather than theory- or question-centric (Avison, Baskerville, & Myers, 2001).

For information systems, action research is increasingly contextualized within Van de Ven's (2007) notion of "Engaged Scholarship" (along with design science), as noted in Munkvold's commentary on Kock's article (in this volume). The original conceptualization of engaged research was as a fifth form of scholarship beyond four other forms: discovery, teaching, application, and integration (Boyer, 1996). In other words, because engaged research is different from discovery research, it is seen as useful only in its humblest sense; as a means for practical problem solving. Van de Ven's work is important because it integrates the scholarship of discovery with engaged scholarship. This integration fits well with the use of action research by scholars.

HCI is often concerned with design. Action research is sometimes mistakenly conflated with design science research  (Järvinen, 2007). After all, "design" (as a verb) is "action". But the paradigms underlying these two research approaches are quite different (Iivari, 2007). While action research is grounded in social psychology (hence centering the human), design science research is grounded in engineering economics (hence centering the artifact, Simon, 1996). For HCI researchers, it is important to recognize when the two approaches are used in isolation, and when these are being integrated. When integrated as a process, the underlying epistemology has to be integrated with the care that Kock's work suggests.

References

  • Avison, D., Baskerville, R., & Myers, M. (2001). Controlling action research projects. Information Technology and People, 14(1), 28-45.
  • Baskerville, R. (1991). Practitioner Autonomy and the Bias of Methods and Tools. In H.-E. Nissen, H. K. Klein & R. A. Hirschheim (Eds.), Information Systems research: Contemporary Approaches & Emergent Traditions (pp. 673-698). Amsterdam: North-Holland.
  • Baskerville, R., & Wood-Harper, A. T. (1998). Diversity in Information Systems Action Research Methods. European Journal of Information Systems, 7(2), 90-107.
  • Boyer, E. H. (1996). The scholarship of engagement. Journal of Public Service and Outreach, 1, 11-20.
  • Iivari, J. (2007). A paradigmatic analysis of Information Systems as a design science. Scandinavian Journal of Information Systems, 19(2), 39-63.
  • Järvinen, P. (2007). Action Research is Similar to Design Science Quality and Quantity 41(1), 37-54.
  • Simon, H. A. (1996). The Sciences of the Artificial (3rd ed.). Cambridge, Mass.: MIT Press.
  • Van de Ven, A. H. (2007). Engaged Scholarship: A Guide for Organizational and Social Research. Oxford: Oxford University Press.
413 shares
Download PDF

Open Access—Link to us!

We believe in Open Access and the democratization of knowledge. Unfortunately, world-class educational materials such as this page are normally hidden behind paywalls or in expensive textbooks.

If you want this to change, , link to us, or join us to help us democratize design knowledge!