David J. Therriault
Dr. Therriault is an Associate Professor in the School of Human Development and Organizational Studies in the College of Education at the University of Florida. He received his undergraduate degree in psychology from the University of New Hampshire and his M. A. and Ph.D. in cognitive psychology from the University of Illinois at Chicago. He was formerly a Postdoctoral Fellow at Florida State University's Psychology Department working with Dr. Rolf Zwaan from 2001-2004.
Dr. Therriault's primary research interests include the representation of text in memory, comprehending time and space in language, the link between attention and intelligence, the use of perceptual symbols in language, creativity and problem solving, and educational issues related to these topics.
- Publication period start
- Publication period end
- Number of co-authors
26.20 Commentary by David J. Therriault
In his chapter on Aesthetic Computing, Paul Fishwick has created a very accessible and compelling argument for applying embodiment to human-computer interfaces. Fishwick’s case is built by examining the different levels of presence in embodied examples. Can you imagine yourself in the landscape of a painting, or virtually manipulating objects while doing math, or becoming a character in a textually described world? Indeed you can, and more importantly there may be learning benefits to doing so. Fishwick provides rich examples of the use of embodiment in software (e.g., a steampunk obesity machine crafted from barrels of water to explain dynamic systems) that provide us with a glimpse into the potential of aesthetic computing. In the balance of this commentary, I briefly discuss the cognitive history and my own experiences researching embodiment, and argue that Fishwick’s insights provides readers with a true glimpse into the future of programing.
Traditional views of cognitive psychology assume that information processing makes use of abstract symbols. Since the 1950’s, the manipulation of abstract/amodal symbols has become the cornerstone of theories examining memory, reading, and thinking. Let us take as an example the psychology of reading. Kintsch’s (1974, 1998) influential research on reading comprehension was built from propositions (i.e., abstract idea units). Kintsch also provided evidence for the psychological reality of propositions, largely shaping our current understanding of what happens in our minds when we read. However, there was always a slice of representation missing from our reading experiences (often referenced as the symbol grounding problem)--a sense of presence or embodiment. How do we explain how propositions acquire meaning, how do we truly experience what we have read? In many ways, Fishwick is tackling the same symbol grounding problem (but within a programming framework).
Embodiment most simply put is attempting to understanding mind through the experiences and perceptions of our bodies (e.g., perceptual symbols, or grounded cognition). Interestingly, research examining how the body can influence our understanding has been most prominent in discourse psychology, specifically text comprehension. Researchers such as Barsalou (1999), Glenberg (1997), Lakoff and Johnson (1999), and Pecher & Zwaan (2005) have all argued that an embodied approach to cognition may have advantages over traditional views of mental representation.
My own introduction to embodiment came during the earlier 2000’s working as a post doc with Rolf Zwaan. It was a wonderful experience, one were we spent the bulk of our time talking about how we would test the links between cognition and action, over good coffee or even better scotch, and see these ideas come to life in the lab. Prior to starting my position, other faculty members had cautioned me that researching grounded cognition was “kooky” and that attention on the topic would soon pass.
Since that time, there have been a host of compelling demonstrations of the psychological reality of embodiment. For example, we know that when listeners hear a story with their eyes closed they move them as if viewing the story in the real world (Spivey, Richardson, Tyler, &Young, 2000), that areas of the brain employed when doing a physical task are the same used when reading about that task (Feldman & Narayanan, 2004), and that even our judgments of morality can influence our perception, such as the perceived level of light in a room (Banerjee, Chatterjee, & Sinha, 2012). Our own research (Kaschak et al. (2002)) provided evidence that the perception of motion makes use of some of the same neural machinery needed to understand a verbal description of motion.
Research exploring an embodied view of cognition continues to flourish but there is currently no unified theory. Most researchers strongly advocate for either a symbolic or embodied view of cognition, but the field is moving away from this dichotomy. For example, Louwerse (2007) argues for exploring the relative contribution of both symbols and embodiment. Fishwick’s chapter represents a truly novel approach to embodiment: applying it to create a better understanding of such things as system dynamics, number sense, or programming. This view is also refreshing, in that, embodied computing isn’t intended to supplant the rich symbols tradition in coding. But embodiment may positively augment how we interact with computers in the future, and who wouldn’t want that?
In sum, I find Fishwick’s chapter a successful venture and one that has implications beyond undergirding (forgive the pun) the study of aesthetic computing. Programming becomes a more attractive domain for folks in my discipline to continue the study of mental representation.