Egon L. van den Broek
M.Sc., Ph.D., Ph.D.
- Personal Homepage
- University of Twente (http://www.utwente.nl/en)
Egon L. van den Broek, MSc (2001) in artificial intelligence,
PhD (2005) in image retrieval, and PhD (2011) in affective
computing. He is consultant and assistant professor (University of Twente and Radboud University Medical Center Nijmegen, The Netherlands). He guided 50+ students, has 150+ scientific publications, 5 patent applications, and various awards. Egon serves in boards of advice, editorial boards, and program committees and is founding editor-in-chief of the Pan Stanford Series in Artificial Intelligence.
- Publication period start
- Publication period end
- Number of co-authors
Number of publications with favourite co-authors
Most productive colleagues in number of publications
Poel, Mannes, Nijboer, Femke, Broek, Egon L. van den, Fairclough, Stephen, Nijholt, Anton (2012): Brain computer interfaces as intelligent sensors for enhancing human-computer interaction. In: Proceedings of the 2012 International Conference on Multimodal Interfaces , 2012, . pp. 379-382. http://dx.doi.org/10.1145/2388676.2388761
Sluis, Frans van der, Broek, Egon L. van den, Dijk, Betsy van (2010): Information Retrieval eXperience (IRX): Towards a Human-Centered Personalized Model of Rel. In: Proceedings of the 2010 IEEE/WIC/ACM International Conference on Web Intelligence and International Conference on Intelligent Agent Technology August 31 - September 3, 2010, Toronto, Canada. pp. 322-325. http://dx.doi.org/10.1109/WI-IAT.2010.222
Meijer, Frank, Broek, Egon L. van den, Schouten, Theo E., Damgrave, Roy G. J., Ridder, Huib de (2010): Synthetic environments as visualization method for product design. In: Rogowitz, Bernice E., Pappas, Thrasyvoulos N. (eds.) Human Vision and Electronic Imaging XV - part of the IS&T-SPIE Electronic Imaging Symposium San Jose, CA, USA, 2010, January 18-21. pp. 752712. http://dx.doi.org/10.1117/12.843691
Schouten, Theo E., Broek, Egon L. van den (2010): Incremental Distance Transforms (IDT). In: 20th International Conference on Pattern Recognition - ICPR 2010 Istanbul, Turkey, 2010, 23-26 August 2010. pp. 237-240. http://dx.doi.org/10.1109/ICPR.2010.67
Sluis, Frans van der, Broek, Egon L. van den (2010): Modeling User Knowledge from Queries: Introducing a Metric for Knowledge. In: An, Aijun, Lingras, Pawan, Petty, Sheila, Huang, Runhe (eds.) Active Media Technology, 6th International Conference - AMT 2010 August 28-30, 2010, Toronto, Canada. pp. 395-402. http://dx.doi.org/10.1007/978-3-642-15470-6_41
Sluis, Frans van der, Dijk, Betsy. van, Broek, Egon L. van den (2010): Aiming for user experience in information retrieval: towards user-centered relevance (UCR). In: Proceedings of the 33rd Annual International ACM SIGIR Conference on Research and Development in Information Retrieval , 2010, . pp. 924. http://doi.acm.org/10.1145/1835449.1835697
Sluis, Frans van der, Dijk, Betsy van, Broek, Egon L. van den (2010): Aiming for user experience in information retrieval: towards user-centered relevance (UCR). In: Crestani, Fabio, Marchand-Maillet, Stéphane, Chen, Hsin-Hsi, Efthimiadis, Efthimis N., Savoy, Jacques (eds.) Proceeding of the 33rd International ACM SIGIR Conference on Research and Development in Information Retrieval - SIGIR 2010 Geneva, Switzerland, 2010, July 19-23. pp. 924. http://doi.acm.org/10.1145/1835449.1835697
Broek, Egon L. van den, Sluis, Frans van der, Schouten, Theo E. (2010): User-Centered Digital Preservation of Multimedia. In ERCIM News, 2010 (80) pp. . http://ercim-news.ercim.eu/en80/special/user-centered-digital-preservation-of-multimedia
Landman, Renske B., Broek, Egon L. van den, Gieskes, José F. B. (2009): Creating Shared Mental Models: The Support of Visual Language. In: Luo, Yuhua (eds.) Cooperative Design, Visualization, and Engineering, 6th International Conference - CDVE 2009 September 20-23, 2009, Luxembourg, Luxembourg. pp. 161-168. http://dx.doi.org/10.1007/978-3-642-04265-2_23
Bialoskorski, Leticia S. S., Westerink, Joyce H. D. M., Broek, Egon L. van den (2009): Mood Swings: An Affective Interactive Art System. In: Nijholt, Anton, Reidsma, Dennis, Hondorp, Hendri (eds.) Intelligent Technologies for Interactive Entertainment, Third International Conference - INTETAIN 2009 Amsterdam, The Netherlands, 2009, June 22-24. pp. 181-186. http://dx.doi.org/10.1007/978-3-642-02315-6_17
Broek, Egon L. van den, Janssen, Joris H., Westerink, Joyce H. D. M., Healey, Jennifer A. (2009): Prerequisites for Affective Signal Processing (ASP). In: Encarnação, Pedro, Veloso, António (eds.) BIOSIGNALS 2009 - Proceedings of the International Conference on Bio-inspired Systems and Signal Processing Porto, Portugal, 2009, January 14-17. pp. 426-433.
Drunen, Annemiek van, Broek, Egon L. van den, Spink, Andrew J., Heffelaar, Tobias (2009): Exploring workload and attention measurements with uLog mouse data. In Behavior Research Methods, 41, ., 0 (41) pp. 868-875. http://brm.psychonomic-journals.org/content/41/3/868
Bialoskorski, Leticia S. S., Westerink, Joyce H. D. M., Broek, Egon L. van den (2009): Mood Swings: design and evaluation of affective interactive art. In New Review of Hypermedia and Multimedia, 15 (2) pp. 173-191. http://www.informaworld.com/10.1080/13614560903131898
Broek, Egon L. van den, Schut, Marleen H., Westerink, Joyce H. D. M., Tuinenbreijer, Kees (2009): Unobtrusive Sensing of Emotions (USE). In JAISE, 1 (3) pp. 287-299. http://dx.doi.org/10.3233/AIS-2009-0034
Broek, Egon L. van den, Lisý, Viliam, Westerink, Joyce H. D. M., Schut, Marleen H., Tuinenbreijer, Kees (2009): Biosignals as an Advanced Man-Machine Interface. In: Azevedo, Luis, Londral, Ana Rita (eds.) Proceedings of the Second International Conference on Health Informatics - HEALTHINF 2009 Porto, Portugal, 2009, January 14-17. pp. 15-24.
Broek, Egon L. van den, Meijer, Frank, Miedema, Jan, Wang, Huaxin, Voort, Mascha C. van der, Vergeest, Joris S. M. (2008): Synthetic Environments for Cooperative Product Design. In: Luo, Yuhua (eds.) Cooperative Design, Visualization, and Engineering, 5th International Conference - CDVE 2008 September 21-25, 2008, Calvià, Mallorca, Spain. pp. 1-10. http://dx.doi.org/10.1007/978-3-540-88011-0_1
Schouten, Theo E., Broek, Egon L. van den (2008): Inverse perspective transformation for video surveillance. In: Bouman, Charles A., Miller, Eric L., Pollak, Ilya (eds.) Computational Imaging VI, part of the IS&T-SPIE Electronic Imaging Symposium San Jose, CA, USA, 2008, January 28-29. pp. 681415. http://dx.doi.org/10.1117/12.767236
Broek, Egon L. van den, Schouten, Theo E., Kisters, P. M. F. (2008): Modeling human color categorization. In Pattern Recognition Letters, 29 (8) pp. 1136-1144. http://dx.doi.org/10.1016/j.patrec.2007.09.006
Broek, Egon L. van den, Jonker, Catholijn M., Sharpanskykh, Alexei, Treur, Jan, Yolum, Pinar (2006): Formal Modeling and Analysis of Organizations. In: Boissier, Olivier, Padget, Julian A., Dignum, Virginia, Lindemann, Gabriela, Matson, Eric T., Ossowski, Sascha, Sichman, Jaime Simão, Vázquez-Salceda, Javier (eds.) AAMAS 2005 International Workshops on Agents, Norms and Institutions for Regulated Multi-Agent Systems, ANIREM 2005, and Organizations in Multi-Agent Systems, OOOP 2005, Utrecht, , 2006, . pp. 18-34. http://dx.doi.org/10.1007/11775331_2
Broek, Egon L. van den, Schut, Marleen H., Westerink, Joyce H. D. M., Herk, Jan van, Tuinenbreijer, Kees (2006): Computing Emotion Awareness Through Facial Electromyography. In: Huang, Thomas S., Sebe, Nicu, Lew, Michael S., Pavlovic, Vladimir, Kolsch, Mathias, Galata, Aphrodite, Kisacanin, Branislav (eds.) Computer Vision in Human-Computer Interaction - ECCV 2006 Workshop on HCI May 13, 2006, Graz, Austria. pp. 52-63. http://dx.doi.org/10.1007/11754336_6
Broek, Egon L. van den, Schut, Marleen H., Tuinenbreijer, Kees, Westerink, Joyce H. D. M. (2006): Communication and Persuasion Technology: Psychophysiology of Emotions and User-Profiling. In: IJsselsteijn, Wijnand, Kort, Yvonne de, Midden, Cees J. H., Eggen, Berry, Hoven, Elise van den (eds.) PERSUASIVE 2006 - Persuasive Technology, First International Conference on Persuasive Technology for Human Well-Being May 18-19, 2006, Eindhoven, The Netherlands. pp. 154-157. http://dx.doi.org/10.1007/11755494_21
Broek, Egon L. van den, Rikxoort, Eva M. van (2005): Parallel-Sequential Texture Analysis. In: Singh, Sameer, Singh, Maneesha, Apte, Chidanand, Perner, Petra (eds.) Pattern Recognition and Image Analysis - Third International Conference on Advances in Pattern Recognition - ICAPR 2005 August 22-25, 2005, Bath, UK. pp. 532-541. http://dx.doi.org/10.1007/11552499_59
Doesburg, Willem A. van, Heuvelink, Annerieke, Broek, Egon L. van den (2005): TACOP: a cognitive agent for a naval training simulation environment. In: Pechoucek, Michal, Steiner, Donald, Thompson, Simon G. (eds.) 4rd International Joint Conference on Autonomous Agents and Multiagent Systems AAMAS 2005 - Special Track for Industrial Applications July 25-29, 2005, Utrecht, The Netherlands. pp. 34-41. http://doi.acm.org/10.1145/1082473.1082801
Doesburg, Willem A. van, Heuvelink, Annerieke, Broek, Egon L. van den (2005): TACOP: a cognitive agent for a naval training simulation environment. In: Dignum, Frank, Dignum, Virginia, Koenig, Sven, Kraus, Sarit, Singh, Munindar P., Wooldridge, Michael (eds.) 4th International Joint Conference on Autonomous Agents and Multiagent Systems AAMAS 2005 July 25-29, 2005, Utrecht, The Netherlands. pp. 1363-1364. http://doi.acm.org/10.1145/1082473.1082774
Broek, Egon L. van den, Jonker, Catholijn M., Sharpanskykh, Alexei, Treur, Jan, Yolum, Pinar (2005): Formal Modeling and Analysis of Organizations. In: Verbeeck, Katja, Tuyls, Karl, Nowé, Ann, Manderick, Bernard, Kuijpers, Bart (eds.) BNAIC 2005 - Proceedings of the Seventeenth Belgium-Netherlands Conference on Artificial Intelligence October 17-18, 2005, Brussels, Belgium. pp. 391-392.
Broek, Egon L. van den, Rikxoort, Eva M. van, Schouten, Theo E. (2005): Human-Centered Object-Based Image Retrieval. In: Singh, Sameer, Singh, Maneesha, Apte, Chidanand, Perner, Petra (eds.) Pattern Recognition and Image Analysis - Third International Conference on Advances in Pattern Recognition - ICAPR 2005 August 22-25, 2005, Bath, UK. pp. 492-501. http://dx.doi.org/10.1007/11552499_55
Doesburg, Willem A. van, Heuvelink, Annerieke, Broek, Egon L. van den (2005): TACOP: A Cognitive Agent for a Naval Training Simulation Environment. In: Verbeeck, Katja, Tuyls, Karl, Nowé, Ann, Manderick, Bernard, Kuijpers, Bart (eds.) BNAIC 2005 - Proceedings of the Seventeenth Belgium-Netherlands Conference on Artificial Intelligence October 17-18, 2005, Brussels, Belgium. pp. 395-396.
Schouten, Theo E., Broek, Egon L. van den (2004): Fast Exact Euclidean Distance (FEED) Transformation. In: ICPR 2004 , 2004, . pp. 594-597. http://doi.ieeecomputersociety.org/10.1109/ICPR.2004.375
Egon L. van den
12.6 Commentary by Egon L. van den Broek
On the bodily expressions of emotion, be aware: More than a century of research!
When thinking about this commentary, ideas popped up and emotions emerged. What to comment on? Kia Höök delivered an excellent chapter. She mentions three angles to approach emotion in technology from (cf. Van den Broek, 2011), namely: affective computing, affective interaction, and technology as experience. In this commentary, I will narrow the focus to affective computing solely. Furthermore, I have also chosen to take a step back and be so bold as to take a methodological perspective with a historical flavor. Why? Well, throughout the years I have discovered more and more literature that touches the core of affective computing but appears to be unknown (e.g., Arnold, 1968; Candland, 1962; Dunbar, 1954). This commentary is founded on two books from a time long before the term affective computing was coined, the 50s and 60s of the previous century. Both books are taken from completely distinct branches of science. Knowledge on science’s history can prevent us, both practitioners and scientists, from repeating mistakes. As such, this commentary touches upon the essence of science itself.
Kia Höök provides a concise overview of emotion in technology. She embraced affective interaction instead of affective computing. In contrast, in this commentary, I have taken the affective computing standpoint. Moreover, Kia Höök has taken a design perspective, where this commentary touches upon and questions the fundaments of emotions in technology. Lessons had been learned but have already been forgotten (Arnold, 1968; Candland, 1962; Dunbar, 1954). Consequently, affective computing tends to reinvent the wheel, at least to some extent. Yes, this is a bold claim, a very bold claim but I hope that after reading this commentary, you as a reader may share my concerns.
In 1954, 5 years before her death, Flanders Dunbar delivered the fourth edition of “Emotions and bodily changes: A survey of literature on psychosomatic interrelationships 1910-1953”. With this impressive volume, she provides an exhaustive and structured review of scientific literature of (roughly) the first half of the previous century on emotions and bodily changes. The volume's title is well chosen and reflects its content nicely. This makes this book undoubtedly valuable for the community of affective computing. However, as far as I know, outside my own work (e.g., Van den Broek, 2011), not a single reference is made to this book in any affective computing article, report, or book. I can only hope that I have missed quite a few ...
Flanders Dunbar starts her book (1954) with:
Where the work of Dunbar illustrates that the origins of affective computing can be traced back to more than a century ago, this quote illustrates that knowledge on the interaction between body and mind was already known more than 25 centuries ago! Let us now identify some core concepts as mentioned in the quote from Dunbar (1954), which are crucial for affective computing.
The old Greek already noted that “the body could not be cured without the mind” (cf. Kia Höök’s chapter). So, both are indisputably related and, hence, in principle, the measurement of emotions should be feasible. This is well illustrated by the remark that “the cure of many diseases is unknown to the physicians of Hellas”, as the Greek culture was devoted to the body and not to the mind. Recent work confirmed this relation. For example, when chronic stress is experienced, similar physiological responses emerge as were present during the stressful events from which the stress originates. If such physiological responses persist, they can cause pervasive and structural chemical imbalances in people’s physiological systems, including their autonomic and central nervous system, their neuroendocrine system, their immune system, and even in their brain (Brosschot, 2010). This brings us to the need for the “knowledge of the whole of things”, a holistic view, perhaps closely related to what Kia Höök denotes as Technology as Experience. Although the previous enumeration of people’s physiological systems can give the impression that we are close to a holistic model, it should be noted that this is in sharp contrast with the current level of science. For example, with (chronic) stress, a thorough understanding is still missing. This can be explained by the complexity of human’s physiological systems, the continuous interaction of all systems, and their integral dynamic nature. However, Brosschot (2010) considers emotions as if these can be isolated and attributed to bodily processes only. I firmly agree with Kia Höök that dynamics beyond the body should also be taken into account. Moreover, as Kia Höök also notes, in relation to computing entities, the interaction consists of much more than emotions; however, the same is true when no computing is involved at all.
25 centuries ago scientists did not apply modern statistics; however, 1 century ago, scientists did already apply statistics; for example, Fisher invented the ANOVA class of statistical models in 1918. This provided the means to test and generalize findings on emotions and bodily changes and boosted the development of behavioral sciences in general (Dunbar, 1954). Moreover, this work fits into Rosalind W. Picard’s definition of affective computing: “… a set of ideas on what I call “affective computing,” computing that relates to, arises from, or influences emotions.” (Picard, 1995, p. 1) At least it fits when taken as the traditional interpretation of computing (i.e., to determine by mathematical means). However, the added value of affective computing would be its engineering component, in particular, signal processing and pattern recognition (Van den Broek, 2011). This would enable machines to sense emotions, reason about them, and perhaps develop them themselves. This would mark a new era of computing.
With the invention of computing machinery, shortly after World War II, a new type of statistics was developed: pattern recognition. In his edited volume “Methodologies of Pattern Recognition” (1969), Satosi Watanabe collected a set of papers that were presented or meant to be presented at the International Conference on Methodologies of Pattern Recognition in 1968. Watanabe started his book with defining pattern recognition:
Watanabe denotes pattern recognition by computers as the “mechanization of the most fundamental human function of (i) perception and (ii) concept formation.” Up to this date human pattern recognition in general is largely unsolved. We do not understand how we, as humans, process affective signals (Van den Broek, 2011). Moreover, the perception of signals and, subsequently, patterns is one thing; their interpretation in terms of emotions is something completely different. This issue refers to content validity; that is, (i) the agreement among experts on the domain of emotions; (ii) the degree to which a (low level) percept adequately represents an emotion; and (iii) the degree to which (a set of) percepts adequately represents all aspects of the emotions under investigation.
The issue of concept formation relates to the process of construct validation, which aims to develop a ground truth (or an ontology or semantic network), constructed around the emotions investigated. Such a framework requires theoretically grounded, observable, operational definitions of all constructs and the relations between them. Such a network aims to provide a verifiable theoretical framework. The lack of such a network is one of the most pregnant problems affective computing is coping with. Kia Höök describes emotions as if we can pinpoint them. Although intuitively this is indeed the case, in practice it proves to be very hard to define emotions (Duffy, 1941; Kleinginna & Kleinginna, 1981).
Par excellence, humans can recognize patterns in noisy environments. Moreover, the ease with which humans adapt to new situations, to new patterns remains striking. Moreover, this is in sharp contrast with the performance of signal processing and pattern recognition algorithms. Often, these perform well in a controlled environment; however, in the “real world” their performance deteriorates (Healey, 2008). This problem refers to the influence of the context on measurements, which is also denoted as ecological validity. Due to a lack of real world research, in general, the ecological validity of research on affective computing is limited and its use often still has to be shown in “real world” practice. However, as Kia Höök illustrates, some nice exceptions to this statement have been presented throughout the last decade.
In 1941, Elizabeth Duffy published her article “An explanation of ‘emotional’ phenomena without the use of the concept ‘emotion’” in which she starts by stating that she considers “… ‘emotion’, as a scientific concept, is worse than useless. … ‘Emotion’ apparently did not represent a separate and distinguishable condition.” Although this statement is 60 years old it is still (or, again) up to date, perhaps even more than ever (cf. Kleinginna & Kleinginna, 1981). Almost fifty years later, in 1990, John T. Cacioppo and Louis G. Tassinary expressed a similar concern; however, they more generally addressed the complexity of psychophysiological relations. These “are conceptualized in terms of their specificity (e.g., one-to-one versus many-to-one) and their generality (e.g., situation or person specific versus cross-situational and pancultural).” (Cacioppo & Tassinary, 1990). They proposed a model, which “yields four classes of psychophysiological relations: (a) outcomes, (b) concomitants, (c) markers, and (d) invariants.” Although Cacioppo and Tassinary (1990) discuss the influence of context, they do not operationalize it; hence, this discussion’s value for affective computing is limited. Nevertheless, articles such as this are food for thought. Regrettably, attempts such as this are rare in the community of affective computing; consequently, the field’s research methods are fragile and a solid theoretical framework is missing (Van den Broek, 2011).
To ensure sufficient advancement, it has been proposed to develop computing entities that respond on their user(s) physiological response(s), without the use of any interpretation of them in terms of emotions or cognitive processes (Tractinsky, 2004). This approach has been shown to be feasible for several areas of application. However, this approach also undermines the position of affective computing itself as a field of research. It suggests that emotion research has to mature further before affective computing can be brought to practice. This would be an honest conclusion but a crude one for the field of affective computing. It implies that affective computing should take a few steps back before making its leap forward. A good starting point for this process would be the hot topics on emotion research that Gross (2010, p. 215) summarized in his article “The future's so bright, I gotta wear shades” (see also Van den Broek, 2011).
Taken together, Kia Höök should be acknowledged for her concise overview of emotion in technology. In her chapter she takes the affective interaction standpoint. In contrast, with this commentary, I have taken an affective computing standpoint. Moreover, Kia Höök has taken a design perspective, where this commentary touches upon the fundaments of emotions in technology. I pose that if anything, affective computing has to learn more about its roots (e.g., Arnold, 1968; Candland, 1962; Dunbar, 1954); then, affective computing can and probably will have a bright future!