Joyce H. D. M. Westerink

Personal Homepage

Joyce H.D.M. Westerink (1960) studied physics and took her Ph.D. in 1991 on the human-oriented topic of perceived image quality. She joined Philips Research and specialized on human perception, emotion and cognition related to consumer products. Written output of her work can be found in some 50 articles in books and international journals and 20 patents and patent applications.

Publication Statistics

Publication period start
Publication period end
Number of co-authors

Number of publications with favourite co-authors

Productive Colleagues
Most productive colleagues in number of publications


Bialoskorski, Leticia S. S., Westerink, Joyce H. D. M., Broek, Egon L. van den (2009): Mood Swings: An Affective Interactive Art System. In: Nijholt, Anton, Reidsma, Dennis, Hondorp, Hendri (eds.) Intelligent Technologies for Interactive Entertainment, Third International Conference - INTETAIN 2009 Amsterdam, The Netherlands, 2009, June 22-24. pp. 181-186.

Bialoskorski, Leticia S. S., Westerink, Joyce H. D. M., Broek, Egon L. van den (2009): Mood Swings: design and evaluation of affective interactive art. In New Review of Hypermedia and Multimedia, 15 (2) pp. 173-191.

Broek, Egon L. van den, Lisý, Viliam, Westerink, Joyce H. D. M., Schut, Marleen H., Tuinenbreijer, Kees (2009): Biosignals as an Advanced Man-Machine Interface. In: Azevedo, Luis, Londral, Ana Rita (eds.) Proceedings of the Second International Conference on Health Informatics - HEALTHINF 2009 Porto, Portugal, 2009, January 14-17. pp. 15-24.

Broek, Egon L. van den, Janssen, Joris H., Westerink, Joyce H. D. M., Healey, Jennifer A. (2009): Prerequisites for Affective Signal Processing (ASP). In: Encarnação, Pedro, Veloso, António (eds.) BIOSIGNALS 2009 - Proceedings of the International Conference on Bio-inspired Systems and Signal Processing Porto, Portugal, 2009, January 14-17. pp. 426-433.

Broek, Egon L. van den, Schut, Marleen H., Westerink, Joyce H. D. M., Tuinenbreijer, Kees (2009): Unobtrusive Sensing of Emotions (USE). In JAISE, 1 (3) pp. 287-299.

Westerink, Joyce H. D. M., Jager, Marko de, Bonants, Ronald, Bruinink, Marijn, Herk, Jan van, Kort, Yvonne de, IJsselsteijn, Wijnand, Smulders, Fren (2007): Motivation in Home Fitnessing: Effects of Immersion and Movement. In: Jacko, Julie A. (eds.) HCI International 2007 - 12th International Conference - Part IV , 2007, . pp. 544-548.

Broek, Egon L. van den, Schut, Marleen H., Westerink, Joyce H. D. M., Herk, Jan van, Tuinenbreijer, Kees (2006): Computing Emotion Awareness Through Facial Electromyography. In: Huang, Thomas S., Sebe, Nicu, Lew, Michael S., Pavlovic, Vladimir, Kolsch, Mathias, Galata, Aphrodite, Kisacanin, Branislav (eds.) Computer Vision in Human-Computer Interaction - ECCV 2006 Workshop on HCI May 13, 2006, Graz, Austria. pp. 52-63.

Eyck, Anke, Geerlings, Kelvin, Karimova, Dina, Meerbeek, Bernt, Wang, Lu, IJsselsteijn, Wijnand, Kort, Yvonne de, Roersma, Michiel, Westerink, Joyce H. D. M. (2006): Effect of a Virtual Coach on Athletes\' Motivation. In: IJsselsteijn, Wijnand, Kort, Yvonne de, Midden, Cees J. H., Eggen, Berry, Hoven, Elise van den (eds.) PERSUASIVE 2006 - Persuasive Technology, First International Conference on Persuasive Technology for Human Well-Being May 18-19, 2006, Eindhoven, The Netherlands. pp. 158-161.

Broek, Egon L. van den, Schut, Marleen H., Tuinenbreijer, Kees, Westerink, Joyce H. D. M. (2006): Communication and Persuasion Technology: Psychophysiology of Emotions and User-Profiling. In: IJsselsteijn, Wijnand, Kort, Yvonne de, Midden, Cees J. H., Eggen, Berry, Hoven, Elise van den (eds.) PERSUASIVE 2006 - Persuasive Technology, First International Conference on Persuasive Technology for Human Well-Being May 18-19, 2006, Eindhoven, The Netherlands. pp. 154-157.

IJsselsteijn, Wijnand, Kort, Yvonne de, Westerink, Joyce H. D. M., Jager, Marko de, Bonants, Ronald (2006): Virtual Fitness: Stimulating Exercise Behavior through Media Technology. In Presence: Teleoperators and Virtual Environments, 15 (6) pp. 688-698.

Westerink, Joyce H. D. M., Bakker, C., Ridder, H. De, Siepe, H. (2002): Human factors in the design of a personalizable EPG: preference-indication strategies, hab. In Behaviour and Information Technology, 21 (4) pp. 249-258.

Westerink, Joyce H. D. M., Majoor, Betty G. M. M., Rama, Mili Docampo (2000): Interacting with Infotainment Applications: Navigation Patterns and Mental Models. In Behaviour and Information Technology, 19 (2) pp. 97-106.

Eggen, J. H., Kaakma, R., Westerink, Joyce H. D. M. (1996): Layered Protocols: Hands-On Experience. In International Journal of Human-Computer Studies, 44 (1) pp. 45-72.

Westerink, Joyce H. D. M.

12.7 Commentary by Joyce H. D. M. Westerink

Kristina Höök has given us an inspiring view of three directions of research targeted at the crossroads of technology and affect, namely (traditional) Affective Computing, Affective Interaction, and Technology as Experience. She emphasizes that each line of research has contributed to the development of applications for various types of users, since they are complementary in their approach. I can only underline this conclusion from my experiences in industrial research. A few aspects in particular I'd like to single out for further discussion.

Let me start with an assumption that is contained in this and many other texts and views on Affective Computing, but never stated explicitly, namely that for any viable application in this domain, you need a measurement of an emotion-relevant signal. This could be a camera signal, as in Affector, movement signals as in eMoto, or physiological signals as in the Affective Diary. Also much of our own effort has been spent in the pursuit of unobtrusive measurement techniques for emotion-related signals, like our skin conductance wristband (Ouwerkerk, 2011). However, to reach the goal of 'making a machine that deliberately ... influences emotion or other affective phenomena', measurement is not strictly needed. A case in point is any TV-set or MP3 player: we use them all the time to change our mood with music, or have a TV-show experience that propels us through a series of emotions. That it works is because people are similar in their reactions to a certain extent and because TV-show directors and music composers are very skilled in creating emotional experiences for the general audience, or for specific target groups. Nevertheless, in our domain of research, everyone tacitly assumes that measurement of emotion-related signals is necessary, and indeed it allows for a further refinement of the affective influencing. Especially for changes away from the average of the crowd, in the direction of adaptation to individuals. This means that ultimately, individual models are not only necessary in the Affective Interactional approach, as Kia Höök proposes, but also in the Affective Computing paradigm. With the emotion-related measurements aboard, we also immediately enter the domain of closed-loop applications (see Van Gerven et al., 2009, Van den Broek, 2011): the emotion-related measurements are interpreted in terms of affect, then a decision is made what actions are applicable (based on present and previous measurements), and these actions are executed, after which a new measurement is done to check the new situation, etc. etc. (see Figure 1). The closed-loop model basically describes that whenever there are measurement data available, they are used to try and achieve a better situation. In this way, one's (affective) state can be guided in a targeted direction. Our Affective Music Player (Janssen et al., 2011, Van der Zwaag et al., 2009), constructed in the best Affective-Computing tradition, can serve as an example: it measures my personal reactions to music, and uses this information to adapt the playlist to direct me (not others) to a certain chosen target mood. All in all, I conclude that emotion-related measurements, individual models, and closed-loop applications are tightly interlinked in any research line in our domain.

Emotional Closed Loop
Figure 12.1: Emotional Closed Loop

The affective closed loop in Figure 1 reserves a substantial part for interpretation of the emotion-related signal. This interpretation can be done by a human, as Kia Höök advocates along the lines of the Affective Interaction paradigm, and this human can either be the person that is measured (e.g. Affective Diary) or someone else (e.g. Affector). In both cases, the measurement information will be used to reflect on the situation measured, and if needed, to take action to change it (making it a closed loop indeed). If the raw emotion-related signals are presented, we will not stand the chance to lose information that is of value to the user, that is true. But on the other hand, this information might also be overwhelming (at least at first) and a user could benefit from help in the form of an interpretation made by an algorithm (in the Affecting Computing tradition). There is no need to make a choice between the two alternatives, we could think of implementing both. For instance, our electronic wristband does show the raw skin conductance/arousal patterns over the course of a day or week, but we can also give the user a discreet buzz (vibration alarm) whenever an algorithm interprets that tension has risen considerably. Of course, Kia Höök points out that it is difficult to make the correct interpretation as context is varying in many applications, and this is underlined by the fact that much of the research effort in affective computing has gone into algorithms deriving affective states from emotion-related signals. Nevertheless, there are options to try and overcome this: a technological approach is by adding additional sensors to monitor the context, like the accelerometer in our wristband that helps us estimate the activity level of the wearer and with that interpret the skin conductance signal. And another way out is by averaging over multiple measurements in varying circumstances to distil an overall effect. This is for instance done in our Affective Music Player, where the mood impact of a single song is modeled by taking the average affective effect (corrected for the Law of Initial Values) of multiple presentations, and this is proven to be good enough to select songs capable of directing one's mood to a certain state. Moreover, neither the raw emotion-related signal, nor its interpretation is presented to the user of our Affective Music Player: (s)he doesn't want to bother and only experiences that (s)he is brought into a different mood. Concluding, we find that both human and algorithm interpretation of emotion-related signals are important ingredients of future applications, and both are capable to deal with context to some extent.

Kia Höök argues that in normal life, emotions are always part of a larger experience, and that it is this larger experience that we need to support with our affective technology, in line with the Technology-as-Experience direction of research. This will certainly broaden the field of applications to include related fields in which emotions play a role. For example, emotions are important in communication, and building up relationships, and it is foreseeable that affective technologies can help (Janssen et al., 2010). It relates to the 'decide on actions' part of the closed loop in Figure 1: what do we want to do with the information gained? Nevertheless, the broadness of possible goals does not preclude that there are also applications that do have the goal to impact affect itself. A case in point is the Affective Music Player described before, which is exactly intended to direct affect, namely the mood. We have also shown (Van der Zwaag et al., 2011) that the optimal, individually selected, music can indeed help to prevent the emotion (or affective state) of anger in the frustrating traffic situations Kia Höök describes. On the other hand, I am not so sure whether consumers are interested in knowing or influencing their emotions. Despite the abundance of emotion-overloaded reality shows on TV, and despite the fact that emotion as a research topic has become fashionable in recent years, the general public still maintains a 'nice for others, not for me' attitude. In my view, this is related to the emotion/female versus rationality/male distinction Kia Höök mentions: The average male continues to see emotions as a female sign of weakness, of which they do not want to be reminded, not even if our measurement technology gives it a more masculine twist. For the females, it is the other way round: They do feel (more) comfortable with mood and emotions and acknowledge their impact on our everyday life, but they are less inclined to deploy masculine technology to alter them. For this reason also, I agree with Kia Höök , that our affective technologies are most likely to be used in applications that target a broader experience than that of affect alone.

To wrap up, let me highlight what I think is the most important message in Kia Höök 's story: That affective technologies will benefit from individual models (not only for human, but also for algorithm interpretation of the emotion-relate signals measured), and that they can be deployed in a wide range of applications extending far beyond the original domain of measuring and influencing affect. I am looking forward to see them appear incorporated in products and applications in the world around us....


  • Janssen, Bailenson, IJsselsteijn, & Westerink (2010), Intimate heartbeats: Opportunities for affective communication technology. IEEE Transactions on Affective Computing, Vol. 1, No 2, 72-80.
  • Janssen, Van den Broek, Westerink, Tune in to your emotions: A robust personalized affective music player, User Modeling and User-Adaptive Interaction, In Press.
  • Ouwerkerk (2011), Unobtrusive Emotions Sensing in Daily Life, in: Sensing Emotions. The Impact of Context on Experience Measurements (2011), Westerink, Krans, Ouwerkerk (eds.). Philips Research Book Series, volume 12, Springer, Dordrecht, The Netherlands, pages 21-39.
  • Van den Broek (2011), Affective Signal Processing, Unraveling the mystery of emotions, Ph.D. Thesis University of Twente.
  • Van der Zwaag, Westerink, Van den Broek (2009), Deploying music characteristics for an affective music player, Proceedings VOLUME I, 2009 International Conference on Affective Computing & Intelligent Interaction, ACII 2009, September 10-12, 2009, Amsterdam, The Netherlands, pages 459-465.
  • Van der Zwaag, Fairclough, Spiridon, Westerink (2011). The impact of music on affect during anger inducing drives. In S. Dmello et al. (Eds.), 4th international conference on affective computing and intelligent interaction. Part I, LNCS 6974 (pp. 407-416). Memphis, Tennessee, USA: Springer, Heidelberg.
  • Van Gerven, Farquhar, Schaefer, Vlek, Geuze, Nijholt, Ramsey, Haselager, Vuurpijl, Gielen and Desain (2009). The brain–computer interface cycle. Journal of Neural Engineering, Vol.6, No.4, 1-10.