Publication statistics

Pub. period:1990-2011
Pub. count:36
Number of co-authors:80



Co-authors

Number of publications with 3 favourite co-authors:

Billy Lam:
Ian Stavness:
Vincent Tsao:

 

 

Productive colleagues

Sidney Fels's 3 most productive colleagues in number of publications:

Saul Greenberg:140
Bill Buxton:78
Norihiro Hagita:46
 
 
 

Upcoming Courses

go to course
User-Centred Design - Module 2
Starts tomorrow LAST CALL!
go to course
User Experience: The Beginner's Guide
Starts tomorrow LAST CALL!
 
 

Featured chapter

Marc Hassenzahl explains the fascinating concept of User Experience and Experience Design. Commentaries by Don Norman, Eric Reiss, Mark Blythe, and Whitney Hess

User Experience and Experience Design !

 
 

Our Latest Books

 
 
The Social Design of Technical Systems: Building technologies for communities. 2nd Edition
by Brian Whitworth and Adnan Ahmad
start reading
 
 
 
 
Gamification at Work: Designing Engaging Business Software
by Janaki Mythily Kumar and Mario Herger
start reading
 
 
 
 
The Social Design of Technical Systems: Building technologies for communities
by Brian Whitworth and Adnan Ahmad
start reading
 
 
 
 
The Encyclopedia of Human-Computer Interaction, 2nd Ed.
by Mads Soegaard and Rikke Friis Dam
start reading
 
 

Sidney Fels

Ph.D

Picture of Sidney Fels.
Update pic
Has also published under the name of:
"S. Sidney Fels" and "Sidney S. Fels"

Personal Homepage:
http://www.ece.ubc.ca/~ssfels/

Current place of employment:
The University of British Columbia

Sidney has been in the department of Electrical and Computer Engineering at the University of British Columbia since 1998. Sidney received his Ph. D. and M.Sc. in Computer Science at the University of Toronto in 1994 and 1990 respectively. He received his B.A.Sc. in Electrical Engineering at the University of Waterloo in 1988. He was a visiting research at ATR Media Integration & Communications Research Laboratories in Kyoto, Japan from 1996 to 1997. He also worked at Virtual Technologies Inc. in Palo Alto, CA developing the GesturePlus system and the CyberServer in 1995. His research interests are in human-computer interaction, neural networks, intelligent agents and interactive arts. Some of his research projects include Glove-TalkII, Glove-Talk, Iamascope, InvenTcl, French Surfaces, Sound Sculpting and the context-aware mobile assistant project (CMAP). Using the Glove-TalkII system a person could speak with their hands. The device was built to be a virtual artificial vocal tract. The person using the system wore special gloves and used a foot pedal. These devices controlled a model of a vocal tract so that a person could "play" speech much as a musician plays music. His collaborative work on sound sculpting is an extension of this idea to create musical instruments. The Iamascope is an interactive artwork which explores the relationship between people and machines. In Iamascope the participant takes the place of the coloured piece of glass inside the kaleidoscope. The participant's movements cause a cascade of imagery and music to engulf them. His other artwork includes the Forklift Ballet, Video Cubism, PlesioPhone and Waking Dream. He currently heads the Human Communication Technologies (HCT) Laboratory and is the Director of the Media and Graphics Interdisciplinary Centre (MAGIC) at the University of British Columbia.

 

Publications by Sidney Fels (bibliography)

 what's this?
2011
 
Edit | Del

d'Alessandro, Nicolas, Pritchard, Robert, Wang, Johnty and Fels, Sidney (2011): Ubiquitous voice synthesis: interactive manipulation of speech and singing on mobile distributed platforms. In: Proceedings of ACM CHI 2011 Conference on Human Factors in Computing Systems 2011. pp. 335-340. Available online

Vocal production is one of the most ubiquitous and expressive activities of people, yet understanding its production and synthesis remains elusive. When vocal synthesis is elevated to include new forms of singing and sound production, fundamental changes to culture and musical expression emerge. Nowadays, Text-To-Speech (TTS) synthesis seems unable to suggest innovative solutions for new computing trends, such as mobility, interactivity, ubiquitous computing or expressive manipulation. In this paper, we describe our pioneering work in developing interactive voice synthesis beyond the TTS paradigm. We present DiVA and HandSketch as our two current voice-based digital musical instruments. We then discuss the evolution of this performance practice into a new ubiquitous model applied to voice synthesis, and we describe our first prototype using a mobile phone and wireless embodied devices in order to allow a group of users to collaboratively produce voice synthesis in real-time.

© All rights reserved d'Alessandro et al. and/or their publisher

 
Edit | Del

Miller, Gregor, Fels, Sidney, Hajri, Abir Al, Ilich, Michael, Foley-Fisher, Zoltan, Fernandez, Manuel and Jang, Daesik (2011): MediaDiver: viewing and annotating multi-view video. In: Proceedings of ACM CHI 2011 Conference on Human Factors in Computing Systems 2011. pp. 1141-1146. Available online

We propose to bring our novel rich media interface called MediaDiver demonstrating our new interaction techniques for viewing and annotating multiple view video. The demonstration allows attendees to experience novel moving target selection methods (called Hold and Chase), new multi-view selection techniques, automated quality of view analysis to switch viewpoints to follow targets, integrated annotation methods for viewing or authoring meta-content and advanced context sensitive transport and timeline functions. As users have become increasingly sophisticated when managing navigation and viewing of hyper-documents, they transfer their expectations to new media. Our proposal is a demonstration of the technology required to meet these expectations for video. Thus users will be able to directly click on objects in the video to link to more information or other video, easily change camera views and mark-up the video with their own content. The applications of this technology stretch from home video management to broadcast quality media production, which may be consumed on both desktop and mobile platforms.

© All rights reserved Miller et al. and/or their publisher

 
Edit | Del

Anacleto, Junia C., Balbino, Fernando C., Astolfi, Gilberto, Fels, Sidney and Bueno, Andre O. (2011): A cultural knowledge-based method to support the formation of homophilous online communities. In: Proceedings of ACM CHI 2011 Conference on Human Factors in Computing Systems 2011. pp. 2089-2094. Available online

We propose a three-step method to identify people in social networks sites (SNS) who are talking about the same topics, even though they may be from different cultural backgrounds. Our method uses a cultural knowledge base from the OMCS-Br project to normalize cultural differences and find common interest among users based on statements they make various topics in a SNS. We evaluated three initial phrases that were used to search for sentences in a large social network using the cultural translation; we found that 81% of the retrieved sentences were judged to be related to the initial phrases. Thus, we have evidence that cultural normalization can support finding people talking about the same topic in a SNS even when they have different ways of saying the same thing. We believe that these culturally translated similarities can be used in a recommender system to contribute to the formation of homophilous online communities.

© All rights reserved Anacleto et al. and/or their publisher

2010
 
Edit | Del

Finke, Matthias, Kaviani, Nima, Wang, Ivy, Tsao, Vincent, Fels, Sidney and Lea, Rodger (2010): Investigating distributed user interfaces across interactive large displays and mobile devices. In: Proceedings of the 2010 International Conference on Advanced Visual Interfaces 2010. p. 413. Available online

The use of a dual mobile and large screen approach offers a number of intriguing possibilities including a potential solution to the problem of managing conflicts that arise when a large screen is shared in a public setting. Here, we report on a series of experiments carried out to determine quantitative or qualitative effects of user performance when interaction is split across large public and smaller private screens. Our position is that using mobile devices as an auxiliary device for interaction can boost user experience when interacting with large displays.

© All rights reserved Finke et al. and/or their publisher

 
Edit | Del

Stavness, Ian, Lam, Billy and Fels, Sidney (2010): pCubee: a perspective-corrected handheld cubic display. In: Proceedings of ACM CHI 2010 Conference on Human Factors in Computing Systems 2010. pp. 1381-1390. Available online

In this paper, we describe the design of a personal cubic display that offers novel interaction techniques for static and dynamic 3D content. We extended one-screen Fish Tank VR by arranging five small LCD panels into a box shape that is light and compact enough to be handheld. The display uses head-coupled perspective rendering and a real-time physics simulation engine to establish an interaction metaphor of having real objects inside a physical box that a user can hold and manipulate. We evaluated our prototype as a visualization tool and as an input device by comparing it with a conventional LCD display and mouse for a 3D tree-tracing task. We found that bimanual interaction with pCubee and a mouse offered the best performance and was most preferred by users. pCubee has potential in 3D visualization and interactive applications such as games, storytelling and education, as well as viewing 3D maps, medical and architectural data.

© All rights reserved Stavness et al. and/or their publisher

 
Edit | Del

Anacleto, Junia Coutinho, Fels, Sidney and Villena, Johana Mara Rosas (2010): Design of a web-based therapist tool to promote emotional closeness. In: Proceedings of ACM CHI 2010 Conference on Human Factors in Computing Systems 2010. pp. 3565-3570. Available online

We describe progress using a user-centered design process to migrate a family therapy game to a web-based therapist tool, called FamilySense, that supports therapists creating part of the therapeutic game. Using cards with questions about players' daily life and alternative answers considering their cultural context, the game gives parent and child awareness of each other. Online design of different elements for the board, cards and communication provide an effective online therapy tool. Four user-centered design process stages are presented including: design strategies, design questions, stakeholders, prototype and evaluation for each stage. The process has been successful for the migration, achieving an online game environment that shows strong potential for a family therapy tool.

© All rights reserved Anacleto et al. and/or their publisher

 
Edit | Del

Mller, Stefanie, Miller, Gregor and Fels, Sidney (2010): Using temporal video annotation as a navigational aid for video browsing. In: Proceedings of the 2010 ACM Symposium on User Interface Software and Technology 2010. pp. 445-446. Available online

Video is a complex information space that requires advanced navigational aids for effective browsing. The increasing number of temporal video annotations offers new opportunities to provide video navigation according to a user's needs. We present a novel video browsing interface called TAV (Temporal Annotation Viewing) that provides the user with a visual overview of temporal video annotations. TAV enables the user to quickly determine the general content of a video, the location of scenes of interest and the type of annotations that are displayed while watching the video. An ongoing user study will evaluate our novel approach.

© All rights reserved Mller et al. and/or their publisher

 
Edit | Del

McDougall, Zoe and Fels, Sidney (2010): Cultural probes in the design of communication. In: ACM 28th International Conference on Design of Communication 2010. pp. 57-64. Available online

In this paper, we discuss cultural probes and how they can be used to benefit the design of communication community. A cultural probe is an experimental research method that provides "inspirational data" [9] for design. Through a cultural probe study that we undertook we were able to gain new insights into approaches toward a research project regarding collaboration between artists and scientists. Cultural probes offer the possibility for sustainable communication between designers and those being designed for. It does so by allowing a mental shift in the designer to be able to think from the target demographic's perspective so that designs can reflect that population's desires and concerns.

© All rights reserved McDougall and Fels and/or ACM Press

 
Edit | Del

Sugiyama, Bruno A., Anacleto, Junia C., Fels, Sidney and Caseli, Helena M. (2010): Using cultural knowledge to assist communication between people with different cultural background. In: ACM 28th International Conference on Design of Communication 2010. pp. 183-190. Available online

We present a computational application to facilitate text chat-based communication between people with different cultural and language background. We focus on end-to-end communication between people with rudimentary and intermediary knowledge of the second language using computer support rather than using a simple connection with automated computer translation. Through a user-centered design process, involving three increasingly hifidelity prototypes, we created a system that allows users who speak different languages to send text messages between them that begins with an automated translation of their message that does a partial translation but normally has words that are not translated well. These poorly translated words are then searched for in a common sense knowledge base for the sender's culture that contains meanings gleaned from a large open source initiative to collect common sense knowledge. Using these additional concepts and words coupled to a translator, the user can select from a list of translations those that are better suited to the intention of the message. We illustrate the usefulness of our approach empirically to show that users find the augmented translated messages are culturally sensitive and provide better communication experiences than without it. Our study used messaging between Portuguese (Brazilian) and English speakers.

© All rights reserved Sugiyama et al. and/or ACM Press

2009
 
Edit | Del

Greenberg, Saul and Fels, Sidney (2009): Exploring video streams using slit-tear visualizations. In: Proceedings of ACM CHI 2009 Conference on Human Factors in Computing Systems 2009. pp. 3509-3510. Available online

Slit-tear visualizations allow users to selectively visualize pixel paths in a video scene. The slit-tear visualization technique is a generalization of the traditional photographic slit-scanning and more recent video slicing techniques: after a user specifies a pixel path of interest, the system generates a timeline that replicates those pixels for each frame in the video. These rich visualizations of the video data help users to discover and explore spatio-temporal patterns of activity in a video. In this video, we illustrate the use of slit-tear visualizations to detect movement and incidence of activity in a video scene, accentuate directional motion and small changes in the video, and discover patterns of activity between spatially distinct areas of the scene.

© All rights reserved and Greenberg and/or ACM Press

 
Edit | Del

Lanir, Joel, Greenberg, Saul and Fels, Sidney (2009): Supporting transitions in work: informing large display application design by understanding whiteboard use. In: GROUP09 - International Conference on Supporting Group Work 2009. pp. 149-158. Available online

In this paper, we explore the practice of using a whiteboard for multiple tasks, and specifically how users employ whiteboards to smoothly transition between related sets of tasks. Our study underscores several basic, but important affordances of whiteboards that support this practice, including visual persistence, flexibility of interaction primitives, and their situated physicality. We discuss the implications of these findings for the design of large display applications.

© All rights reserved et al. and/or their publisher

2008
 
Edit | Del

Greenberg, Saul and Fels, Sidney (2008): Exploring video streams using slit-tear visualizations. In: Levialdi, Stefano (ed.) AVI 2008 - Proceedings of the working conference on Advanced Visual Interfaces May 28-30, 2008, Napoli, Italy. pp. 191-198. Available online

 
Edit | Del

You, Wei, Fels, Sidney and Lea, Rodger (2008): Studying vision-based multiple-user interaction with in-home large displays. In: Proceedings of the 2008 ACM International Workshop on Human-Centered Multimedia 2008. pp. 19-26. Available online

Large displays at home such as TVs are becoming larger in size and more interactive in functionality. When multiple co-located users share the screen space of a large display, when, where and how to display their media contents becomes an issue. This paper compares the use of automatic versus manual methods for managing personal screen real-estate on large in-home displays. We assume horizontally laid out "personal interaction spaces" as the user interface for multiple users to manage their screen real-estate. In this case, users need to sign in and out as well as have their personal spaces placed on the display. We constructed a computer-vision based system that tracks the identities and positions of multiple people in front of the display to support the user studies that compare the use of tracker-based mechanisms versus manual ones for managing the display. Our results suggest that the tracking system shows promise for a) simplifying the user registration process in conjunction with a manual sign-in/out process and b) effective tracker-based user-centric placement of people's interaction space. Proper integration of manual methods could improve the sense of control and ownership for users.

© All rights reserved You et al. and/or ACM Press

2007
 
Edit | Del

Botta, David, Werlinger, Rodrigo, Gagn, Andr, Beznosov, Konstantin, Iverson, Lee, Fels, Sidney and Fisher, Brian D. (2007): Towards understanding IT security professionals and their tools. In: Proceedings of the 2007 Symposium on Usable Privacy and Security 2007. pp. 100-111. Available online

We report preliminary results of our ongoing field study of IT professionals who are involved in security management. We interviewed a dozen practitioners from five organizations to understand their workplace and tools. We analyzed the interviews using a variation of Grounded Theory and predesigned themes. Our results suggest that the job of IT security management is distributed across multiple employees, often affiliated with different organizational units or groups within a unit and responsible for different aspects of it. The workplace of our participants can be characterized by their responsibilities, goals, tasks, and skills. Three skills stand out as significant in the IT security management workplace: inferential analysis, pattern recognition, and bricolage.

© All rights reserved Botta et al. and/or ACM Press

 
Edit | Del

Mase, Kenji, Sumi, Yasuyuki and Fels, Sidney (2007): Welcome to the special issue on memory and sharing of experience for the Journal of Personal and Ubiquitous Computing. In Personal and Ubiquitous Computing, 11 (4) pp. 213-214. Available online

 
Edit | Del

Sumi, Yasuyuki, Ito, Sadanori, Matsuguchi, Tetsuya, Fels, Sidney, Iwasawa, Shoichiro, Mase, Kenji, Kogure, Kiyoshi and Hagita, Norihiro (2007): Collaborative capturing, interpreting, and sharing of experiences. In Personal and Ubiquitous Computing, 11 (4) pp. 265-271. Available online

 
Edit | Del

Zhang, Cheng Chris, Cho, Sung-Bae and Fels, Sidney (2007): MyView: Personalized Event Retrieval and Video Compositing from Multi-camera Video Images. In: Smith, Michael J. and Salvendy, Gavriel (eds.) Symposium on Human Interface 2007 - Part I July 22-27, 2007, Beijing, China. pp. 549-558. Available online

 
Edit | Del

Miyaoku, Kento and Fels, Sidney (2007): C-Band: A Flexible Ring Tag System for Camera-Based User Interface. In: Shumaker, Randall (ed.) ICVR 2007 - Virtual Reality - Second International Conference - Part 1 July 22-27, 2007, Beijing, China. pp. 320-328. Available online

2005
 
Edit | Del

Tee, Kimberly, Moffatt, Karyn, Findlater, Leah, MacGregor, Eve, McGrenere, Joanna, Purves, Barbara and Fels, Sidney (2005): A visual recipe book for persons with language impairments. In: Proceedings of ACM CHI 2005 Conference on Human Factors in Computing Systems 2005. pp. 501-510. Available online

Cooking is a daily activity for many people. However, traditional text recipes are often prohibitively difficult to follow for people with language disorders, such as aphasia. We have developed a multi-modal application that leverages the retained ability of aphasic individuals to recognize image-based representations of objects, providing a presentation format that can be more easily followed than a traditional text recipe. Through a systematic approach to developing a visual language for cooking, and the subsequent case study evaluation of a prototype developed according to this language, we show that a combination of visual instructions and navigational structure can help individuals with relatively large language deficits to cook more independently.

© All rights reserved Tee et al. and/or ACM Press

 
Edit | Del

Vatikiotis-Bateson, Eric, Burnham, Denis and Fels, Sidney (eds.) Audio-Visual Speech Processing July 2005, 2005, Vancouver, Canada.

 
Edit | Del

Deutscher, Meghan, Hoskinson, Reynald, Takahashi, Sachiyo and Fels, Sidney (2005): Echology: an interactive spatial sound and video artwork. In: Zhang, Hongjiang, Chua, Tat-Seng, Steinmetz, Ralf, Kankanhalli, Mohan S. and Wilcox, Lynn (eds.) Proceedings of the 13th ACM International Conference on Multimedia November 6-11, 2005, Singapore. pp. 937-945. Available online

 
Edit | Del

Fels, Sidney, Kinoshita, Yuichiro, Chen, Tzu-Pei Grace, Takama, Yasufumi, Yohanan, Steve, Takahashi, Sachiyo, Gadd, Ashley and Funahashi, Kenji (2005): Swimming Across the Pacific: A VR Swimming Interface. In IEEE Computer Graphics and Applications, 25 (1) pp. 24-31. Available online

2004
 
Edit | Del

Beamish, Timothy, MacLean, Karon E. and Fels, Sidney (2004): Manipulating music: multimodal interaction for DJs. In: Dykstra-Erickson, Elizabeth and Tscheligi, Manfred (eds.) Proceedings of ACM CHI 2004 Conference on Human Factors in Computing Systems April 24-29, 2004, Vienna, Austria. pp. 327-334. Available online

In this paper we consider the general goal of supporting physical manipulation of digital audio in a specific context: the performance disk jockey (DJ) seeking to migrate from vinyl to digital media. We classify both the DJ's traditional processes and tools and the field's newest technology. D'Groove, our own technological contribution, is a force feedback turntable used to manipulate digital audio in novel ways. We present an observational study of professional DJ's using D'Groove, and discuss this approach's attributes and directions for future augmentation. Finally, we extend our conclusions about the DJ's emerging needs to the broader domain of digital audio manipulation.

© All rights reserved Beamish et al. and/or ACM Press

 
Edit | Del

Fels, Sidney (2004): Designing intimate experiences. In: Nunes, Nuno Jardim and Rich, Charles (eds.) International Conference on Intelligent User Interfaces 2004 January 13-16, 2004, Funchal, Madeira, Portugal. pp. 2-3. Available online

 
Edit | Del

Chen, Tzu-Pei Grace and Fels, Sidney (2004): Exploring gradient-based face navigation interfaces. In: Graphics Interface 2004 May 17-19, 2004, London, Ontario, Canada. pp. 65-72. Available online

We have created a gradient-based face navigation interface that allows users to explore a large face space based on an eigenface technique. This approach to synthesizing faces contrasts with more typical techniques for forming composite faces based on the blending of facial features. We compare three ways of moving through the face space, using two types of sliders and a face-wheel. These are adapted from typical color space interfaces since they are commonly used. However, eigenface dimensions do not have meaningful text labels, unlike primary colors, necessitating the use of faces themselves for the labels of the navigation axes. Results suggest that users can navigate with face-labelled axes. They find slider interfaces best suited to finding the neighborhood of a target face, but that the face-wheel is better for refinement once inside the neighborhood.

© All rights reserved Chen and Fels and/or their publisher

2003
 
Edit | Del

Beamish, Tim, MacLean, Karon E. and Fels, Sidney (2003): Designing the Haptic Turntable for Musical Control. In: HAPTICS 2003 - 11th International Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems 22-23 March, 2003, Los Angeles, CA, USA. pp. 24-31. Available online

2000
 
Edit | Del

Poupyrev, Ivan, Weghorst, Suzanne and Fels, Sidney (2000): Non-Isomorphic 3D Rotational Techniques. In: Turner, Thea, Szwillus, Gerd, Czerwinski, Mary, Peterno, Fabio and Pemberton, Steven (eds.) Proceedings of the ACM CHI 2000 Human Factors in Computing Systems Conference April 1-6, 2000, The Hague, The Netherlands. pp. 540-547. Available online

This paper demonstrates how non-isomorphic rotational mappings and interaction techniques can be designed and used to build effective spatial 3D user interfaces. In this paper, we develop a mathematical framework allowing us to design non-isomorphic 3D rotational mappings and techniques, investigate their usability properties, and evaluate their user performance characteristics. The results suggest that non-isomorphic rotational mappings can be an effective tool in building high-quality manipulation dialogs in 3D interfaces, allowing our subjects to accomplish experimental tasks 13% faster without a statistically detectable loss in accuracy. The current paper will help interface designers to use non-isomorphic rotational mappings effectively.

© All rights reserved Poupyrev et al. and/or ACM Press

 
Edit | Del

Fels, Sidney, Lee, Eric and Mase, Kenji (2000): Techniques for interactive video cubism (poster session). In: ACM Multimedia 2000 2000. pp. 368-370. Available online

1999
 
Edit | Del

Mulder, Axel G.E., Fels, Sidney and Mase, Kenji (1999): Design of Virtual 3D Instruments for Musical Interaction. In: Graphics Interface 99 June 2-4, 1999, Kingston, Ontario, Canada. pp. 76-83. Available online

 
Edit | Del

Fels, Sidney and Mase, Kenji (1999): Iamascope: a graphical musical instrument. In Computers & Graphics, 23 (2) pp. 277-286. Available online

1998
 
Edit | Del

Sumi, Yasuyuki, Etani, Tameyuki, Fels, Sidney, Simonet, Nicolas, Kobayashi, Kaoru and Mase, Kenji (1998): C-MAP: Building a Context-Aware Mobile Assistant for Exhibition Tours. In: Ishida, Toru (ed.) Community Computing and Support Systems, Social Interaction in Networked Communities June, 1998, Kyoto, Japan. pp. 137-154. Available online

1997
 
Edit | Del

Cooperstock, Jeremy R., Fels, Sidney, Buxton, Bill and Smith, Kenneth C. (1997): Reactive Environments. In Communications of the ACM, 40 (9) pp. 65-73.

1995
 
Edit | Del

Fels, Sidney and Hinton, Geoffrey (1995): GloveTalkII: An Adaptive Gesture-to-Formant Interface. In: Katz, Irvin R., Mack, Robert L., Marks, Linn, Rosson, Mary Beth and Nielsen, Jakob (eds.) Proceedings of the ACM CHI 95 Human Factors in Computing Systems Conference May 7-11, 1995, Denver, Colorado. pp. 456-463. Available online

Glove-TalkII is a system which translates hand gestures to speech through an adaptive interface. Hand gestures are mapped continuously to 10 control parameters of a parallel formant speech synthesizer. The mapping allows the hand to act as an artificial vocal tract that produces speech in real time. This gives an unlimited vocabulary, multiple languages in addition to direct control of fundamental frequency and volume. Currently, the best version of Glove-TalkII uses several input devices (including a Cyberglove, a ContactGlove, a polhemus sensor, and a foot-pedal), a parallel formant speech synthesizer and 3 neural networks. The gesture-to-speech task is divided into vowel and consonant production by using a gating network to weight the outputs of a vowel and a consonant neural network. The gating network and the consonant network are trained with examples from the user. The vowel network implements a fixed, user-defined relationship between hand-position and vowel sound and does not require any training examples from the user. Volume, fundamental frequency and stop consonants are produced with a fixed mapping from the input devices. One subject has trained for about 100 hours to speak intelligibly with Glove-TalkII. He passed through eight distinct stages while learning to speak. He speaks slowly with speech quality similar to a text-to-speech synthesizer but with far more natural-sounding pitch variations.

© All rights reserved Fels and Hinton and/or ACM Press

 
Edit | Del

Fels, Sidney and Hinton, Geoffrey E. (1995): Glove-TalkII: Mapping Hand Gestures to Speech Using Neural Networks. In: Tesauro, Gerald, Touretzky, David S. and Leen, Todd K. (eds.) Advances in Neural Information Processing Systems 7, [NIPS Conference, Denver, Colorado, USA, 1994] 1995. pp. 843-850. Available online

1991
 
Edit | Del

Prime, Martin, Hauptmann, Alexander G., Chapin, William L. and Fels, Sidney (1991): Experiences with Computer Glove Input Devices. In: Proceedings of the Human Factors Society 35th Annual Meeting 1991. pp. 413-415.

1990
 
Edit | Del

Fels, Sidney and Hinton, Geoffrey (1990): Building Adaptive Interfaces with Neural Networks: The Glove-Talk Pilot Study. In: Diaper, Dan, Gilmore, David J., Cockton, Gilbert and Shackel, Brian (eds.) INTERACT 90 - 3rd IFIP International Conference on Human-Computer Interaction August 27-31, 1990, Cambridge, UK. pp. 683-688.

A multilayer neural network can learn complicated mappings from inputs to outputs. After learning a mapping from a set of training examples, the network can generalize to new cases. Although the learning can be slow, the network runs extremely rapidly once it has learned so it can be used for real-time applications. To illustrate the potential of this technology for adaptive interfaces, we used a VPL DataGlove connected to a DECtalk speech synthesizer via five neural networks to implement a hand gesture to speech system. Using minor variations of the standard back-propagation learning procedure, the complex mapping of hand movements to speech is learned using data obtained from a single "speaker" in a simple training phase. With a 203 gesture-to-word vocabulary, the wrong word is produced less than 1% of the time, and no word is produced about 7% of the time. Adaptive control of the speaking rate and word stress is also available. The training times and final performance speed are improved by using small, separate networks for each naturally defined subtask. The system demonstrates that neural networks can be used to develop the complex mappings required in a high bandwidth interface that adapts to the individual user.

© All rights reserved Fels and Hinton and/or North-Holland

 
Add publication
Show list on your website
 
 

Join our community and advance:

Your
Skills

Your
Network

Your
Career

 
Join our community!
 
 
 

Page Information

Page maintainer: The Editorial Team
URL: http://www.interaction-design.org/references/authors/sidney_fels.html