Number of co-authors:22
Number of publications with 3 favourite co-authors:Pourang Irani:Xing-Dong Yang:Xiaojun Bi:
George Fitzmaurice's 3 most productive colleagues in number of publications:Bill Buxton:78Jacob O. Wobbrock:71Tovi Grossman:44
go to course
Information Visualization: Getting Dashboards Right
84% booked. Starts in 11 days
Marc Hassenzahl explains the fascinating concept of User Experience and Experience Design. Commentaries by Don Norman, Eric Reiss, Mark Blythe, and Whitney Hess
User Experience and Experience Design !
Our Latest Books
The Social Design of Technical Systems: Building technologies for communities. 2nd Edition
by Brian Whitworth and Adnan Ahmad
Gamification at Work: Designing Engaging Business Software
by Janaki Mythily Kumar and Mario Herger
The Social Design of Technical Systems: Building technologies for communities
by Brian Whitworth and Adnan Ahmad
The Encyclopedia of Human-Computer Interaction, 2nd Ed.
by Mads Soegaard and Rikke Friis Dam
Publications by George Fitzmaurice (bibliography)
Banovic, Nikola, Grossman, Tovi, Matejka, Justin and Fitzmaurice, George (2012): Waken: reverse engineering usage information and interface structure from software videos. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 83-92. Available online
We present Waken, an application-independent system that recognizes UI components and activities from screen captured videos, without any prior knowledge of that application. Waken can identify the cursors, icons, menus, and tooltips that an application contains, and when those items are used. Waken uses frame differencing to identify occurrences of behaviors that are common across graphical user interfaces. Candidate templates are built, and then other occurrences of those templates are identified using a multi-phase algorithm. An evaluation demonstrates that the system can successfully reconstruct many aspects of a UI without any prior application-dependant knowledge. To showcase the design opportunities that are introduced by having this additional meta-data, we present the Waken Video Player, which allows users to directly interact with UI components that are displayed in the video.
© All rights reserved Banovic et al. and/or ACM Press
Li, Wei, Grossman, Tovi and Fitzmaurice, George (2012): GamiCAD: a gamified tutorial system for first time AutoCad users. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 103-112. Available online
We present GamiCAD, a gamified in-product, interactive tutorial system for first time AutoCAD users. We introduce a software event driven finite state machine to model a user's progress through a tutorial, which allows the system to provide real-time feedback and recognize success and failures. GamiCAD provides extensive real-time visual and audio feedback that has not been explored before in the context of software tutorials. We perform an empirical evaluation of GamiCAD, comparing it to an equivalent in-product tutorial system without the gamified components. In an evaluation, users using the gamified system reported higher subjective engagement levels and performed a set of testing tasks faster with a higher completion ratio.
© All rights reserved Li et al. and/or ACM Press
Yang, Xing-Dong, Grossman, Tovi, Wigdor, Daniel and Fitzmaurice, George (2012): Magic finger: always-available input through finger instrumentation. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 147-156. Available online
We present Magic Finger, a small device worn on the fingertip, which supports always-available input. Magic Finger inverts the typical relationship between the finger and an interactive surface: with Magic Finger, we instrument the user's finger itself, rather than the surface it is touching. Magic Finger senses touch through an optical mouse sensor, enabling any surface to act as a touch screen. Magic Finger also senses texture through a micro RGB camera, allowing contextual actions to be carried out based on the particular surface being touched. A technical evaluation shows that Magic Finger can accurately
© All rights reserved Yang et al. and/or ACM Press
Chilana, Parmit K., Ko, Andrew J., Wobbrock, Jacob O., Grossman, Tovi and Fitzmaurice, George (2011): Post-deployment usability: a survey of current practices. In: Proceedings of ACM CHI 2011 Conference on Human Factors in Computing Systems 2011. pp. 2243-2246. Available online
Despite the growing research on usability in the pre-development phase, we know little about post-deployment usability activities. To characterize these activities, we surveyed 333 full-time usability professionals and consultants working in large and small corporations from a wide range of industries. Our results show that, as a whole, usability professionals are currently not playing a substantial role in the post-deployment phase compared to other phases of user-centered design, but when they do, practitioners find their interactions quite valuable. We highlight opportunities in HCI research and practice to bridge this gap by working more closely with software support and maintenance teams. We also raise the need to understand what might be called 'usability maintenance,' that is, the process and procedures, by which usability is maintained after deployment.
© All rights reserved Chilana et al. and/or their publisher
Bi, Xiaojun, Grossman, Tovi, Matejka, Justin and Fitzmaurice, George (2011): Magic desk: bringing multi-touch surfaces into desktop work. In: Proceedings of ACM CHI 2011 Conference on Human Factors in Computing Systems 2011. pp. 2511-2520. Available online
Despite the prominence of multi-touch technologies, there has been little work investigating its integration into the desktop environment. Bringing multi-touch into desktop computing would give users an additional input channel to leverage, enriching the current interaction paradigm dominated by a mouse and keyboard. We provide two main contributions in this domain. First, we describe the results from a study we performed, which systematically evaluates the various potential regions within the traditional desktop configuration that could become multi-touch enabled. The study sheds light on good or bad regions for multi-touch, and also the type of input most appropriate for each of these regions. Second, guided by the results from our study, we explore the design space of multi-touch-integrated desktop experiences. A set of new interaction techniques are coherently integrated into a desktop prototype, called Magic Desk, demonstrating potential uses for multi-touch enabled desktop configurations.
© All rights reserved Bi et al. and/or their publisher
Yang, Xing-Dong, Grossman, Tovi, Irani, Pourang and Fitzmaurice, George (2011): TouchCuts and TouchZoom: enhanced target selection for touch displays using finger proximity sensing. In: Proceedings of ACM CHI 2011 Conference on Human Factors in Computing Systems 2011. pp. 2585-2594. Available online
Although touch-screen laptops are increasing in popularity, users still do not comfortably rely on touch in these environments, as current software interfaces were not designed for being used by the finger. In this paper, we first demonstrate the benefits of using touch as a complementary input modality along with the keyboard and mouse or touchpad in a laptop setting. To alleviate the frustration users experience with touch, we then design two techniques, TouchCuts, a single target expansion technique, and TouchZoom, a multiple target expansion technique. Both techniques facilitate the selection of small icons, by detecting the finger proximity above the display surface, and expanding the target as the finger approaches. In a controlled evaluation, we show that our techniques improve performance in comparison to both the computer mouse and a baseline touch-based target acquisition technique. We conclude by discussing other application scenarios that our techniques support.
© All rights reserved Yang et al. and/or their publisher
Matejka, Justin, Grossman, Tovi and Fitzmaurice, George (2011): Ambient help. In: Proceedings of ACM CHI 2011 Conference on Human Factors in Computing Systems 2011. pp. 2751-2760. Available online
In this paper we present Ambient Help, a system that supports opportunistic learning by providing automatic, context-sensitive learning resources while a user works. Multiple videos and textual help resources are presented ambiently on a secondary display. We define and examine a collection of design consideration for this type of interface. After describing our implementation details, we report on an experiment which shows that Ambient Help supports finding more helpful information, while not having a negative impact on the user's productivity, as compared to a traditional help condition.
© All rights reserved Matejka et al. and/or their publisher
Chilana, Parmit K., Grossman, Tovi and Fitzmaurice, George (2011): Modern software product support processes and the usage of multimedia formats. In: Proceedings of ACM CHI 2011 Conference on Human Factors in Computing Systems 2011. pp. 3093-3102. Available online
Despite being an important channel for end-user assistance, few studies have directly investigated the interactions that occur in modern-day practice of software product support. We present results from a multi-dimensional analysis of product support activities at a leading design software company. We carried out a quantitative analysis of existing support requests, a survey with product support specialists, and follow-up interviews to understand the current practices in product support. In particular, we investigated the utility of different multimedia formats that modern web-based support systems enable. Our results showed that despite the value that these formats bring to support tasks, support specialists still face bottlenecks in remotely resolving software problems. We conclude by highlighting several opportunities in HCI for improving diagnosis and resolution of software issues over the web.
© All rights reserved Chilana et al. and/or their publisher
Rooke, Michael, Grossman, Tovi and Fitzmaurice, George (2011): AppMap: exploring user interface visualizations. In: Proceedings of the 2011 Conference on Graphics Interface 2011. pp. 111-118.
In traditional graphical user interfaces, the majority of UI elements are hidden to the user in the default view, as application designers and users desire more space for their application data. We explore the benefits of dedicating additional screen space for presenting an alternative visualization of an application's user interface. Some potential benefits are to assist users in examining complex software, understanding the extent of an application's capabilities, and exploring the available features. We propose user interface visualizations, alternative representations of an application's interface augmented with usage information. We introduce a design space for UI visualizations and describe some initial prototypes and insights based on this design space. We then present AppMap, our new design, which displays the entire function set of AutoCAD and allows the user to interactively explore the visualization which is augmented with visual overlays displaying analytical data about the functions and their relations. In our initial studies, users welcomed this new presentation of functionality, and the unique information that it presents.
© All rights reserved Rooke et al. and/or their publisher
Matejka, Justin, Grossman, Tovi and Fitzmaurice, George (2011): IP-QAT: in-product questions, answers, & tips. In: Proceedings of the 2011 ACM Symposium on User Interface Software and Technology 2011. pp. 175-184. Available online
We present IP-QAT, a new community-based question and answer system for software users. Unlike most community forums, IP-QAT is integrated into the actual software application, allowing users to easily post questions, answers and tips without having to leave the application. Our in-product implementation is context-aware and shows relevant posts based on a user's recent activity. It is also designed with minimal transaction costs to encourage users to easily post, include annotated images and file attachments, as well as tag their posts with relevant UI components. We describe a robust cloud-based system implementation, which allowed us to release IP-QAT to 37 users for a 2 week field study. Our study showed that IP-QAT increased user contributions, and subjectively, users found our system more useful and easier to use, in comparison to the existing commercial discussion board.
© All rights reserved Matejka et al. and/or ACM Press
Li, Wei, Grossman, Tovi, Matejka, Justin and Fitzmaurice, George (2011): TwitApp: in-product micro-blogging for design sharing. In: Proceedings of the 2011 ACM Symposium on User Interface Software and Technology 2011. pp. 185-194. Available online
We describe TwitApp, an enhanced micro-blogging system integrated within AutoCAD for design sharing. TwitApp integrates rich content and still keeps the sharing transaction cost low. In TwitApp, tweets are organized by their project, and users can follow or unfollow each individual project. We introduce the concept of automatic tweet drafting and other novel features such as enhanced real-time search and integrated live video streaming. The TwitApp system leverages the existing Twitter micro-blogging system. We also contribute a study which provides insights on these concepts and associated designs, and demonstrates potential user excitement of such tools.
© All rights reserved Li et al. and/or ACM Press
Ekstrand, Michael, Li, Wei, Grossman, Tovi, Matejka, Justin and Fitzmaurice, George (2011): Searching for software learning resources using application context. In: Proceedings of the 2011 ACM Symposium on User Interface Software and Technology 2011. pp. 195-204. Available online
Users of complex software applications frequently need to consult documentation, tutorials, and support resources to learn how to use the software and further their understanding of its capabilities. Existing online help systems provide limited context awareness through "what's this?" and similar techniques. We examine the possibility of making more use of the user's current context in a particular application to provide useful help resources. We provide an analysis and taxonomy of various aspects of application context and how they may be used in retrieving software help artifacts with web browsers, present the design of a context-aware augmented web search system, and describe a prototype implementation and initial user study of this system. We conclude with a discussion of open issues and an agenda for further research.
© All rights reserved Ekstrand et al. and/or ACM Press
Annett, Michelle, Grossman, Tovi, Wigdor, Daniel and Fitzmaurice, George (2011): Medusa: a proximity-aware multi-touch tabletop. In: Proceedings of the 2011 ACM Symposium on User Interface Software and Technology 2011. pp. 337-346. Available online
We present Medusa, a proximity-aware multi-touch tabletop. Medusa uses 138 inexpensive proximity sensors to: detect a user's presence and location, determine body and arm locations, distinguish between the right and left arms, and map touch point to specific users and specific hands. Our tracking algorithms and hardware designs are described. Exploring this unique design, we develop and report on a collection of interactions enabled by Medusa in support of multi-user collaborative design, specifically within the context of Proxi-Sketch, a multi-user UI prototyping tool. We discuss design issues, system implementation, limitations, and generalizable concepts throughout the paper.
© All rights reserved Annett et al. and/or ACM Press
Fernquist, Jennifer, Grossman, Tovi and Fitzmaurice, George (2011): Sketch-sketch revolution: an engaging tutorial system for guided sketching and application learning. In: Proceedings of the 2011 ACM Symposium on User Interface Software and Technology 2011. pp. 373-382. Available online
We describe Sketch-Sketch Revolution, a new tutorial system that allows any user to experience the success of drawing content previously created by an expert artist. Sketch-Sketch Revolution not only guides users through the application user interface, it also provides assistance with the actual sketching. In addition, the system offers an authoring tool that enables artists to create content and then automatically generates a tutorial from their recorded workflow history. Sketch-Sketch Revolution is a unique hybrid tutorial system that combines in-product, content-centric and reactive tutorial methods to provide an engaging learning experience. A qualitative user study showed that our system successfully taught users how to interact with a drawing application user interface, gave users confidence they could recreate expert content, and was uniformly considered useful and easy to use.
© All rights reserved Fernquist et al. and/or ACM Press
Grossman, Tovi and Fitzmaurice, George (2010): ToolClips: an investigation of contextual video assistance for functionality understanding. In: Proceedings of ACM CHI 2010 Conference on Human Factors in Computing Systems 2010. pp. 1515-1524. Available online
We investigate the use of on-line contextual video assistance to improve the learnability of software functionality. After discussing motivations and design goals for such forms of assistance, we present our new technique, ToolClips. ToolClips augment traditional tooltips to provide users with quick and contextual access to both textual and video assistance. In an initial study we found that users successfully integrated ToolClip usage into the flow of their primary tasks to overcome learnability difficulties. In a second study, we found that with ToolClips, users successfully completed 7 times as many unfamiliar tasks, in comparison to using a commercial professionally developed on-line help system. Users also retained the information obtained from ToolClips, performing tasks significantly faster one week later.
© All rights reserved Grossman and Fitzmaurice and/or their publisher
Lepinski, G. Julian, Grossman, Tovi and Fitzmaurice, George (2010): The design and evaluation of multitouch marking menus. In: Proceedings of ACM CHI 2010 Conference on Human Factors in Computing Systems 2010. pp. 2233-2242. Available online
Despite the considerable quantity of research directed towards multitouch technologies, a set of standardized UI components have not been developed. Menu systems provide a particular challenge, as traditional GUI menus require a level of pointing precision inappropriate for direct finger input. Marking menus are a promising alternative, but have yet to be investigated or adapted for use within multitouch systems. In this paper, we first investigate the human capabilities for performing directional chording gestures, to assess the feasibility of multitouch marking menus. Based on the positive results collected from this study, and in particular, high angular accuracy, we discuss our new multitouch marking menu design, which can increase the number of items in a menu, and eliminate a level of depth. A second experiment showed that multitouch marking menus perform significantly faster than traditional hierarchal marking menus, reducing acquisition times in both novice and expert usage modalities.
© All rights reserved Lepinski et al. and/or their publisher
Song, Hyunyoung, Guimbretiere, Francois, Grossman, Tovi and Fitzmaurice, George (2010): MouseLight: bimanual interactions on digital paper using a pen and a spatially-aware mobile projector. In: Proceedings of ACM CHI 2010 Conference on Human Factors in Computing Systems 2010. pp. 2451-2460. Available online
MouseLight is a spatially-aware standalone mobile projector with the form factor of a mouse that can be used in combination with digital pens on paper. By interacting with the projector and the pen bimanually, users can visualize and modify the virtually augmented contents on top of the paper, and seamlessly transition between virtual and physical information. We present a high fidelity hardware prototype of the system and demonstrate a set of novel interactions specifically tailored to the unique properties of MouseLight. MouseLight differentiates itself from related systems such as PenLight in two aspects. First, MouseLight presents a rich set of bimanual interactions inspired by the ToolGlass interaction metaphor, but applied to physical paper. Secondly, our system explores novel displaced interactions, that take advantage of the independent input and output that is spatially aware of the underneath paper. These properties enable users to issue remote commands such as copy and paste or search. We also report on a preliminary evaluation of the system which produced encouraging observations and feedback.
© All rights reserved Song et al. and/or their publisher
Grossman, Tovi, Matejka, Justin and Fitzmaurice, George (2010): Chronicle: capture, exploration, and playback of document workflow histories. In: Proceedings of the 2010 ACM Symposium on User Interface Software and Technology 2010. pp. 143-152. Available online
We describe Chronicle, a new system that allows users to explore document workflow histories. Chronicle captures the entire video history of a graphical document, and provides links between the content and the relevant areas of the history. Users can indicate specific content of interest, and see the workflows, tools, and settings needed to reproduce the associated results, or to better understand how it was constructed to allow for informed modification. Thus, by storing the rich information regarding the document's history workflow, Chronicle makes any working document a potentially powerful learning tool. We outline some of the challenges surrounding the development of such a system, and then describe our implementation within an image editing application. A qualitative user study produced extremely encouraging results, as users unanimously found the system both useful and easy to use.
© All rights reserved Grossman et al. and/or their publisher
Matejka, Justin, Li, Wei, Grossman, Tovi and Fitzmaurice, George (2009): CommunityCommands: command recommendations for software applications. In: Proceedings of the ACM Symposium on User Interface Software and Technology 2009. pp. 193-202. Available online
We explore the use of modern recommender system technology to address the problem of learning software applications. Before describing our new command recommender system, we first define relevant design considerations. We then discuss a 3 month user study we conducted with professional users to evaluate our algorithms which generated customized recommendations for each user. Analysis shows that our item-based collaborative filtering algorithm generates 2.1 times as many good suggestions as existing techniques. In addition we present a prototype user interface to ambiently present command recommendations to users, which has received promising initial user feedback.
© All rights reserved Matejka et al. and/or their publisher
Khan, Azam, Matejka, Justin, Fitzmaurice, George, Kurtenbach, Gord, Burtnyk, Nicolas and Buxton, Bill (2009): Toward the Digital Design Studio: Large Display Explorations. In Human-Computer Interaction, 24 (1) pp. 9-47. Available online
Inspired by our automotive and product design customers using large displays in design centers, visualization studios, and meeting rooms around the world, we have been exploring the use and potential of large display installations for almost a decade. Our research has touched on many aspects of this rich design space, from individual tools to complete systems, and has generally moved through the life cycle of a design artifact: from the creation phase, through communication and collaboration, to presentation and dissemination. As we attempt to preserve creative flow through the phases, we introduce social structures and constraints that drive the design of possible point solutions in the larger context of a digital design studio trail environment built in the lab. Although many of the interactions presented are viable across several design phases, this article focuses primarily on facilitating collaboration. We conclude with critical lessons learned of both what avenues have been fruitful and which roads to avoid. This article lightly covers the whole design process and attempts to inform readers of key factors to consider when designing for designers.
© All rights reserved Khan et al. and/or Taylor and Francis
Join our community and advance:
Page maintainer: The Editorial Team