Proceedings of the 2012 ACM Symposium on User Interface Software and Technology


 
Time and place:

2012
Conf. description:
UIST is the premier forum for innovations in developing human-computer interfaces. The symposium brings together user-interface researchers and practitioners with an interest in techniques, tools, and technology for constructing high-quality, innovative user interfaces.
Help us!
Do you know when the next conference is? If yes, please add it to the calendar!
Series:
This is a preferred venue for people like Scott E. Hudson, Ravin Balakrishnan, Brad A. Myers, Steven K. Feiner, and Takeo Igarashi. Part of the UIST - Symposium on User Interface Software and Technology conference series.
Other years:
Publisher:
ACM Press
EDIT

References from this conference (2012)

The following articles are from "Proceedings of the 2012 ACM Symposium on User Interface Software and Technology":

 what's this?

Articles

p. 1-2

Livingstone, Margaret (2012): What art can tell us about the brain. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 1-2. Available online

Artists have been doing experiments on vision longer than neurobiologists. Some major works of art have provided insights as to how we see; some of these insights are so fundamental that they can be understood in terms of the underlying neurobiology. For example, artists have long realized that color and luminance can play independent roles in visual perception. Picasso said, "Colors are only symbols. Reality is to be found in luminance alone." This observation has a parallel in the functional subdivision of our visual systems, where color and luminance are processed by the newer, primate-specific What system, and the older, colorblind, Where (or How) system. Many techniques developed over the centuries by artists can be understood in terms of the parallel organization of our visual systems. I will explore how the segregation of color and luminance processing are the basis for why some Impressionist paintings seem to shimmer, why some op art paintings seem to move, some principles of Matisse's use of color, and how the Impressionists painted "air". Central and peripheral vision are distinct, and I will show how the differences in resolution across our visual field make the Mona Lisa's smile elusive, and produce a dynamic illusion in Pointillist paintings, Chuck Close paintings, and photomosaics. I will explore how artists have intuited important features about how our brains extract relevant information about faces and objects, and I will discuss why learning disabilities may be associated with artistic talent.

© All rights reserved Livingstone and/or ACM Press

p. 103-112

Li, Wei, Grossman, Tovi and Fitzmaurice, George (2012): GamiCAD: a gamified tutorial system for first time AutoCad users. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 103-112. Available online

We present GamiCAD, a gamified in-product, interactive tutorial system for first time AutoCAD users. We introduce a software event driven finite state machine to model a user's progress through a tutorial, which allows the system to provide real-time feedback and recognize success and failures. GamiCAD provides extensive real-time visual and audio feedback that has not been explored before in the context of software tutorials. We perform an empirical evaluation of GamiCAD, comparing it to an equivalent in-product tutorial system without the gamified components. In an evaluation, users using the gamified system reported higher subjective engagement levels and performed a set of testing tasks faster with a higher completion ratio.

© All rights reserved Li et al. and/or ACM Press

p. 113-122

Laput, Gierad, Adar, Eytan, Dontcheva, Mira and Li, Wilmot (2012): Tutorial-based interfaces for cloud-enabled applications. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 113-122. Available online

Powerful image editing software like Adobe Photoshop and GIMP have complex interfaces that can be hard to master. To help users perform image editing tasks, we introduce tutorial-based applications (tapps) that retain the step-by-step structure and descriptive text of tutorials but can also automatically apply tutorial steps to new images. Thus, tapps can be used to batch process many images automatically, similar to traditional macros. Tapps also support interactive exploration of parameters, automatic variations, and direct manipulation (e.g., selection, brushing). Another key feature of tapps is that they execute on remote instances of Photoshop, which allows users to edit their images on any Web-enabled device. We demonstrate a working prototype system called TappCloud for creating, managing and using tapps. Initial user feedback indicates support for both the interactive features of tapps and their ability to automate image editing. We conclude with a discussion of approaches and challenges of pushing monolithic direct-manipulation GUIs to the cloud.

© All rights reserved Laput et al. and/or ACM Press

p. 123-130

Lyons, Kent, Nguyen, David, Ashbrook, Daniel and White, Sean (2012): Facet: a multi-segment wrist worn system. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 123-130. Available online

We present Facet, a multi-display wrist worn system consisting of multiple independent touch-sensitive segments joined into a bracelet. Facet automatically determines the pose of the system as a whole and of each segment individually. It further supports multi-segment touch, yielding a rich set of touch input techniques. Our work builds on these two primitives to allow the user to control how applications use segments alone and in coordination. Applications can expand to use more segments, collapses to encompass fewer, and be swapped with other segments. We also explore how the concepts from Facet could apply to other devices in this design space.

© All rights reserved Lyons et al. and/or ACM Press

p. 13-22

Marquardt, Nicolai, Hinckley, Ken and Greenberg, Saul (2012): Cross-device interaction via micro-mobility and f-formations. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 13-22. Available online

GroupTogether is a system that explores cross-device interaction using two sociological constructs. First, F-formations concern the distance and relative body orientation among multiple users, which indicate when and how people position themselves as a group. Second, micro-mobility describes how people orient and tilt devices towards one another to promote fine-grained sharing during co-present collaboration. We sense these constructs using: (a) a pair of overhead Kinect depth cameras to sense small groups of people, (b) low-power 8GHz band radio modules to establish the identity, presence, and coarse-grained relative locations of devices, and (c) accelerometers to detect tilting of slate devices. The resulting system supports fluid, minimally disruptive techniques for co-located collaboration by leveraging the proxemics of people as well as the proxemics of devices.

© All rights reserved Marquardt et al. and/or ACM Press

p. 131-136

Ogata, Masa, Sugiura, Yuta, Osawa, Hirotaka and Imai, Michita (2012): iRing: intelligent ring using infrared reflection. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 131-136. Available online

We present the iRing, an intelligent input ring device developed for measuring finger gestures and external input. iRing recognizes rotation, finger bending, and external force via an infrared (IR) reflection sensor that leverages skin characteristics such as reflectance and softness. Furthermore, iRing allows using a push and stroke input method, which is popular in touch displays. The ring design has potential to be used as a wearable controller because its accessory shape is socially acceptable, easy to install, and safe, and iRing does not require extra devices. We present examples of iRing applications and discuss its validity as an inexpensive wearable interface and as a human sensing device.

© All rights reserved Ogata et al. and/or ACM Press

p. 137-146

Bi, Xiaojun, Chelba, Ciprian, Ouyang, Tom, Partridge, Kurt and Zhai, Shumin (2012): Bimanual gesture keyboard. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 137-146. Available online

Gesture keyboards represent an increasingly popular way to input text on mobile devices today. However, current gesture keyboards are exclusively unimanual. To take advantage of the capability of modern multi-touch screens, we created a novel bimanual gesture text entry system, extending the gesture keyboard paradigm from one finger to multiple fingers. To address the complexity of recognizing bimanual gesture, we designed and implemented two related interaction methods, finger-release and space-required, both based on a new multi-stroke gesture recognition algorithm. A formal experiment showed that bimanual gesture behaviors were easy to learn. They improved comfort and reduced the physical demand relative to unimanual gestures on tablets. The results indicated that these new gesture keyboards were valuable complements to unimanual gesture and regular typing keyboards.

© All rights reserved Bi et al. and/or ACM Press

p. 147-156

Yang, Xing-Dong, Grossman, Tovi, Wigdor, Daniel and Fitzmaurice, George (2012): Magic finger: always-available input through finger instrumentation. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 147-156. Available online

We present Magic Finger, a small device worn on the fingertip, which supports always-available input. Magic Finger inverts the typical relationship between the finger and an interactive surface: with Magic Finger, we instrument the user's finger itself, rather than the surface it is touching. Magic Finger senses touch through an optical mouse sensor, enabling any surface to act as a touch screen. Magic Finger also senses texture through a micro RGB camera, allowing contextual actions to be carried out based on the particular surface being touched. A technical evaluation shows that Magic Finger can accurately

© All rights reserved Yang et al. and/or ACM Press

p. 157-166

Gooch, Daniel and Watts, Leon (2012): YourGloves, hothands and hotmits: devices to hold hands at a distance. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 157-166. Available online

There is a growing body of work in HCI on the design of communication technologies to help support lovers in long distance relationships. We build upon this work by presenting an exploratory study of hand-holding prototypes. Our work distinguishes itself by basing distance communication metaphors on elements of familiar, simple co-located behaviours. We argue that the combined evocative power of unique co-created physical representations of the absent other can be used by separated lovers to generate powerful and positive experiences, in turn sustaining romantic connections at a distance.

© All rights reserved Gooch and Watts and/or ACM Press

p. 167-176

Kim, David, Hilliges, Otmar, Izadi, Shahram, Butler, Alex D., Chen, Jiawen, Oikonomidis, Iason and Olivier, Patrick (2012): Digits: freehand 3D interactions anywhere using a wrist-worn gloveless sensor. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 167-176. Available online

Digits is a wrist-worn sensor that recovers the full 3D pose of the user's hand. This enables a variety of freehand interactions on the move. The system targets mobile settings, and is specifically designed to be low-power and easily reproducible using only off-the-shelf hardware. The electronics are self-contained on the user's wrist, but optically image the entirety of the user's hand. This data is processed using a new pipeline that robustly samples key parts of the hand, such as the tips and lower regions of each finger. These sparse samples are fed into new kinematic models that leverage the biomechanical constraints of the hand to recover the 3D pose of the user's hand. The proposed system works without the need for full instrumentation of the hand (for example using data gloves), additional sensors in the environment, or depth cameras which are currently prohibitive for mobile scenarios due to power and form-factor considerations. We demonstrate the utility of Digits for a variety of application scenarios, including 3D spatial interaction with mobile devices, eyes-free interaction on-the-move, and gaming. We conclude with a quantitative and qualitative evaluation of our system, and discussion of strengths, limitations and future work.

© All rights reserved Kim et al. and/or ACM Press

p. 177-188

Pham, Hubert, Paluska, Justin Mazzola, Miller, Rob and Ward, Steve (2012): Clui: a platform for handles to rich objects. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 177-188. Available online

On the desktop, users are accustomed to having visible handles to objects that they want to organize, share, or manipulate. Web applications today feature many classes of such objects, like flight itineraries, products for sale, people, recipes, and businesses, but there are no interoperable handles for high-level semantic objects that users can grab. This paper proposes Clui, a platform for exploring a new data type, called a Webit, that provides uniform handles to rich objects. Clui uses plugins to 1) create Webits on existing pages by extracting semantic data from those pages, and 2) augmenting existing sites with drag and drop targets that accept and interpret Webits. Users drag and drop Webits between sites to transfer data, auto-fill search forms, map associated locations, or share Webits with others. Clui enables experimentation with handles to semantic objects and the standards that underlie them.

© All rights reserved Pham et al. and/or ACM Press

p. 189-196

Kato, Jun, McDirmid, Sean and Cao, Xiang (2012): DejaVu: integrated support for developing interactive camera-based programs. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 189-196. Available online

The increasing popularity of interactive camera-based programs highlights the inadequacies of conventional IDEs in developing these programs given their distinctive attributes and workflows. We present DejaVu, an IDE enhancement that eases the development of these programs by enabling programmers to visually and continuously monitor program data in consistency with the frame-based pipeline of computer-vision programs; and to easily record, review, and reprocess temporal data to iteratively improve the processing of non-reproducible camera input. DejaVu was positively received by three experienced programmers of interactive camera-based programs in our preliminary user trial.

© All rights reserved Kato et al. and/or ACM Press

p. 197-206

Mortier, Richard, Rodden, Tom, Tolmie, Peter, Lodge, Tom, Spencer, Robert, Crabtree, Andy, Sventek, Joe and Koliousis, Alexandros (2012): Homework: putting interaction into the infrastructure. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 197-206. Available online

This paper presents a user driven redesign of the domestic network infrastructure that draws upon a series of ethnographic studies of home networks. We present an infrastructure based around a purpose built access point that has modified the handling of protocols and services to reflect the interactive needs of the home. The developed infrastructure offers a novel measurement framework that allows a broad range of infrastructure information to be easily captured and made available to interactive applications. This is complemented by a diverse set of novel interactive control mechanisms and interfaces for the underlying infrastructure. We also briefly reflect on the technical and user issues arising from deployments.

© All rights reserved Mortier et al. and/or ACM Press

p. 207-218

Abouzied, Azza, Hellerstein, Joseph and Silberschatz, Avi (2012): DataPlay: interactive tweaking and example-driven correction of graphical database queries. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 207-218. Available online

Writing complex queries in SQL is a challenge for users. Prior work has developed several techniques to ease query specification but none of these techniques are applicable to a particularly difficult class of queries: quantified queries. Our hypothesis is that users prefer to specify quantified queries interactively by trial-and-error. We identify two impediments to this form of interactive trial-and-error query specification in SQL: (i) changing quantifiers often requires global syntactical query restructuring, and (ii) the absence of non-answers from SQL's results makes verifying query correctness difficult. We remedy these issues with DataPlay, a query tool with an underlying graphical query language, a unique data model and a graphical interface. DataPlay provides two interaction features that support trial-and-error query specification. First, DataPlay allows users to directly manipulate a graphical query by changing quantifiers and modifying dependencies between constraints. Users receive real-time feedback in the form of updated answers and non-answers. Second, DataPlay can auto-correct a user's query, based on user feedback about which tuples to keep or drop from the answers and non-answers. We evaluated the effectiveness of each interaction feature with a user study and we found that direct query manipulation is more effective than auto-correction for simple queries but auto-correction is more effective than direct query manipulation for more complex queries.

© All rights reserved Abouzied et al. and/or ACM Press

p. 219-228

Wightman, Doug, Ye, Zi, Brandt, Joel and Vertegaal, Roel (2012): SnipMatch: using source code context to enhance snippet retrieval and parameterization. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 219-228. Available online

Programmers routinely use source code snippets to increase their productivity. However, locating and adapting code snippets to the current context still takes time: for example, variables must be renamed, and dependencies included. We believe that when programmers decide to invest time in creating a new code snippet from scratch, they would also be willing to spend additional effort to make that code snippet configurable and easy to integrate. To explore this insight, we built SnipMatch, a plug-in for the Eclipse IDE. SnipMatch introduces a simple markup that allows snippet authors to specify search patterns and integration instructions. SnipMatch leverages this information, in conjunction with current code context, to improve snippet search and parameterization. For example, when a search query includes local variables, SnipMatch suggests compatible snippets, and automatically adapts them by substituting in these variables. In the lab, we observed that participants integrated snippets faster when using SnipMatch than when using standard Eclipse. Findings from a public deployment to 93 programmers suggest that SnipMatch has become integrated into the work practices of real users.

© All rights reserved Wightman et al. and/or ACM Press

p. 229-238

Oney, Stephen, Myers, Brad and Brandt, Joel (2012): ConstraintJS: programming interactive behaviors for the web by integrating constraints and states. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 229-238. Available online

Interactive behaviors in GUIs are often described in terms of states, transitions, and constraints, where the constraints only hold in certain states. These constraints maintain relationships among objects, control the graphical layout, and link the user interface to an underlying data model. However, no existing Web implementation technology provides direct support for all of these, so the code for maintaining constraints and tracking state may end up spread across multiple languages and libraries. In this paper we describe ConstraintJS, a system that integrates constraints and finite-state machines (FSMs) with Web languages. A key role for the FSMs is to enable and disable constraints based on the interface's current mode, making it possible to write constraints that sometimes hold. We illustrate that constraints combined with FSMs can be a clearer way of defining many interactive behaviors with a series of examples.

© All rights reserved Oney et al. and/or ACM Press

p. 23-34

Lasecki, Walter, Miller, Christopher, Sadilek, Adam, Abumoussa, Andrew, Borrello, Donato, Kushalnagar, Raja and Bigham, Jeffrey (2012): Real-time captioning by groups of non-experts. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 23-34. Available online

Real-time captioning provides deaf and hard of hearing people immediate access to spoken language and enables participation in dialogue with others. Low latency is critical because it allows speech to be paired with relevant visual cues. Currently, the only reliable source of real-time captions are expensive stenographers who must be recruited in advance and who are trained to use specialized keyboards. Automatic speech recognition (ASR) is less expensive and available on-demand, but its low accuracy, high noise sensitivity, and need for training beforehand render it unusable in real-world situations. In this paper, we introduce a new approach in which groups of non-expert captionists (people who can hear and type) collectively caption speech in real-time on-demand. We present Legion:Scribe, an end-to-end system that allows deaf people to request captions at any time. We introduce an algorithm for merging partial captions into a single output stream in real-time, and a captioning interface designed to encourage coverage of the entire audio stream. Evaluation with 20 local participants and 18 crowd workers shows that non-experts can provide an effective solution for captioning, accurately covering an average of 93.2% of an audio stream with only 10 workers and an average per-word latency of 2.9 seconds. More generally, our model in which multiple workers contribute partial inputs that are automatically merged in real-time may be extended to allow dynamic groups to surpass constituent individuals (even experts) on a variety of human performance tasks.

© All rights reserved Lasecki et al. and/or ACM Press

p. 239-250

Roesner, Franziska, Fogarty, James and Kohno, Tadayoshi (2012): User interface toolkit mechanisms for securing interface elements. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 239-250. Available online

User interface toolkit research has traditionally assumed that developers have full control of an interface. This assumption is challenged by the mashup nature of many modern interfaces, in which different portions of a single interface are implemented by multiple, potentially mutually distrusting developers (e.g., an Android application embedding a third-party advertisement). We propose considering security as a primary goal for user interface toolkits. We motivate the need for security at this level by examining today's mashup scenarios, in which security and interface flexibility are not simultaneously achieved. We describe a security-aware user interface toolkit architecture that secures interface elements while providing developers with the flexibility and expressivity traditionally desired in a user interface toolkit. By challenging trust assumptions inherent in existing approaches, this architecture effectively addresses important interface-level security concerns.

© All rights reserved Roesner et al. and/or ACM Press

p. 251-260

Joshi, Neel, Mehta, Sisil, Drucker, Steven, Stollnitz, Eric, Hoppe, Hugues, Uyttendaele, Matt and Cohen, Michael (2012): Cliplets: juxtaposing still and dynamic imagery. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 251-260. Available online

We explore creating ""cliplets"", a form of visual media that juxtaposes still image and video segments, both spatially and temporally, to expressively abstract a moment. Much as in ""cinemagraphs"", the tension between static and dynamic elements in a cliplet reinforces both aspects, strongly focusing the viewer's attention. Creating this type of imagery is challenging without professional tools and training. We develop a set of idioms, essentially spatiotemporal mappings, that characterize cliplet elements, and use these idioms in an interactive system to quickly compose a cliplet from ordinary handheld video. One difficulty is to avoid artifacts in the cliplet composition without resorting to extensive manual input. We address this with automatic alignment, looping optimization and feathering, simultaneous matting and compositing, and Laplacian blending. A key user-interface challenge is to provide affordances to define the parameters of the mappings from input time to output time while maintaining a focus on the cliplet being created. We demonstrate the creation of a variety of cliplet types. We also report on informal feedback as well as a more structured survey of users.

© All rights reserved Joshi et al. and/or ACM Press

p. 261-270

Pindat, Cyprien, Pietriga, Emmanuel, Chapuis, Olivier and Puech, Claude (2012): JellyLens: content-aware adaptive lenses. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 261-270. Available online

Focus+context lens-based techniques smoothly integrate two levels of detail using spatial distortion to connect the magnified region and the context. Distortion guarantees visual continuity, but causes problems of interpretation and focus targeting, partly due to the fact that most techniques are based on statically-defined, regular lens shapes, that result in far-from-optimal magnification and distortion. JellyLenses dynamically adapt to the shape of the objects of interest, providing detail-in-context visualizations of higher relevance by optimizing what regions fall into the focus, context and spatially-distorted transition regions. This both improves the visibility of content in the focus region and preserves a larger part of the context region. We describe the approach and its implementation, and report on a controlled experiment that evaluates the usability of JellyLenses compared to regular fisheye lenses, showing clear performance improvements with the new technique for a multi-scale visual search task.

© All rights reserved Pindat et al. and/or ACM Press

p. 271-280

Karnik, Abhijit, Plasencia, Diego Martinez, Mayol-Cuevas, Walterio and Subramanian, Sriram (2012): PiVOT: personalized view-overlays for tabletops. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 271-280. Available online

We present PiVOT, a tabletop system aimed at supporting mixed-focus collaborative tasks. Through two view-zones, PiVOT provides personalized views to individual users while presenting an unaffected and unobstructed shared view to all users. The system supports multiple personalized views which can be present at the same spatial location and yet be only visible to the users it belongs to. The system also allows the creation of personal views that can be either 2D or (auto-stereoscopic) 3D images. We first discuss the motivation and the different implementation principles required for realizing such a system, before exploring different designs able to address the seemingly opposing challenges of shared and personalized views. We then implement and evaluate a sample prototype to validate our design ideas and present a set of sample applications to demonstrate the utility of the system.

© All rights reserved Karnik et al. and/or ACM Press

p. 281-286

Chevalier, Fanny, Dragicevic, Pierre and HURTER, Christophe (2012): Histomages: fully synchronized views for image editing. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 281-286. Available online

We present Histomages, a new interaction model for image editing that considers color histograms as spatial rearrangements of image pixels. Users can select pixels on image histograms as they would select image regions and directly manipulate them to adjust their colors. Histomages are also affected by other image tools such as paintbrushes. We explore some possibilities offered by this interaction model, and discuss the four key principles behind it as well as their implications for the design of feature-rich software in general.

© All rights reserved Chevalier et al. and/or ACM Press

p. 287-298

Ribeiro, Andre and Igarashi, Takeo (2012): Sketch-editing games: human-machine communication, game theory and applications. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 287-298. Available online

We study uncertainty in graphical-based interaction (with special attention to sketches). We argue that a comprehensive model for the problem must include the interaction participants (and their current beliefs), their possible actions and their past sketches. It's yet unclear how to frame and solve the former problem, considering all the latter elements. We suggest framing the problem as a game and solving it with a game-theoretical solution, which leads to a framework for the design of new two-way, sketch-based user interfaces. In special, we use the framework to design a game that can progressively learn visual models of objects from user sketches, and use the models in real-world interactions. Instead of an abstract visual criterion, players in this game learn models to optimize interaction (the game's duration). This two-way sketching game addresses problems essential in emerging interfaces (such as learning and how to deal with interpretation errors). We review possible applications in robotic sketch-to-command, hand gesture recognition, media authoring and visual search, and evaluate two. Evaluations demonstrate how players improve performance with repeated play, and the influence of interaction aspects on learning.

© All rights reserved Ribeiro and Igarashi and/or ACM Press

p. 299-308

Tsandilas, Theophanis (2012): Interpreting strokes on paper with a mobile assistant. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 299-308. Available online

Digital pen technology has allowed for the easy transfer of pen data from paper to the computer. However, linking handwritten content with the digital world remains a hard problem as it requires the translation of unstructured and highly personal vocabularies into structured ones that computers can easily understand and process. Automatic recognition can help to this direction, but as it is not always reliable, solutions require the active cooperation between users and recognition algorithms. This work examines the use of portable touch-screen devices in connection with pen and paper to help users direct and refine the interpretation of their strokes on paper. We explore four techniques of bi-manual interaction that combine touch and pen-writing, where user attention is divided between the original strokes on paper and their interpretation by the electronic device. We demonstrate the techniques through a mobile interface for writing music that complements the automatic recognition with interactive user-driven interpretation. An experiment evaluates the four techniques and provides insights about their strengths and limitations.

© All rights reserved Tsandilas and/or ACM Press

p. 3-12

Huang, Jeff, Etzioni, Oren, Zettlemoyer, Luke, Clark, Kevin and Lee, Christian (2012): RevMiner: an extractive interface for navigating reviews on a smartphone. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 3-12. Available online

Smartphones are convenient, but their small screens make searching, clicking, and reading awkward. Thus, perusing product reviews on a smartphone is difficult. In response, we introduce RevMiner -- a novel smartphone interface that utilizes Natural Language Processing techniques to analyze and navigate reviews. RevMiner was run over 300K Yelp restaurant reviews extracting attribute-value pairs, where attributes represent restaurant attributes such as sushi and service, and values represent opinions about the attributes such as fresh or fast. These pairs were aggregated and used to: 1) answer queries such as "cheap Indian food", 2) concisely present information about each restaurant, and 3) identify similar restaurants. Our user studies demonstrate that on a smartphone, participants preferred RevMiner's interface to tag clouds and color bars, and that they preferred RevMiner's results to Yelp's, particularly for conjunctive queries (e.g., "great food and huge portions"). Demonstrations of RevMiner are available at revminer.com.

© All rights reserved Huang et al. and/or ACM Press

p. 309-318

Hamilton, William, Kerne, Andruid and Robbins, Tom (2012): High-performance pen + touch modality interactions: a real-time strategy game eSports context. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 309-318. Available online

We used the situated context of real-time strategy (RTS) games to address the design and evaluation of new pen + touch interaction techniques. RTS play is a popular genre of Electronic Sports (eSports), games played and spectated at an extremely high level. Interaction techniques are critical for eSports players, because they so directly impact performance. Through this process, new techniques and implications for pen + touch and bi-manual interaction emerged. We enhance non-dominant hand (NDH) interaction with edge-constrained affordances, anchored to physical features of interactive surfaces, effectively increasing target width. We develop bi-manual overloading, an approach to reduce the total number of occurrences of NDH retargeting. The novel isosceles lasso select technique facilitates selection of complex object subsets. Pen-in-hand interaction, dominant hand touch interaction performed with the pen stowed in the palm, also emerged as an efficient and expressive interaction paradigm.

© All rights reserved Hamilton et al. and/or ACM Press

p. 319-326

Liang, Rong-Hao, Cheng, Kai-Yin, Su, Chao-Huai, Weng, Chien-Ting, Chen, Bing-Yu and Yang, De-Nian (2012): GaussSense: attachable stylus sensing using magnetic sensor grid. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 319-326. Available online

This work presents GaussSense, which is a back-of-device sensing technique for enabling input on an arbitrary surface using stylus by exploiting magnetism. A 2mm-thick Hall sensor grid is developed to sense magnets that are embedded in the stylus. Our system can sense the magnetic field that is emitted from the stylus when it is within 2cm of any non-ferromagnetic surface. Attaching the sensor behind an arbitrary thin surface enables the stylus input to be recognized by analyzing the distribution of the applied magnetic field. Attaching the sensor grid to the back of a touchscreen device and incorporating magnets into the corresponding stylus enable the system 1) to distinguish touch events that are caused by a finger from those caused by the stylus, 2) to sense the tilt angle of the stylus and the pressure with which it is applied, and 3) to detect where the stylus hovers over the screen. A pilot study reveals that people were satisfied with the novel sketching experiences based on this system.

© All rights reserved Liang et al. and/or ACM Press

p. 327-330

Liu, Shenwei and Guimbretiere, Francois (2012): FlexAura: a flexible near-surface range sensor. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 327-330. Available online

The availability of flexible capacitive sensors that can be fitted around mice, smartphones, and pens carries great potential in leveraging grasp as a new interaction modality. Unfortunately, most capacitive sensors only track interaction directly on the surface, making it harder to differentiate among grips and constraining user movements. We present a new optical range sensor design based on high power infrared LEDs and photo-transistors, which can be fabricated on a flexible PCB and wrapped around a wide variety of graspable objects including pens, mice, smartphones, and slates. Our sensor offers a native resolution of 10 dpi with a sensing range of up to 30mm (1.2") and sampling speed of 50Hz. Based on our prototype wrapped around the barrel of a pen, we present a summary of the characteristics of the sensor and describe the sensor output in several typical pen grips. Our design is versatile enough to apply not only to pens but to a wide variety of graspable objects including smartphones and slates.

© All rights reserved Liu and Guimbretiere and/or ACM Press

p. 331-340

Lee, David, Son, KyoungHee, Lee, Joon Hyub and Bae, Seok-Hyung (2012): PhantomPen: virtualization of pen head for digital drawing free from pen occlusion & visual parallax. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 331-340. Available online

We present PhantomPen, a direct pen input device whose pen head is virtualized onto the tablet display surface and visually connected to a graspable pen barrel in order to achieve digital drawing free from pen occlusion and visual parallax. As the pen barrel approaches the display, the virtual pen head smoothly appears as if the rendered virtual pen head and the physical pen barrel are in unity. The virtual pen head provides visual feedback by changing its virtual form according to pen type, color, and thickness while the physical pen tip, hidden in the user's sight, provides tactile feedback. Three experiments were carefully designed based on an analysis of drawings by design professionals and observations of design drawing classes. With these experiments that simulate natural drawing we proved significant performance advantages of PhantomPen. PhantomPen was at least as usable as the normal stylus in basic line drawing, and was 17% faster in focus region drawing (26% faster in extreme focus region drawing). PhantomPen also reduced error rate by 40% in a typical drawing setup where users have to manage a complex combination of pen and stroke properties.

© All rights reserved Lee et al. and/or ACM Press

p. 341-350

Quinn, Philip, Cockburn, Andy, Casiez, Gry, Roussel, Nicolas and Gutwin, Carl (2012): Exposing and understanding scrolling transfer functions. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 341-350. Available online

Scrolling is controlled through many forms of input devices, such as mouse wheels, trackpad gestures, arrow keys, and joysticks. Performance with these devices can be adjusted by introducing variable transfer functions to alter the range of expressible speed, precision, and sensitivity. However, existing transfer functions are typically "black boxes" bundled into proprietary operating systems and drivers. This presents three problems for researchers: (1) a lack of knowledge about the current state of the field; (2) a difficulty in replicating research that uses scrolling devices; and (3) a potential experimental confound when evaluating scrolling devices and techniques. These three problems are caused by gaps in researchers' knowledge about what device and movement factors are important for scrolling transfer functions, and about how existing devices and drivers use these factors. We fill these knowledge gaps with a framework of transfer function factors for scrolling, and a method for analysing proprietary transfer functions -- demonstrating how state of the art commercial devices accommodate some of the human control phenomena observed in prior studies.

© All rights reserved Quinn et al. and/or ACM Press

p. 35-44

Ramakers, Raf, Vanacken, Davy, Luyten, Kris, Coninx, Karin and Schning, Johannes (2012): Carpus: a non-intrusive user identification technique for interactive surfaces. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 35-44. Available online

Interactive surfaces have great potential for co-located collaboration because of their ability to track multiple inputs simultaneously. However, the multi-user experience on these devices could be enriched significantly if touch points could be associated with a particular user. Existing approaches to user identification are intrusive, require users to stay in a fixed position, or suffer from poor accuracy. We present a non-intrusive, high-accuracy technique for mapping touches to their corresponding user in a collaborative environment. By mounting a high-resolution camera above the interactive surface, we are able to identify touches reliably without any extra instrumentation, and users are able to move around the surface at will. Our technique, which leverages the back of users' hands as identifiers, supports walk-up-and-use situations in which multiple people interact on a shared surface.

© All rights reserved Ramakers et al. and/or ACM Press

p. 351-358

Roussel, Nicolas, Casiez, Gry, Aceituno, Jonathan and Vogel, Daniel (2012): Giving a hand to the eyes: leveraging input accuracy for subpixel interaction. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 351-358. Available online

We argue that the current practice of using integer positions for pointing events artificially constrains human precision capabilities. The high sensitivity of current input devices can be harnessed to enable precise direct manipulation ""in between"" pixels, called subpixel interaction. We provide detailed analysis of subpixel theory and implementation, including the critical component of revised control-display gain transfer functions. A prototype implementation is described with several illustrative examples. Guidelines for subpixel domain applicability are provided and an overview of required changes to operating systems and graphical user interface frameworks are discussed.

© All rights reserved Roussel et al. and/or ACM Press

p. 359-366

Rubin, Steve, Berthouzoz, Floraine, Mysore, Gautham, Li, Wilmot and Agrawala, Maneesh (2012): UnderScore: musical underlays for audio stories. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 359-366. Available online

Audio producers often use musical underlays to emphasize key moments in spoken content and give listeners time to reflect on what was said. Yet, creating such underlays is time-consuming as producers must carefully (1) mark an emphasis point in the speech (2) select music with the appropriate style, (3) align the music with the emphasis point, and (4) adjust dynamics to produce a harmonious composition. We present UnderScore, a set of semi-automated tools designed to facilitate the creation of such underlays. The producer simply marks an emphasis point in the speech and selects a music track. UnderScore automatically refines, aligns and adjusts the speech and music to generate a high-quality underlay. UnderScore allows producers to focus on the high-level design of the underlay; they can quickly try out a variety of music and test different points of emphasis in the story. Amateur producers, who may lack the time or skills necessary to author underlays, can quickly add music to their stories. An informal evaluation of UnderScore suggests that it can produce high-quality underlays for a variety of examples while significantly reducing the time and effort required of radio producers.

© All rights reserved Rubin et al. and/or ACM Press

p. 367-378

Ahmed, Faisal, Borodin, Yevgen, Soviak, Andrii, Islam, Muhammad, Ramakrishnan, I. V. and Hedgpeth, Terri (2012): Accessible skimming: faster screen reading of web pages. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 367-378. Available online

In our information-driven web-based society, we are all gradually falling ""victims"" to information overload [5]. However, while sighted people are finding ways to sift through information faster, Internet users who are blind are experiencing an even greater information overload. These people access computers and Internet using screen-reader software, which reads the information on a computer screen sequentially using computer-generated speech. While sighted people can learn how to quickly glance over the headlines and news articles online to get the gist of information, people who are blind have to use keyboard shortcuts to listen through the content narrated by a serial audio interface. This interface does not give them an opportunity to know what content to skip and what to listen to. So, they either listen to all of the content or listen to the first part of each sentence or paragraph before they skip to the next one. In this paper, we propose an automated approach to facilitate non-visual skimming of web pages. We describe the underlying algorithm, outline a non-visual skimming interface, and report on the results of automated experiments, as well as on our user study with 23 screen-reader users. The results of the experiments suggest that we have been moderately successful in designing a viable algorithm for automatic summarization that could be used for non-visual skimming. In our user studies, we confirmed that people who are blind could read and search through online articles faster and were able to understand and remember most of what they have read with our skimming system. Finally, all 23 participants expressed genuine interest in using non-visual skimming in the future.

© All rights reserved Ahmed et al. and/or ACM Press

p. 379-388

Schmidt, Dominik, Molyneaux, David and Cao, Xiang (2012): PICOntrol: using a handheld projector for direct control of physical devices through visible light. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 379-388. Available online

Today's environments are populated with a growing number of electric devices which come in diverse form factors and provide a plethora of functions. However, rich interaction with these devices can become challenging if they need be controlled from a distance, or are too small to accommodate user interfaces on their own. In this work, we explore PICOntrol, a new approach using an off-the-shelf handheld pico projector for direct control of physical devices through visible light. The projected image serves a dual purpose by simultaneously presenting a visible interface to the user, and transmitting embedded control information to inexpensive sensor units integrated with the devices. To use PICOntrol, the user points the handheld projector at a target device, overlays a projected user interface on its sensor unit, and performs various GUI-style or gestural interactions. PICOntrol enables direct, visible, and rich interactions with various physical devices without requiring central infrastructure. We present our prototype implementation as well as explorations of its interaction space through various application examples.

© All rights reserved Schmidt et al. and/or ACM Press

p. 389-402

Gupta, Ankit, Fox, Dieter, Curless, Brian and Cohen, Michael (2012): DuploTrack: a real-time system for authoring and guiding Duplo block assembly. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 389-402. Available online

We demonstrate a realtime system which infers and tracks the assembly process of a snap-together block model using a Kinect sensor. The inference enables us to build a virtual replica of the model at every step. Tracking enables us to provide context specific visual feedback on a screen by augmenting the rendered virtual model aligned with the physical model. The system allows users to author a new model and uses the inferred assembly process to guide its recreation by others. We propose a novel way of assembly guidance where the next block to be added is rendered in blinking mode with the tracked virtual model on screen. The system is also able to detect any mistakes made and helps correct them by providing appropriate feedback. We focus on assemblies of Duplo blocks. We discuss the shortcomings of existing methods of guidance -- static figures or recorded videos -- and demonstrate how our method avoids those shortcomings. We also report on a user study to compare our system with standard figure-based guidance methods found in user manuals. The results of the user study suggest that our method is able to aid users' structural perception of the model better, leads to fewer assembly errors, and reduces model construction time.

© All rights reserved Gupta et al. and/or ACM Press

p. 403-412

Sankar, Aditya and Seitz, Steven (2012): Capturing indoor scenes with smartphones. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 403-412. Available online

In this paper, we present a novel smartphone application designed to easily capture, visualize and reconstruct homes, offices and other indoor scenes. Our application leverages data from smartphone sensors such as the camera, accelerometer, gyroscope and magnetometer to help model the indoor scene. The output of the system is two-fold; first, an interactive visual tour of the scene is generated in real time that allows the user to explore each room and transition between connected rooms. Second, with some basic interactive photogrammetric modeling the system generates a 2D floor plan and accompanying 3D model of the scene, under a Manhattan-world assumption. The approach does not require any specialized equipment or training and is able to produce accurate floor plans.

© All rights reserved Sankar and Seitz and/or ACM Press

p. 413-422

Wilson, Andrew, Benko, Hrvoje, Izadi, Shahram and Hilliges, Otmar (2012): Steerable augmented reality with the beamatron. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 413-422. Available online

Steerable displays use a motorized platform to orient a projector to display graphics at any point in the room. Often a camera is included to recognize markers and other objects, as well as user gestures in the display volume. Such systems can be used to superimpose graphics onto the real world, and so are useful in a number of augmented reality and ubiquitous computing scenarios. We contribute the Beamatron, which advances steerable displays by drawing on recent progress in depth camera-based interactions. The Beamatron consists of a computer-controlled pan and tilt platform on which is mounted a projector and Microsoft Kinect sensor. While much previous work with steerable displays deals primarily with projecting corrected graphics onto a discrete set of static planes, we describe computational techniques that enable reasoning in 3D using live depth data. We show two example applications that are enabled by the unique capabilities of the Beamatron: an augmented reality game in which a player can drive a virtual toy car around a room, and a ubiquitous computing demo that uses speech and gesture to move projected graphics throughout the room.

© All rights reserved Wilson et al. and/or ACM Press

p. 423-434

Held, Robert, Gupta, Ankit, Curless, Brian and Agrawala, Maneesh (2012): 3D puppetry: a Kinect-based interface for 3D animation. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 423-434. Available online

We present a system for producing 3D animations using physical objects (i.e., puppets) as input. Puppeteers can load 3D models of familiar rigid objects, including toys, into our system and use them as puppets for an animation. During a performance, the puppeteer physically manipulates these puppets in front of a Kinect depth sensor. Our system uses a combination of image-feature matching and 3D shape matching to identify and track the physical puppets. It then renders the corresponding 3D models into a virtual set. Our system operates in real time so that the puppeteer can immediately see the resulting animation and make adjustments on the fly. It also provides 6D virtual camera \rev{and lighting} controls, which the puppeteer can adjust before, during, or after a performance. Finally our system supports layered animations to help puppeteers produce animations in which several characters move at the same time. We demonstrate the accessibility of our system with a variety of animations created by puppeteers with no prior animation experience.

© All rights reserved Held et al. and/or ACM Press

p. 435-444

Chen, Jiawen, Izadi, Shahram and Fitzgibbon, Andrew (2012): KinÊtre: animating the world with the human body. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 435-444. Available online

Kintre allows novice users to scan arbitrary physical objects and bring them to life in seconds. The fully interactive system allows diverse static meshes to be animated using the entire human body. Traditionally, the process of mesh animation is laborious and requires domain expertise, with rigging specified manually by an artist when designing the character. Kintre makes creating animations a more playful activity, conducted by novice users interactively "at runtime". This paper describes the Kintre system in full, highlighting key technical contributions and demonstrating many examples of users animating meshes of varying shapes and sizes. These include non-humanoid meshes and incomplete surfaces produced by 3D scanning -- two challenging scenarios for existing mesh animation systems. Rather than targeting professional CG animators, Kintre is intended to bring mesh animation to a new audience of novice users. We demonstrate potential uses of our system for interactive storytelling and new forms of physical gaming.

© All rights reserved Chen et al. and/or ACM Press

p. 445-452

Lehtinen, Ville, Oulasvirta, Antti, Salovaara, Antti and Nurmi, Petteri (2012): Dynamic tactile guidance for visual search tasks. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 445-452. Available online

Visual search in large real-world scenes is both time consuming and frustrating, because the search becomes serial when items are visually similar. Tactile guidance techniques can facilitate search by allowing visual attention to focus on a subregion of the scene. We present a technique for dynamic tactile cueing that couples hand position with a scene position and uses tactile feedback to guide the hand actively toward the target. We demonstrate substantial improvements in task performance over a baseline of visual search only, when the scene's complexity increases. Analyzing task performance, we demonstrate that the effect of visual complexity can be practically eliminated through improved spatial precision of the guidance.

© All rights reserved Lehtinen et al. and/or ACM Press

p. 45-54

Cross, Andrew, Cutrell, Edward and Thies, William (2012): Low-cost audience polling using computer vision. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 45-54. Available online

Electronic response systems known as "clickers" have demonstrated educational benefits in well-resourced classrooms, but remain out-of-reach for most schools due to their prohibitive cost. We propose a new, low-cost technique that utilizes computer vision for real-time polling of a classroom. Our approach allows teachers to ask a multiple-choice question. Students respond by holding up a qCard: a sheet of paper that contains a printed code, similar to a QR code, encoding their student IDs. Students indicate their answers (A, B, C or D) by holding the card in one of four orientations. Using a laptop and an off-the-shelf webcam, our software automatically recognizes and aggregates the students' responses and displays them to the teacher. We built this system and performed initial trials in secondary schools in Bangalore, India. In a 25-student classroom, our system offers 99.8% recognition accuracy, captures 97% of responses within 10 seconds, and costs 15 times less than existing electronic solutions.

© All rights reserved Cross et al. and/or ACM Press

p. 453-464

Ng, Albert, Lepinski, Julian, Wigdor, Daniel, Sanders, Steven and Dietz, Paul (2012): Designing for low-latency direct-touch input. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 453-464. Available online

Software designed for direct-touch interfaces often utilize a metaphor of direct physical manipulation of pseudo "real-world" objects. However, current touch systems typically take 50-200ms to update the display in response to a physical touch action. Utilizing a high performance touch demonstrator, subjects were able to experience touch latencies ranging from current levels down to about 1ms. Our tests show that users greatly prefer lower latencies, and noticeable improvement continued well below 10ms. This level of performance is difficult to achieve in commercial computing systems using current technologies. As an alternative, we propose a hybrid system that provides low-fidelity visu-al feedback immediately, followed by high-fidelity visuals at standard levels of latency.

© All rights reserved Ng et al. and/or ACM Press

p. 465-476

Weir, Daryl, Rogers, Simon, Murray-Smith, Roderick and Lchtefeld, Markus (2012): A user-specific machine learning approach for improving touch accuracy on mobile devices. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 465-476. Available online

We present a flexible Machine Learning approach for learning user-specific touch input models to increase touch accuracy on mobile devices. The model is based on flexible, non-parametric Gaussian Process regression and is learned using recorded touch inputs. We demonstrate that significant touch accuracy improvements can be obtained when either raw sensor data is used as an input or when the device's reported touch location is used as an input, with the latter marginally outperforming the former. We show that learned offset functions are highly nonlinear and user-specific and that user-specific models outperform models trained on data pooled from several users. Crucially, significant performance improvements can be obtained with a small (≈200) number of training examples, easily obtained for a particular user through a calibration game or from keyboard entry data.

© All rights reserved Weir et al. and/or ACM Press

p. 477-486

Kin, Kenrick, Hartmann, Bjrn, DeRose, Tony and Agrawala, Maneesh (2012): Proton++: a customizable declarative multitouch framework. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 477-486. Available online

Proton++ is a declarative multitouch framework that allows developers to describe multitouch gestures as regular expressions of touch event symbols. It builds on the Proton framework by allowing developers to incorporate custom touch attributes directly into the gesture description. These custom attributes increase the expressivity of the gestures, while preserving the benefits of Proton: automatic gesture matching, static analysis of conflict detection, and graphical gesture creation. We demonstrate Proton++'s flexibility with several examples: a direction attribute for describing trajectory, a pinch attribute for detecting when touches move towards one another, a touch area attribute for simulating pressure, an orientation attribute for selecting menu items, and a screen location attribute for simulating hand ID. We also use screen location to simulate user ID and enable simultaneous recognition of gestures by multiple users. In addition, we show how to incorporate timing into Proton++ gestures by reporting touch events at a regular time interval. Finally, we present a user study that suggests that users are roughly four times faster at interpreting gestures written using Proton++ than those written in procedural event-handling code commonly used today.

© All rights reserved Kin et al. and/or ACM Press

p. 487-496

Murugappan, Sundar, Vinayak, A, Elmqvist, Niklas and Ramani, Karthik (2012): Extended multitouch: recovering touch posture and differentiating users using a depth camera. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 487-496. Available online

Multitouch surfaces are becoming prevalent, but most existing technologies are only capable of detecting the user's actual points of contact on the surface and not the identity, posture, and handedness of the user. In this paper, we define the concept of extended multitouch interaction as a richer input modality that includes all of this information. We further present a practical solution to achieve this on tabletop displays based on mounting a single commodity depth camera above a horizontal surface. This will enable us to not only detect when the surface is being touched, but also recover the user's exact finger and hand posture, as well as distinguish between different users and their handedness. We validate our approach using two user studies, and deploy the technique in a scratchpad tool and in a pen + touch sketch tool.

© All rights reserved Murugappan et al. and/or ACM Press

p. 497-508

Block, Florian, Wigdor, Daniel, Phillips, Brenda Caldwell, Horn, Michael S. and Shen, Chia (2012): FlowBlocks: a multi-touch ui for crowd interaction. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 497-508. Available online

Multi-touch technology lends itself to collaborative crowd interaction (CI). However, common tap-operated widgets are impractical for CI, since they are susceptible to accidental touches and interference from other users. We present a novel multi-touch interface called FlowBlocks in which every UI action is invoked through a small sequence of user actions: dragging parametric UI-Blocks, and dropping them over operational UI-Docks. The FlowBlocks approach is advantageous for CI because it a) makes accidental touches inconsequential; and b) introduces design parameters for mutual awareness, concurrent input, and conflict management. FlowBlocks was successfully used on the floor of a busy natural history museum. We present the complete design space and describe a year-long iterative design and evaluation process which employed the Rapid Iterative Test and Evaluation (RITE) method in a museum setting.

© All rights reserved Block et al. and/or ACM Press

p. 509-518

Rendl, Christian, Greindl, Patrick, Haller, Michael, Zirkl, Martin, Stadlober, Barbara and Hartmann, Paul (2012): PyzoFlex: printed piezoelectric pressure sensing foil. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 509-518. Available online

Ferroelectric material supports both pyro- and piezoelectric effects that can be used for sensing pressures on large, bended surfaces. We present PyzoFlex, a pressure-sensing input device that is based on a ferroelectric material. It is constructed with a sandwich structure of four layers that can be printed easily on any material. We use this material in combination with a high-resolution Anoto-sensing foil to support both hand and pen input tracking. The foil is bendable, energy-efficient, and it can be produced in a printing process. Even a hovering mode is feasible due to its pyroelectric effect. In this paper, we introduce this novel input technology and discuss its benefits and limitations.

© All rights reserved Rendl et al. and/or ACM Press

p. 519-528

Follmer, Sean, Leithinger, Daniel, Olwal, Alex, Cheng, Nadia and Ishii, Hiroshi (2012): Jamming user interfaces: programmable particle stiffness and sensing for malleable and shape-changing devices. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 519-528. Available online

Malleable and organic user interfaces have the potential to enable radically new forms of interactions and expressiveness through flexible, free-form and computationally controlled shapes and displays. This work, specifically focuses on particle jamming as a simple, effective method for flexible, shape-changing user interfaces where programmatic control of material stiffness enables haptic feedback, deformation, tunable affordances and control gain. We introduce a compact, low-power pneumatic jamming system suitable for mobile devices, and a new hydraulic-based technique with fast, silent actuation and optical shape sensing. We enable jamming structures to sense input and function as interaction devices through two contributed methods for high-resolution shape sensing using: 1) index-matched particles and fluids, and 2) capacitive and electric field sensing. We explore the design space of malleable and organic user interfaces enabled by jamming through four motivational prototypes that highlight jamming's potential in HCI, including applications for tabletops, tablets and for portable shape-changing mobile devices.

© All rights reserved Follmer et al. and/or ACM Press

p. 529-536

Sugiura, Yuta, Inami, Masahiko and Igarashi, Takeo (2012): A thin stretchable interface for tangential force measurement. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 529-536. Available online

We have developed a simple skin-like user interface that can be easily attached to curved as well as flat surfaces and used to measure tangential force generated by pinching and dragging interactions. The interface consists of several photoreflectors that consist of an IR LED and a phototransistor and elastic fabric such as stocking and rubber membrane. The sensing method used is based on our observation that photoreflectors can be used to measure the ratio of expansion and contraction of a stocking using the changes in transmissivity of IR light passing through the stocking. Since a stocking is thin, stretchable, and nearly transparent, it can be easily attached to various types of objects such as mobile devices, robots, and different parts of the body as well as to various types of conventional pressure sensors without altering the original shape of the object. It can also present natural haptic feedback in accordance with the amount of force exerted. A system using several such sensors can determine the direction of a two-dimensional force. A variety of example applications illustrated the utility of this sensing system.

© All rights reserved Sugiura et al. and/or ACM Press

p. 537-544

Harrison, Chris, Sato, Munehiko and Poupyrev, Ivan (2012): Capacitive fingerprinting: exploring user differentiation by sensing electrical properties of the human body. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 537-544. Available online

At present, touchscreens can differentiate multiple points of contact, but not who is touching the device. In this work, we consider how the electrical properties of humans and their attire can be used to support user differentiation on touchscreens. We propose a novel sensing approach based on Swept Frequency Capacitive Sensing, which measures the impedance of a user to the environment (i.e., ground) across a range of AC frequencies. Different people have different bone densities and muscle mass, wear different footwear, and so on. This, in turn, yields different impedance profiles, which allows for touch events, including multitouch gestures, to be attributed to a particular user. This has many interesting implications for interactive design. We describe and evaluate our sensing approach, demonstrating that the technique has considerable promise. We also discuss limitations, how these might be overcome, and next steps.

© All rights reserved Harrison et al. and/or ACM Press

p. 545-554

Goel, Mayank, Wobbrock, Jacob and Patel, Shwetak (2012): GripSense: using built-in sensors to detect hand posture and pressure on commodity mobile phones. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 545-554. Available online

We introduce GripSense, a system that leverages mobile device touchscreens and their built-in inertial sensors and vibration motor to infer hand postures including one- or two-handed interaction, use of thumb or index finger, or use on a table. GripSense also senses the amount of pres-sure a user exerts on the touchscreen despite a lack of direct pressure sensors by inferring from gyroscope readings when the vibration motor is "pulsed." In a controlled study with 10 participants, GripSense accurately differentiated device usage on a table vs. in hand with 99.67% accuracy and when in hand, it inferred hand postures with 84.26% accuracy. In addition, GripSense distinguished three levels of pressure with 95.1% accuracy. A usability analysis of GripSense was conducted in three custom applications and showed that pressure input and hand-posture sensing can be useful in a number of scenarios.

© All rights reserved Goel et al. and/or ACM Press

p. 55-62

Rzeszotarski, Jeffrey and Kittur, Aniket (2012): CrowdScape: interactively visualizing user behavior and output. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 55-62. Available online

Crowdsourcing has become a powerful paradigm for accomplishing work quickly and at scale, but involves significant challenges in quality control. Researchers have developed algorithmic quality control approaches based on either worker outputs (such as gold standards or worker agreement) or worker behavior (such as task fingerprinting), but each approach has serious limitations, especially for complex or creative work. Human evaluation addresses these limitations but does not scale well with increasing numbers of workers. We present CrowdScape, a system that supports the human evaluation of complex crowd work through interactive visualization and mixed initiative machine learning. The system combines information about worker behavior with worker outputs, helping users to better understand and harness the crowd. We describe the system and discuss its utility through grounded case studies. We explore other contexts where CrowdScape's visualizations might be useful, such as in user studies.

© All rights reserved Rzeszotarski and Kittur and/or ACM Press

p. 555-562

Hoggan, Eve, Stewart, Craig, Haverinen, Laura, Jacucci, Giulio and Lantz, Vuokko (2012): Pressages: augmenting phone calls with non-verbal messages. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 555-562. Available online

ForcePhone is a mobile synchronous haptic communication system. During phone calls, users can squeeze the side of the device and the pressure level is mapped to vibrations on the recipient's device. The pressure/vibrotactile messages supported by ForcePhone are called pressages. Using a lab-based study and a small field study, this paper addresses the following questions: how can haptic interpersonal communication be integrated into a standard mobile device? What is the most appropriate feedback design for pressages? What types of non-verbal cues can be represented by pressages? Do users make use of pressages during their conversations? The results of this research indicate that such a system has value as a communication channel in real-world settings with users expressing greetings, presence and emotions through pressages.

© All rights reserved Hoggan et al. and/or ACM Press

p. 563-568

Harrison, Chris, Xiao, Robert and Hudson, Scott (2012): Acoustic barcodes: passive, durable and inexpensive notched identification tags. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 563-568. Available online

We present acoustic barcodes, structured patterns of physical notches that, when swiped with e.g., a fingernail, produce a complex sound that can be resolved to a binary ID. A single, inexpensive contact microphone attached to a surface or object is used to capture the waveform. We present our method for decoding sounds into IDs, which handles variations in swipe velocity and other factors. Acoustic barcodes could be used for information retrieval or to triggering interactive functions. They are passive, durable and inexpensive to produce. Further, they can be applied to a wide range of materials and objects, including plastic, wood, glass and stone. We conclude with several example applications that highlight the utility of our approach, and a user study that explores its feasibility.

© All rights reserved Harrison et al. and/or ACM Press

p. 569-578

Fourney, Adam and Terry, Michael (2012): PICL: portable in-circuit learner. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 569-578. Available online

This paper introduces the PICL, the portable in-circuit learner. The PICL explores the possibility of providing standalone, low-cost, programming-by-demonstration machine learning capabilities to circuit prototyping. To train the PICL, users attach a sensor to the PICL, demonstrate example input, then specify the desired output (expressed as a voltage) for the given input. The current version of the PICL provides two learning modes, binary classification and linear regression. To streamline training and also make it possible to train on highly transient signals (such as those produced by a camera flash or a hand clap), the PICL includes a number of input inferencing techniques. These techniques make it possible for the PICL to learn with as few as one example. The PICL's behavioural repertoire can be expanded by means of various output adapters, which serve to transform the output in useful ways when prototyping. Collectively, the PICL's capabilities allow users of systems such as the Arduino or littleBits electronics kit to quickly add basic sensor-based behaviour, with little or no programming required.

© All rights reserved Fourney and Terry and/or ACM Press

p. 579-588

Savage, Valkyrie, Zhang, Xiaohan and Hartmann, Bjrn (2012): Midas: fabricating custom capacitive touch sensors to prototype interactive objects. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 579-588. Available online

An increasing number of consumer products include user interfaces that rely on touch input. While digital fabrication techniques such as 3D printing make it easier to prototype the shape of custom devices, adding interactivity to such prototypes remains a challenge for many designers. We introduce Midas, a software and hardware toolkit to support the design, fabrication, and programming of flexible capacitive touch sensors for interactive objects. With Midas, designers first define the desired shape, layout, and type of touch sensitive areas, as well as routing obstacles, in a sensor editor. From this high-level specification, Midas automatically generates layout files with appropriate sensor pads and routed connections. These files are then used to fabricate sensors using digital fabrication processes, e.g., vinyl cutters and conductive ink printers. Using step-by-step assembly instructions generated by Midas, designers connect these sensors to the Midas microcontroller, which detects touch events. Once the prototype is assembled, designers can define interactivity for their sensors: Midas supports both record-and-replay actions for controlling existing local applications and WebSocket-based event output for controlling novel or remote applications. In a first-use study with three participants, users successfully prototyped media players. We also demonstrate how Midas can be used to create a number of touch-sensitive interfaces.

© All rights reserved Savage et al. and/or ACM Press

p. 589-598

Willis, Karl, Brockmeyer, Eric, Hudson, Scott and Poupyrev, Ivan (2012): Printed optics: 3D printing of embedded optical elements for interactive devices. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 589-598. Available online

We present an approach to 3D printing custom optical elements for interactive devices labelled Printed Optics. Printed Optics enable sensing, display, and illumination elements to be directly embedded in the casing or mechanical structure of an interactive device. Using these elements, unique display surfaces, novel illumination techniques, custom optical sensors, and embedded optoelectronic components can be digitally fabricated for rapid, high fidelity, highly customized interactive devices. Printed Optics is part of our long term vision for interactive devices that are 3D printed in their entirety. In this paper we explore the possibilities for this vision afforded by fabrication of custom optical elements using today's 3D printing technology.

© All rights reserved Willis et al. and/or ACM Press

p. 599-606

Mueller, Stefanie, Lopes, Pedro and Baudisch, Patrick (2012): Interactive construction: interactive fabrication of functional mechanical devices. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 599-606. Available online

Personal fabrication tools, such as laser cutters and 3D printers allow users to create precise objects quickly. However, working through a CAD system removes users from the workpiece. Recent interactive fabrication tools reintroduce this directness, but at the expense of precision. In this paper, we introduce constructable, an interactive drafting table that produces precise physical output in every step. Users interact by drafting directly on the workpiece using a hand-held laser pointer. The system tracks the pointer, beautifies its path, and implements its effect by cutting the workpiece using a fast high-powered laser cutter. Constructable achieves precision through tool-specific constraints, user-defined sketch lines, and by using the laser cutter itself for all visual feedback, rather than using a screen or projection. We demonstrate how constructable allows creating simple but functional devices, including a simple gearbox, that cannot be created with traditional interactive fabrication tools.

© All rights reserved Mueller et al. and/or ACM Press

p. 63-74

Talton, Jerry, Yang, Lingfeng, Kumar, Ranjitha, Lim, Maxine, Goodman, Noah and Mech, Radomr (2012): Learning design patterns with Bayesian grammar induction. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 63-74. Available online

Design patterns have proven useful in many creative fields, providing content creators with archetypal, reusable guidelines to leverage in projects. Creating such patterns, however, is a time-consuming, manual process, typically relegated to a few experts in any given domain. In this paper, we describe an algorithmic method for learning design patterns directly from data using techniques from natural language processing and structured concept learning. Given a set of labeled, hierarchical designs as input, we induce a probabilistic formal grammar over these exemplars. Once learned, this grammar encodes a set of generative rules for the class of designs, which can be sampled to synthesize novel artifacts. We demonstrate the method on geometric models and Web pages, and discuss how the learned patterns can drive new interaction mechanisms for content creators.

© All rights reserved Talton et al. and/or ACM Press

p. 75-82

Wu, Min, Bhowmick, Arin and Goldberg, Joseph (2012): Adding structured data in unstructured web chat conversation. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 75-82. Available online

Web chat is becoming the primary customer contact channel in customer relationship management (CRM), and Question/Answer/Lookup (QAL) is the dominant communication pattern in CRM agent-to-customer chat. Text-based web chat for QAL has two main usability problems. Chat transcripts between agents and customers are not tightly integrated into agent-side applications, requiring customer service agents to re-enter customer typed data. Also, sensitive information posted in chat sessions in plain text raises security concerns. The addition of web form widgets to web chat not only solves both of these problems but also adds new usability benefits to QAL. Forms can be defined beforehand or, more flexibly, dynamically composed. Two preliminary user studies were conducted. An agent-side study showed that adding inline forms to web chat decreased overall QAL completion time by 47 percent and increased QAL accuracy by removing all potential human errors. A customer-side study showed that web chat with inline forms is intuitive to customers.

© All rights reserved Wu et al. and/or ACM Press

p. 83-92

Banovic, Nikola, Grossman, Tovi, Matejka, Justin and Fitzmaurice, George (2012): Waken: reverse engineering usage information and interface structure from software videos. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 83-92. Available online

We present Waken, an application-independent system that recognizes UI components and activities from screen captured videos, without any prior knowledge of that application. Waken can identify the cursors, icons, menus, and tooltips that an application contains, and when those items are used. Waken uses frame differencing to identify occurrences of behaviors that are common across graphical user interfaces. Candidate templates are built, and then other occurrences of those templates are identified using a multi-phase algorithm. An evaluation demonstrates that the system can successfully reconstruct many aspects of a UI without any prior application-dependant knowledge. To showcase the design opportunities that are introduced by having this additional meta-data, we present the Waken Video Player, which allows users to directly interact with UI components that are displayed in the video.

© All rights reserved Banovic et al. and/or ACM Press

p. 93-102

Chi, Pei-Yu, Ahn, Sally, Ren, Amanda, Dontcheva, Mira, Li, Wilmot and Hartmann, Bjrn (2012): MixT: automatic generation of step-by-step mixed media tutorials. In: Proceedings of the 2012 ACM Symposium on User Interface Software and Technology 2012. pp. 93-102. Available online

Users of complex software applications often learn concepts and skills through step-by-step tutorials. Today, these tutorials are published in two dominant forms: static tutorials composed of images and text that are easy to scan, but cannot effectively describe dynamic interactions; and video tutorials that show all manipulations in detail, but are hard to navigate. We hypothesize that a mixed tutorial with static instructions and per-step videos can combine the benefits of both formats. We describe a comparative study of static, video, and mixed image manipulation tutorials with 12 participants and distill design guidelines for mixed tutorials. We present MixT, a system that automatically generates step-by-step mixed media tutorials from user demonstrations. MixT segments screencapture video into steps using logs of application commands and input events, applies video compositing techniques to focus on salient information, and highlights interactions through mouse trails. Informal evaluation suggests that automatically generated mixed media tutorials were as effective in helping users complete tasks as tutorials that were created manually.

© All rights reserved Chi et al. and/or ACM Press




User-contributed notes

Give us your opinion! Do you have any comments/additions
that you would like other visitors to see?

 
comment You (your email) say: Aug 25th, 2014
#1
Aug 25
Add a thoughtful commentary or note to this page ! 
 

your homepage, facebook profile, twitter, or the like
will be spam-protected
How many?
= e.g. "6"
User ExperienceBy submitting you agree to the Site Terms
 
 
 
 

Changes to this page (conference)

23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Added
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified
23 Nov 2012: Modified

Page Information

Page maintainer: The Editorial Team
This is a tablet-optimized version of http://www.interaction-design.org/references/conferences/proceedings_of_the_2012_acm_symposium_on_user_interface_software_and_technology.html

Upcoming Courses

go to course
Quality Web Communication: The Beginner's Guide
Starts the day after tomorrow !
go to course
User-Centred Design - Module 2
87% booked. Starts in 8 days
 
 

Featured chapter

Marc Hassenzahl explains the fascinating concept of User Experience and Experience Design. Commentaries by Don Norman, Eric Reiss, Mark Blythe, and Whitney Hess

User Experience and Experience Design !

 
 

Our Latest Books

 
 
Gamification at Work: Designing Engaging Business Software
by Janaki Mythily Kumar and Mario Herger
start reading
 
 
 
 
The Social Design of Technical Systems: Building technologies for communities
by Brian Whitworth and Adnan Ahmad
start reading
 
 
 
 
The Encyclopedia of Human-Computer Interaction, 2nd Ed.
by Mads Soegaard and Rikke Friis Dam
start reading
 
 

Upcoming Courses

go to course
Quality Web Communication: The Beginner's Guide
Starts the day after tomorrow !
go to course
User-Centred Design - Module 2
87% booked. Starts in 8 days