Publication statistics

Pub. period:1983-2012
Pub. count:21
Number of co-authors:25


Number of publications with 3 favourite co-authors:

Ying Yin:
Tevfik Metin Sezgin:
Tracy A. Hammond:



Productive colleagues

Randall Davis's 3 most productive colleagues in number of publications:

James A. Landay:91
Pamela Samuelson:55
Robert C. Miller:42

Upcoming Courses

go to course
Get Your First Job as a UX or Interaction Designer
go to course
Information Visualization: Getting Dashboards Right
90% booked. Starts in 5 days

Featured chapter

Marc Hassenzahl explains the fascinating concept of User Experience and Experience Design. Commentaries by Don Norman, Eric Reiss, Mark Blythe, and Whitney Hess

User Experience and Experience Design !


Our Latest Books

The Glossary of Human Computer Interaction
by Mads Soegaard and Rikke Friis Dam
start reading
The Social Design of Technical Systems: Building technologies for communities. 2nd Edition
by Brian Whitworth and Adnan Ahmad
start reading
Gamification at Work: Designing Engaging Business Software
by Janaki Mythily Kumar and Mario Herger
start reading
The Social Design of Technical Systems: Building technologies for communities
by Brian Whitworth and Adnan Ahmad
start reading
The Encyclopedia of Human-Computer Interaction, 2nd Ed.
by Mads Soegaard and Rikke Friis Dam
start reading

Randall Davis


Publications by Randall Davis (bibliography)

 what's this?
Edit | Del

Song, Yale, Morency, Louis-Philippe and Davis, Randall (2012): Multimodal human behavior analysis: learning correlation and interaction across modalities. In: Proceedings of the 2012 International Conference on Multimodal Interfaces 2012. pp. 27-30.

Multimodal human behavior analysis is a challenging task due to the presence of complex nonlinear correlations and interactions across modalities. We present a novel approach to this problem based on Kernel Canonical Correlation Analysis (KCCA) and Multi-view Hidden Conditional Random Fields (MV-HCRF). Our approach uses a nonlinear kernel to map multimodal data to a high-dimensional feature space and finds a new projection of the data that maximizes the correlation across modalities. We use a multi-chain structured graphical model with disjoint sets of latent variables, one set per modality, to jointly learn both view-shared and view-specific sub-structures of the projected data, capturing interaction across modalities explicitly. We evaluate our approach on a task of agreement and disagreement recognition from nonverbal audio-visual cues using the Canal 9 dataset. Experimental results show that KCCA makes capturing nonlinear hidden dynamics easier and MV-HCRF helps learning interaction across modalities.

© All rights reserved Song et al. and/or ACM Press

Edit | Del

Hammond, Tracy and Davis, Randall (2010): Creating the perception-based LADDER sketch recognition language. In: Proceedings of DIS10 Designing Interactive Systems 2010. pp. 141-150.

Sketch recognition is automated understanding of hand-drawn diagrams. Current sketch recognition systems exist for only a handful of domains, which contain on the order of 10-20 shapes. Our goal was to create a generalized method for recognition that could work for many domains, increasing the number of shapes that could be recognized in real-time, while maintaining a high accuracy. In an effort to effectively recognize shapes while allowing drawing freedom (both drawing-style freedom and perceptually-valid variations), we created the shape description language modeled after the way people naturally describe shapes to 1) create an intuitive and easy to understand description, providing transparency to the underlying recognition process, and 2) to improve recognition by providing recognition flexibility (drawing freedom) that is aligned with how humans perceive shapes. This paper describes the results of a study performed to see how users naturally describe shapes. A sample of 35 subjects described or drew approximately 16 shapes each. Results show a common vocabulary related to Gestalt grouping and singularities. Results also show that perception, similarity, and context play an important role in how people describe shapes. This study resulted in a language (LADDER) that allows shape recognizers for any domain to be automatically generated from a single hand-drawn example of each shape. Sketch systems for over 30 different domains have been automatically generated based on this language. The largest domain contained 923 distinct shapes, and achieved a recognition accuracy of 83% (and

© All rights reserved Hammond and Davis and/or their publisher

Edit | Del

Correa, Andrew, Walter, Matthew R., Fletcher, Luke, Glass, Jim, Teller, Seth and Davis, Randall (2010): Multimodal interaction with an autonomous forklift. In: Proceedings of the 5th ACM/IEEE International Conference on Human Robot Interaction 2010. pp. 243-250.

We describe a multimodal framework for interacting with an autonomous robotic forklift. A key element enabling effective interaction is a wireless, handheld tablet with which a human supervisor can command the forklift using speech and sketch. Most current sketch interfaces treat the canvas as a blank slate. In contrast, our interface uses live and synthesized camera images from the forklift as a canvas, and augments them with object and obstacle information from the world. This connection enables users to "draw on the world," enabling a simpler set of sketched gestures. Our interface supports commands that include summoning the forklift and directing it to lift, transport, and place loads of palletized cargo. We describe an exploratory evaluation of the system designed to identify areas for detailed study. Our framework incorporates external signaling to interact with humans near the vehicle. The robot uses audible and visual annunciation to convey its current state and intended actions. The system also provides seamless autonomy handoff: any human can take control of the robot by entering its cabin, at which point the forklift can be operated manually until the human exits.

© All rights reserved Correa et al. and/or their publisher

Edit | Del

Yin, Ying and Davis, Randall (2010): Toward natural interaction in the real world: real-time gesture recognition. In: Proceedings of the 2010 International Conference on Multimodal Interfaces 2010. p. 15.

Using a new hand tracking technology capable of tracking 3D hand postures in real-time, we developed a recognition system for continuous natural gestures. By natural gestures, we mean those encountered in spontaneous interaction, rather than a set of artificial gestures chosen to simplify recognition. To date we have achieved 95.6% accuracy on isolated gesture recognition, and 73% recognition rate on continuous gesture recognition, with data from 3 users and twelve gesture classes. We connected our gesture recognition system to Google Earth, enabling real time gestural control of a 3D map. We describe the challenges of signal accuracy and signal interpretation presented by working in a real-world environment, and detail how we overcame them.

© All rights reserved Yin and Davis and/or ACM Press

Edit | Del

Hammond, Tracy A. and Davis, Randall (2009): Recognizing interspersed sketches quickly. In: Proceedings of the 2009 Conference on Graphics Interface 2009. pp. 157-166.

Sketch recognition is the automated recognition of hand-drawn diagrams. When allowing users to sketch as they would naturally, users may draw shapes in an interspersed manner, starting a second shape before finishing the first. In order to provide freedom to draw interspersed shapes, an exponential combination of subshapes must be considered. Because of this, most sketch recognition systems either choose not to handle interspersing, or handle only a limited pre-defined amount of interspersing. Our goal is to eliminate such interspersing drawing constraints from the sketcher. This paper presents a high-level recognition algorithm that, while still exponential, allows for complete interspersing freedom, running in near real-time through early effective sub-tree pruning. At the core of the algorithm is an indexing technique that takes advantage of geometric sketch recognition techniques to index each shape for efficient access and fast pruning during recognition. We have stress-tested our algorithm to show that the system recognizes shapes in less than a second even with over a hundred candidate subshapes on screen.

© All rights reserved Hammond and Davis and/or their publisher

Edit | Del

Sezgin, Tevfik Metin and Davis, Randall (2008): Sketch recognition in interspersed drawings using time-based graphical models. In Computers & Graphics, 32 (5) pp. 500-510.

Edit | Del

Davis, Randall (2007): Magic Paper: Sketch-Understanding Research. In IEEE Computer, 40 (9) pp. 34-41.

Edit | Del

Sezgin, Tevfik Metin and Davis, Randall (2007): Sketch Interpretation Using Multiscale Models of Temporal Patterns. In IEEE Computer Graphics and Applications, 27 (1) pp. 28-37.

Edit | Del

Hammond, Tracy and Davis, Randall (2006): Interactive learning of structural shape descriptions from automatically generated near-miss examples. In: Proceedings of the 2006 International Conference on Intelligent User Interfaces 2006. pp. 210-217.

Sketch interfaces provide more natural interaction than the traditional mouse and palette tool, but can be time consuming to build if they have to be built anew for each new domain. A shape description language, such as the LADDER language we created, can significantly reduce the time necessary to create a sketch interface by enabling automatic generation of the interface from a domain description. However, structural shape descriptions, whether written by users or created automatically by the computer, are frequently over- or under- constrained. We present a technique to debug over- and under-constrained shapes using a novel form of active learning that generates its own suspected near-miss examples. Using this technique we implemented a graphical debugging tool for use by sketch interface developers.

© All rights reserved Hammond and Davis and/or ACM Press

Edit | Del

Hammond, Tracy and Davis, Randall (2005): LADDER, a sketching language for user interface developers. In Computers & Graphics, 29 (4) pp. 518-532.

Edit | Del

Stahovich, Thomas F., Davis, Randall, Miller, Robert C., Landay, James A. and Saund, Eric (2005): Pen-based computing. In Computers & Graphics, 29 (4) pp. 477-479.

Edit | Del

Adler, Aaron and Davis, Randall (2004): Speech and sketching for multimodal design. In: Nunes, Nuno Jardim and Rich, Charles (eds.) International Conference on Intelligent User Interfaces 2004 January 13-16, 2004, Funchal, Madeira, Portugal. pp. 214-216.

While sketches are commonly and effectively used in the early stages of design, some information is far more easily conveyed verbally than by sketching. In response, we have combined sketching with speech, enabling a more natural form of communication. We studied the behavior of people sketching and speaking, and from this derived a set of rules for segmenting and aligning the signals from both modalities. Once the inputs are aligned, we use both modalities in interpretation. The result is a more natural interface to our system.

© All rights reserved Adler and Davis and/or ACM Press

Edit | Del

Alvarado, Christine and Davis, Randall (2004): SketchREAD: a multi-domain sketch recognition engine. In: Proceedings of the 2004 ACM Symposium on User Interface Software and Technology 2004. pp. 23-32.

We present SketchREAD, a multi-domain sketch recognition engine capable of recognizing freely hand-drawn diagrammatic sketches. Current computer sketch recognition systems are difficult to construct, and either are fragile or accomplish robustness by severely limiting the designer\'s drawing freedom. Our system can be applied to a variety of domains by providing structural descriptions of the shapes in that domain; no training data or programming is necessary. Robustness to the ambiguity and uncertainty inherent in complex, freely-drawn sketches is achieved through the use of context. The system uses context to guide the search for possible interpretations and uses a novel form of dynamically constructed Bayesian networks to evaluate these interpretations. This process allows the system to recover from low-level recognition errors (e.g., a line misclassified as an arc) that would otherwise result in domain level recognition errors. We evaluated Sketch-READ on real sketches in two domains -- family trees and circuit diagrams -- and found that in both domains the use of context to reclassify low-level shapes significantly reduced recognition error over a baseline system that did not reinterpret low-level classifications. We also discuss the system\'s potential role in sketch based user interfaces.

© All rights reserved Alvarado and Davis and/or ACM Press

Edit | Del

Eisenstein, Jacob and Davis, Randall (2004): Visual and linguistic information in gesture classification. In: Sharma, Rajeev, Darrell, Trevor, Harper, Mary P., Lazzari, Gianni and Turk, Matthew (eds.) Proceedings of the 6th International Conference on Multimodal Interfaces - ICMI 2004 October 13-15, 2004, State College, PA, USA. pp. 113-120.

Edit | Del

Uzuner, zlem, Davis, Randall and Katz, Boris (2004): Using Empirical Methods for Evaluating Expression and Content Similarity. In: HICSS 2004 2004. .

Edit | Del

Eisenstein, Jacob and Davis, Randall (2004): Visual and linguistic information in gesture classification. In: Proceedings of the 2004 International Conference on Multimodal Interfaces 2004. pp. 113-120.

Classification of natural hand gestures is usually approached by applying pattern recognition to the movements of the hand. However, the gesture categories most frequently cited in the psychology literature are fundamentally multimodal; the definitions make reference to the surrounding linguistic context. We address the question of whether gestures are naturally multimodal, or whether they can be classified from hand-movement data alone. First, we describe an empirical study showing that the removal of auditory information significantly impairs the ability of human raters to classify gestures. Then we present an automatic gesture classification system based solely on an n-gram model of linguistic context; the system is intended to supplement a visual classifier, but achieves 66% accuracy on a three-class classification problem on its own. This represents higher accuracy than human raters achieve when presented with the same information.

© All rights reserved Eisenstein and Davis and/or their publisher

Edit | Del

Foltz, Mark A. and Davis, Randall (2001): Query by Attention: Visually Searchable Information Maps. In: IV 2001 2001. pp. 85-.

Edit | Del

Davis, Randall (2001): The digital dilemma. In Communications of the ACM, 44 (2) pp. 77-83.

Edit | Del

Davis, Randall, Samuelson, Pamela, Kapor, Mitchell and Reichman, Jerome (1996): A New View of Intellectual Property and Software. In Communications of the ACM, 39 (3) pp. 21-30.

Edit | Del

Davis, Randall (1983): Reasoning from First Principles in Electronic Troubleshooting. In International Journal of Man-Machine Studies, 19 (5) pp. 403-423.

While expert systems have traditionally been built using large collections of rules based on empirical associations, interest has grown recently in the use of systems that reason "from first principles", i.e. from an understanding of causality of the device being examined. Our work explores the use of such models in troubleshooting digital electronics. In discussing troubleshooting we show why the traditional approach -- test generation -- solves a different problem and we discuss a number of its practical shortcomings. We consider next the style of debugging known as discrepancy detection and demonstrate why it is a fundamental advance over traditional test generation. Further exploration, however, demonstrates that in its standard form discrepancy detection encounters interesting limits in dealing with commonly known classes of faults. We suggest that the problem arises from a number of interesting implicit assumptions typically made when using the technique. In discussing how to repair the problems uncovered, we argue for the primacy of models of causal interaction, rather than the traditional fault models. We point out the importance of making these models explicit, separated from the troubleshooting mechanism, and retractable in much the same sense that inferences are retracted in current systems. We report on progress to date in implementing this approach and demonstrate the diagnosis of a bridge fault -- a traditionally difficult problem -- using our approach.

© All rights reserved Davis and/or Academic Press

Edit | Del

Davis, Randall and Shrobe, Howard E. (1983): Representing Structure and Behavior of Digital Hardware. In IEEE Computer, 16 (10) pp. 75-82.

Add publication
Show list on your website

Join our community and advance:




Join our community!

Page Information

Page maintainer: The Editorial Team