About the author

Picture of Alan Blackwell. © Alan Blackwell
I only have one big research question, but I attack it from a lot of different angles. The question is representation. How do people make, see and use things that carry meaning? The angles from which I attack my question include various ways in which representations are applied (including design processes, interacting with technology, computer programming, visualisation), vario...   
Read more >> 

Commentaries by:

 

Ben Shneiderman

Picture of Ben Shneiderman.

Ben Shneiderman is a Professor in the Department of Computer Science, Founding Director (1983-2000) of the Human-Computer Interaction Laboratory, and Member of the Institute for Advanced Computer Studies at the University of Maryland at College Park. He has ta...

More about Ben >>
 

Clive Richards

Picture of Clive Richards.

Clive Richards, MPhil PhD(RCA) FCSD FRSA, is a Visiting Professor to the Faculty of Arts and Architecture, University of Brighton. He has worked in a wide range of art and design fields, including commercial practice (technical illustration, information graphi...

More about Clive >>
 

Peter C-H. Cheng

Picture of Peter C-H. Cheng.

Peter Cheng’s main research interest is in the nature of representational systems, spanning the design of external representations and the cognitive processes that deal with internal mental representations. Knowledge rich representations used for higher forms...

More about Peter >>
 

Brad A. Myers

Picture of Brad A. Myers.

Brad A. Myers is a Professor in the Human-Computer Interaction Institute in the School of Computer Science at Carnegie Mellon University. He is an ACM Fellow, winner of six best paper awards, and a member of the CHI Academy, an honor bestowed on the principal ...

More about Brad >>
 

Editors of this chapter:

 

Mads Soegaard

Picture of Mads Soegaard. Copyright of Mads Soegaard and Interaction-Design.org through the Creative Commons Share-Alike licence.

Previously, I've worked at The Danish National Technological Institute worki...

Read more >>
 

Rikke Friis Dam

Picture of Rikke Friis Dam. Copyright of Rikke Friis Dam and Interaction-Design.org through the Creative Commons Share-Alike licence.

Rikke Dam holds a Master's Degree in philosophy from the University of Aarhus, a...

Read more >>
 

Peer Reviewers

Reviewer 1: Name suppressed
Reviewer 2: Name suppressed

Peer-review is based on the reviewing guidelines and coordinated by the Reviewing Board.

Get Notified!

Get notified when new chapters are added to the encyclopedia!

Your Email
 

Featured chapter

Authoritative overview of Social Computing by Tom Erickson - veteran researcher at IBM Research Lab. It includes 9 HD videos filmed in Copenhagen.

Read Thomas's chapter

5. Visual Representation

 

How can you design computer displays that are as meaningful as possible to human viewers? Answering this question requires understanding of visual representation - the principles by which markings on a surface are made and interpreted. The analysis in this article addresses the most important principles of visual representation for screen design, introduced with examples from the early history of graphical user interfaces. In most cases, these principles have been developed and elaborated within whole fields of study and professional skill - typography, cartography, engineering and architectural draughting, art criticism and semiotics. Improving on the current conventions requires serious skill and understanding. Nevertheless, interaction designers should be able, when necessary, to invent new visual representations.

Video 5.1: Introduction to Visual Representation by Alan Blackwell.
Video 5.2: Alan Blackwell on applying theories of Visual Representation.

5.1 Typography and text

For many years, computer displays resembled paper documents. This does not mean that they were simplistic or unreasonably constrained. On the contrary, most aspects of modern industrial society have been successfully achieved using the representational conventions of paper, so those conventions seem to be powerful ones. Information on paper can be structured using tabulated columns, alignment, indentation and emphasis, borders and shading. All of those were incorporated into computer text displays. Interaction conventions, however, were restricted to operations of the typewriter rather than the pencil. Each character typed would appear at a specific location. Locations could be constrained, like filling boxes on a paper form. And shortcut command keys could be defined using onscreen labels or paper overlays. It is not text itself, but keyboard interaction with text that is limited and frustrating compared to what we can do with paper (Sellen & Harper 2001).

But despite the constraints on keyboard interaction, most information on computer screens is still represented as text. Conventions of typography and graphic design help us to interpret that text as if it were on a page, and human readers benefit from many centuries of refinement in text document design. Text itself, including many writing systems as well as specialised notations such as algebra, is a visual representation that has its own research and educational literature. Documents that contain a mix of bordered or coloured regions containing pictures, text and diagrammatic elements can be interpreted according to the conventions of magazine design, poster advertising, form design, textbooks and encyclopaedias. Designers of screen representations should take care to properly apply the specialist knowledge of those graphic and typographic professions. Position on the page, use of typographic grids, and genre-specific illustrative conventions should all be taken into account.

Contemporary example from the grid system website
Figure 5.1: Contemporary example from the grid system website.
Example of a symbolic algebra expression (the single particle solution to Schrodinger's equation)
Figure 5.2: Example of a symbolic algebra expression (the single particle solution to Schrodinger's equation).
Table layout of funerals from the plague in London in 1665
Figure 5.3: Table layout of funerals from the plague in London in 1665.
Tabular layout of the first page of the Gutenberg Bible: Volume 1, Old Testament, Epistle of St. Jerome. The Gutenberg Bible was printed by Johannes Gutenberg, in Mainz, Germany in the 1450s
Figure 5.4: Tabular layout of the first page of the Gutenberg Bible: Volume 1, Old Testament, Epistle of St. Jerome. The Gutenberg Bible was printed by Johannes Gutenberg, in Mainz, Germany in the 1450s.

5.1.1 Summary

Most screen-based information is interpreted according to textual and typographic conventions, in which graphical elements are arranged within a visual grid, occasionally divided or contained with ruled and coloured borders.
Where to learn more:

  • thegridsystem.org
  • Resnick, Elizabeth (2003): Design for Communication: Conceptual Graphic Design Basics. Wiley
    Design for Communication offers a unique approach to mastering the basic design principles, conceptual problem-solving methods, and critical-thinking skills that distinguish graphic designers from desktop technicians. This book presents forty-two basic to advanced graphic design and typography assignments collaboratively written by college educators to teach the fundamental processes, concepts, and techniques through hands-on applications. Each assignment is illustrated with actual student solutions, and each includes a process narrative and an educator's critical analysis revealing the reasoning behind the creative strategies employed by each individual student solution.

    © All rights reserved Resnick and/or Wiley


5.2 Maps and graphs

The computer has, however, also acquired a specialised visual vocabulary and conventions. Before the text-based computer terminal (or 'glass teletype') became ubiquitous, cathode ray tube displays were already used to display oscilloscope waves and radar echoes. Both could be easily interpreted because of their correspondence to existing paper conventions. An oscilloscope uses a horizontal time axis to trace variation of a quantity over time, as pioneered by William Playfair in his 1786 charts of the British economy. A radar screen shows direction and distance of objects from a central reference point, just as the Hereford Mappa Mundi of 1300 organised places according to their approximate direction and distance from Jerusalem. Many visual displays on computers continue to use these ancient but powerful inventions - the map and the graph. In particular, the first truly large software project, the SAGE air defense system, set out to present data in the form of an augmented radar screen - an abstract map, on which symbols and text could be overlaid. The first graphics computer, the Lincoln Laboratory Whirlwind, was created to show maps, not text.

The technique invented by William Playfair, for visual representation of time series data.
Figure 5.5: The technique invented by William Playfair, for visual representation of time series data.
Time series data as shown on an oscilloscope screen
Figure 5.6: Time series data as shown on an oscilloscope screen.
Early radar screen from HMS Belfast built in 1936
Figure 5.7: Early radar screen from HMS Belfast built in 1936.
Early weather radar - Hurricane Abby approaching the coast of British Honduras in 1960
Figure 5.8: Early weather radar - Hurricane Abby approaching the coast of British Honduras in 1960.
The Hereford Mappa Mundi of 1300 organised places according to their approximate direction and distance from Jerusalem
Figure 5.9: The Hereford Mappa Mundi of 1300 organised places according to their approximate direction and distance from Jerusalem.
The SAGE system in use. The SAGE system used light guns as interaction devices.
Figure 5.10: The SAGE system in use. The SAGE system used light guns as interaction devices.
The Whirlwind computer at the MIT Lincoln Laboratory
Figure 5.11: The Whirlwind computer at the MIT Lincoln Laboratory.

5.2.1 Summary

Basic diagrammatic conventions rely on quantitative correspondence between a direction on the surface and a continuous quantity such as time or distance. These should follow established conventions of maps and graphs.
Where to learn more:

MacEachren, Alan M. (2004): How Maps Work: Representation, Visualization, and Design. The Guilford Press

5.3 Schematic drawings

Ivan Sutherland's groundbreaking PhD research with Whirlwind's successor TX-2 introduced several more sophisticated alternatives (Sutherland 1963). The use of a light pen allowed users to draw arbitrary lines, rather than relying on control keys to select predefined options. An obvious application, in the engineering context of Massachusetts Institute of Technology (MIT) where Sutherland worked, was to make engineering drawings such as the girder bridge in figure 13. Lines on the screen are scaled versions of the actual girders, and text information can be overlaid to give details of force calculations. Plans of this kind, as a visual representation, are closely related to maps. However, where the plane of a map corresponds to a continuous surface, engineering drawings need not be continuous. Each set of connected components must share the same scale, but white space indicates an interpretive break, so that independent representations can potentially share the same divided surface - a convention introduced in Diderot's encyclopedia of 1772, which showed pictures of multiple objects on a page, but cut them loose from any shared pictorial context.

The TX-2 graphics computer, running Ivan Sutherland's Sketchpad software
Figure 5.12: The TX-2 graphics computer, running Ivan Sutherland's Sketchpad software.
An example of a force diagram created using Sutherland's Sketchpad
Figure 5.13: An example of a force diagram created using Sutherland's Sketchpad.
A page from the Encyclopédie of Diderot and d'Alembert, combining pictorial elements with diagrammatic lines and categorical use of white space.
Figure 5.14: A page from the Encyclopédie of Diderot and d'Alembert, combining pictorial elements with diagrammatic lines and categorical use of white space.

5.3.1 Summary

Engineering drawing conventions allow schematic views of connected components to be shown in relative scale, and with text annotations labelling the parts. White space in the representation plane can be used to help the reader distinguish elements from each other rather than directly representing physical space.
Where to learn more:

  • Engineering draughting textbooks
  • Ferguson, Eugene S. (1994): Engineering and the Mind's Eye. MIT Press

5.4 Pictures

The examples so far may seem rather abstract. Isn't the most 'natural' visual representation simply a picture of the thing you are trying to represent? In that case, what is so hard about design? Just point a camera, and take the picture. It seems like pictures are natural and intuitive, and anyone should be able to understand what they mean. Of course, you might want the picture to be more or less artistic, but that isn't a technical concern, is it? Well, Ivan Sutherland also suggested the potential value that computer screens might offer as artistic tools. His Sketchpad system was used to create a simple animated cartoon of a winking girl. We can use this example to ask whether pictures are necessarily 'natural', and what design factors are relevant to the selection or creation of pictures in an interaction design context.

We would not describe Sutherland's girl as 'realistic', but it is an effective representation of a girl. In fact, it is an unusually good representation of a winking girl, because all the other elements of the picture are completely abstract and generic. It uses a conventional graphic vocabulary of lines and shapes that are understood in our culture to represent eyes, mouths and so on - these elements do not draw attention to themselves, and therefore highlight the winking eye. If a realistic picture of an actual person was used instead, other aspects of the image (the particular person) might distract the viewer from this message.

Sutherland's 'Winking Girl' drawing, created with the Sketchpad system.
Figure 5.15: Sutherland's 'Winking Girl' drawing, created with the Sketchpad system.

It is important, when considering the design options for pictures, to avoid the 'resemblance fallacy', i.e. that drawings are able to depict real object or scenes because the viewer's perception of the flat image simulates the visual perception of a real scene. In practice, all pictures rely on conventions of visual representation, and are relatively poor simulations of natural engagement with physical objects, scenes and people. We are in the habit of speaking approvingly of some pictures as more 'realistic' than others (photographs, photorealistic ray-traced renderings, 'old master' oil paintings), but this simply means that they follow more rigorously a particular set of conventions. The informed designer is aware of a wide range of pictorial conventions and options.

As an example of different pictorial conventions, consider the ways that scenes can be rendered using different forms of artistic perspective. The invention of linear perspective introduced a particular convention in which the viewer is encouraged to think of the scene as perceived through a lens or frame while holding his head still, so that nearby objects occupy a disproportionate amount of the visual field. Previously, pictorial representations more often varied the relative size of objects according to their importance - a kind of 'semantic' perspective. Modern viewers tend to think of the perspective of a camera lens as being most natural, due to the ubiquity of photography, but we still understand and respect alternative perspectives, such as the isometric perspective of the pixel art group eBoy, which has been highly influential on video game style.

Example of an early work by Masaccio, demonstrating a 'perspective' in which relative size shows symbolic importance
Figure 5.16: Example of an early work by Masaccio, demonstrating a 'perspective' in which relative size shows symbolic importance.
Example of the strict isometric perspective used by the eBoy group
Figure 5.17: Example of the strict isometric perspective used by the eBoy group.
Masaccio's mature work The Tribute Money, demonstrating linear perspective
Figure 5.18: Masaccio's mature work The Tribute Money, demonstrating linear perspective.

As with most conventions of pictorial representation, new perspective rendering conventions are invented and esteemed for their accuracy by critical consensus, and only more slowly adopted by untrained readers. The consensus on preferred perspective shifts across cultures and historical periods. It would be naďve to assume that the conventions of today are the final and perfect product of technical evolution. As with text, we become so accustomed to interpreting these representations that we are blind to the artifice. But professional artists are fully aware of the conventions they use, even where they might have mechanical elements - the way that a photograph is framed changes its meaning, and a skilled pencil drawing is completely unlike visual edge-detection thresholds. A good pictorial representation need not simulate visual experience any more than a good painting of a unicorn need resemble an actual unicorn. When designing user interfaces, all of these techniques are available for use, and new styles of pictorial rendering are constantly being introduced.

5.4.1 Summary

Pictorial representations, including line drawings, paintings, perspective renderings and photographs rely on shared interpretive conventions for their meaning. It is naďve to treat screen representations as though they were simulations of experience in the physical world.
Where to learn more:

  • Micklewright, Keith (2005): Drawing: Mastering the Language of Visual Expression. Harry N. Abrams
    Drawing is a language, a necessary skill for anyone who wants to express ideas or feelings in written images. Like all languages, it can be mastered with practice and instruction. Author Keith Micklewright distills a lifetime of hard thinking about drawing, presenting techniques-along with exercises-that help us become fluent at visual communication. The advantage of his approach is that drawing is seen as a flexible form of expression rather than a set of mechanical skills. There is no right way to draw creatively, anymore than there is one style of writing creatively. To drive this point home, Micklewright illustrates the book with marvelous drawings by great artists, from Old Masters to the present, that range from precise portraiture to ecstatic nature studies. There is no other book on the subject that combines such a deep, lucid text with such a generous collection of inspirational art.

    © All rights reserved Micklewright and/or Harry N. Abrams


  • Stroebel, Leslie, Todd, Hollis and Zakia, Richard (1979): Visual Concepts for Photographers. Focal Press

5.5 Node-and-link diagrams

The first impulse of a computer scientist, when given a pencil, seems to be to draw boxes and connect them with lines. These node and link diagrams can be analysed in terms of the graph structures that are fundamental to the study of algorithms (but unrelated to the visual representations known as graphs or charts). A predecessor of these connectivity diagrams can be found in electrical circuit schematics, where the exact location of components, and the lengths of the wires, can be arranged anywhere, because they are irrelevant to the circuit function. Another early program created for the TX-2, this time by Ivan Sutherland's brother Bert, allowed users to create circuit diagrams of this kind. The distinctive feature of a node-and-link connectivity diagram is that, since the position of each node is irrelevant to the operation of the circuit, it can be used to carry other information. Marian Petre's research into the work of electronics engineers (Petre 1995) catalogued the ways in which they positioned components in ways that were meaningful to human readers, but not to the computer - like the blank space between Diderot's objects this is a form of 'secondary notation' - use of the plane to assist the reader in ways not related to the technical content.

Circuit connectivity diagrams have been most widely popularised through the London Underground diagram, an invention of electrical engineer Henry Beck. The diagram clarified earlier maps by exploiting the fact that most underground travellers are only interested in order and connectivity, not location, of the stations on the line. (Sadly, the widespread belief that a 'diagram' will be technical and hard to understand means that most people describe this as the London Undergound 'map', despite Beck's insistence on his original term).

Henry Beck's London Underground Diagram (1933)
Figure 5.19: Henry Beck's London Underground Diagram (1933).
Node and link diagram of the kind often drawn by computing professionals
Figure 5.20: Node and link diagram of the kind often drawn by computing professionals.
Map of the London Underground network, as it was printed before the design of Beck's diagram (1932).
Figure 5.21: Map of the London Underground network, as it was printed before the design of Beck's diagram (1932).

5.5.1 Summary

Node and link diagrams are still widely perceived as being too technical for broad acceptance. Nevertheless, they can present information about ordering and relationships clearly, especially if consideration is given to the value of allowing human users to specify positions.
Where to learn more:

  • Diagrammatic representation books
  • Lowe, Ric (1992): Successful Instructional Diagram.

5.6 Icons and symbols

Maps frequently use symbols to indicate specific kinds of landmark. Sometimes these are recognisably pictorial (the standard symbols for tree and church), but others are fairly arbitrary conventions (the symbol for a railway station). As the resolution of computer displays increased in the 1970s, a greater variety of symbols could be differentiated, by making them more detailed, as in the MIT SDMS (Spatial Data Management System) that mapped a naval battle scenario with symbols for different kinds of ship. However, the dividing line between pictures and symbols is ambiguous. Children's drawings of houses often use conventional symbols (door, four windows, triangle roof and chimney) whether or not their own house has two storeys, or a fireplace. Letters of the Latin alphabet are shapes with completely arbitrary relationship to their phonetic meaning, but the Korean phonetic alphabet is easier to learn because the forms mimic the shape of the mouth when pronouncing those sounds. The field of semiotics offers sophisticated ways of analysing the basis on which marks correspond to meanings. In most cases, the best approach for an interaction designer is simply to adopt familiar conventions. When these do not exist, the design task is more challenging.

It is unclear which of the designers working on the Xerox Star coined the term 'icon' for the small pictures symbolising different kinds of system object. David Canfield Smith winningly described them as being like religious icons, which he said were pictures standing for (abstract) spiritual concepts. But 'icon' is also used as a technical term in semiotics. Unfortunately, few of the Xerox team had a sophisticated understanding of semiotics. It was fine art PhD Susan Kare's design work on the Apple Macintosh that established a visual vocabulary which has informed the genre ever since. Some general advice principles are offered by authors such as Horton (1994), but the successful design of icons is still sporadic. Many software publishers simply opt for a memorable brand logo, while others seriously misjudge the kinds of correspondence that are appropriate (my favourite blooper was a software engineering tool in which a pile of coins was used to access the 'change' command).

It has been suggested that icons, being pictorial, are easier to understand than text, and that pre-literate children, or speakers of different languages, might thereby be able to use computers without being able to read. In practice, most icons simply add decoration to text labels, and those that are intended to be self-explanatory must be supported with textual tooltips. The early Macintosh icons, despite their elegance, were surprisingly open to misinterpretation. One PhD graduate of my acquaintance believed that the Macintosh folder symbol was a briefcase (the folder tag looked like a handle), which allowed her to carry her files from place to place when placed inside it. Although mistaken, this belief never caused her any trouble - any correspondence can work, so long as it is applied consistently.

In art, the term Icon (from Greek, eikon, "image") commonly refers to religious paintings in Eastern Orthodox, Oriental Orthodox, and Eastern-rite Catholic jurisdictions. Here a 6th-century encaustic icon from Saint Catherine's Monastery, Mount Sinai
Figure 5.22: In art, the term Icon (from Greek, eikon, "image") commonly refers to religious paintings in Eastern Orthodox, Oriental Orthodox, and Eastern-rite Catholic jurisdictions. Here a 6th-century encaustic icon from Saint Catherine's Monastery, Mount Sinai.
In computing, David Canfield Smith described computer icons as being like religious icons, which he said were pictures standing for (abstract) spiritual concepts.
Figure 5.23: In computing, David Canfield Smith described computer icons as being like religious icons, which he said were pictures standing for (abstract) spiritual concepts.

5.6.1 Summary

The design of simple and memorable visual symbols is a sophisticated graphic design skill. Following established conventions is the easiest option, but new symbols must be designed with an awareness of what sort of correspondence is intended - pictorial, symbolic, metonymic (e.g. a key to represent locking), bizarrely mnemonic, but probably not monolingual puns.
Where to learn more:

Napoles, Veronica (1987): Corporate Identity Design.

5.7 Visual metaphor

The ambitious graphic designs of the Xerox Star/Alto and Apple Lisa/Macintosh were the first mass-market visual interfaces. They were marketed to office professionals, making the 'cover story' that they resembled an office desktop a convenient explanatory device. Of course, as was frequently noted at the time, these interfaces behaved nothing like a real desktop. The mnemonic symbol for file deletion (a wastebasket) was ridiculous if interpreted as an object placed on a desk. And nobody could explain why the desk had windows in it (the name was derived from the 'clipping window' of the graphics architecture used to implement them - it was at some later point that they began to be explained as resembling sheets of paper on a desk). There were immediate complaints from luminaries such as Alan Kay and Ted Nelson that strict analogical correspondence to physical objects would become obstructive rather than instructive. Nevertheless, for many years the marketing story behind the desktop metaphor was taken seriously, despite the fact that all attempts to improve the Macintosh design with more elaborate visual analogies, as in General Magic and Microsoft Bob, subsequently failed.

The 'desktop' can be far more profitably analysed (and extended) by understanding the representational conventions that it uses. The size and position of icons and windows on the desktop has no meaning, they are not connected, and there is no visual perspective, so it is neither a map, graph nor picture. The real value is the extent to which it allows secondary notation, with the user creating her own meaning by arranging items as she wishes. Window borders separate areas of the screen into different pictorial, text or symbolic contexts as in the typographic page design of a textbook or magazine. Icons use a large variety of conventions to indicate symbolic correspondence to software operations and/or company brands, but they are only occasionally or incidentally organised into more complex semiotic structures.

Apple marketed the visual metaphor in 1983 as a key benefit of the Lisa computer. This advertisement said 'You can work with Lisa the same familiar way you work at your desk'. However a controlled study by Carroll and Mazur (1986) found that the claim for immediately familiar operation may have been exaggerated.
Figure 5.24: Apple marketed the visual metaphor in 1983 as a key benefit of the Lisa computer. This advertisement said 'You can work with Lisa the same familiar way you work at your desk'. However a controlled study by Carroll and Mazur (1986) found that the claim for immediately familiar operation may have been exaggerated.
The Xerox Alto and Apple Lisa, early products in which bitmapped displays allowed pictorial icons to be used as mnemonic cues within the 'desktop metaphor'
Figure 5.25: The Xerox Alto and Apple Lisa, early products in which bitmapped displays allowed pictorial icons to be used as mnemonic cues within the 'desktop metaphor'
Apple Lisa
Figure 5.26: Apple Lisa.

5.7.1 Summary

Theories of visual representation, rather than theories of visual metaphor, are the best approach to explaining the conventional Macintosh/Windows 'desktop'. There is huge room for improvement.
Where to learn more:

Blackwell, Alan (2006): The reification of metaphor as a design tool. In ACM Transactions on Computer-Human Interaction, 13 (4) pp. 490-530
Despite causing many debates in human-computer interaction (HCI), the term "metaphor" remains a central element of design practice. This article investigates the history of ideas behind user-interface (UI) metaphor, not only technical developments, but also less familiar perspectives from education, philosophy, and the sociology of science. The historical analysis is complemented by a study of attitudes toward metaphor among HCI researchers 30 years later. Working from these two streams of evidence, we find new insights into the way that theories in HCI are related to interface design, and offer recommendations regarding approaches to future UI design research.

© All rights reserved Blackwell and/or ACM Press

5.8 Unified theories of visual representation

The analysis in this article has addressed the most important principles of visual representation for screen design, introduced with examples from the early history of graphical user interfaces. In most cases, these principles have been developed and elaborated within whole fields of study and professional skill - typography, cartography, engineering and architectural draughting, art criticism and semiotics. Improving on the current conventions requires serious skill and understanding. Nevertheless, interaction designers should be able, when necessary, to invent new visual representations.

One approach is to take a holistic perspective on visual language, information design, notations, or diagrams. Specialist research communities in these fields address many relevant factors from low-level visual perception to critique of visual culture. Across all of them, it can be necessary to ignore (or not be distracted by) technical and marketing claims, and to remember that all visual representations simply comprise marks on a surface that are intended to correspond to things understood by the reader. The two dimensions of the surface can be made to correspond to physical space (in a map), to dimensions of an object, to a pictorial perspective, or to continuous abstract scales (time or quantity). The surface can also be partitioned into regions that should be interpreted differently. Within any region, elements can be aligned, grouped, connected or contained in order to express their relationships. In each case, the correspondence between that arrangement, and the intended interpretation, must be understood by convention, explained, or derived from the structural and perceptual properties of marks on the plane. Finally, any individual element might be assigned meaning according to many different semiotic principles of correspondence.

The following table summarises holistic views, as introduced above, drawing principally on the work of Bertin, Richards, MacEachren, Blackwell & Engelhardt and Engelhardt.
Where to learn more:

Engelhardt, Yuri (2002). The Language of Graphics. A framework for the analysis of syntax and meaning in maps, charts and diagrams (PhD Thesis). University of Amsterdam
 

Graphic Resources

Correspondence

Design Uses

Marks

Shape
Orientation
Size
Texture
Saturation
Colour
Line

Literal (visual imitation of physical features)
Mapping (quantity, relative scale)
Conventional (arbitrary)

Mark position, identify category (shape, texture colour)
Indicate direction (orientation, line)
Express magnitude (saturation, size, length)
Simple symbols and colour codes

Symbols

Geometric elements
Letter forms
Logos and icons
Picture elements
Connective elements

Topological (linking)
Depictive (pictorial conventions)
Figurative (metonym, visual puns)
Connotative (professional and cultural association)
Acquired (specialist literacies)

Texts and symbolic calculi
Diagram elements
Branding
Visual rhetoric
Definition of regions

Regions

Alignment grids
Borders and frames
Area fills
White space
Gestalt integration

Containment
Separation
Framing (composition, photography)
Layering

Identifying shared membership
Segregating or nesting multiple surface conventions in panels
Accommodating labels, captions or legends

Surfaces

The plane
Material object on which the marks are imposed (paper, stone)
Mounting, orientation and display context
Display medium

Literal (map)
Euclidean (scale and angle)
Metrical (quantitative axes)
Juxtaposed or ordered (regions, catalogues)
Image-schematic
Embodied/situated

Typographic layouts
Graphs and charts
Relational diagrams
Visual interfaces
Secondary notations
Signs and displays

Table 5.1: Summary of the ways in which graphical representations can be applied in design, via different systems of correspondence.
 
 
bubbles long map
Table 5.2: Screenshot from the site gapminder.org, illustrating a variety of correspondence conventions used in different parts of the page.
 
 

As an example of how one might analyse (or working backwards, design) a complex visual representation, consider the case of musical scores. These consist of marks on a paper surface, bound into a multi-page book, that is placed on a stand at arms length in front of a performer. Each page is vertically divided into a number of regions, visually separated by white space and grid alignment cues. The regions are ordered, with that at the top of the page coming first. Each region contains two quantitative axes, with the horizontal axis representing time duration, and the vertical axis pitch. The vertical axis is segmented by lines to categorise pitch class. Symbols placed at a given x-y location indicate a specific pitched sound to be initiated at a specific time. A conventional symbol set indicates the duration of the sound. None of the elements use any variation in colour, saturation or texture. A wide variety of text labels and annotation symbols are used to elaborate these basic elements. Music can be, and is, also expressed using many other visual representations (see e.g. Duignan for a survey of representations used in digital music processing).

5.9 Where to learn more

The historical examples of early computer representations used in this article are mainly drawn from Sutherland (Ed. Blackwell and Rodden 2003), Garland (1994), and Blackwell (2006). Historical reviews of visual representation in other fields include Ferguson (1992), Pérez-Gómez & Pelletier (1997), McCloud (1993), Tufte (1983). Reviews of human perceptual principles can be found in Gregory (1970), Ittelson (1996), Ware (2004), Blackwell (2002). Advice on principles of interaction with visual representation is distributed throughout the HCI literature, but classics include Norman (1988), Horton (1994), Shneiderman ( Shneiderman & Plaisant 2009, Card et al 1999, Bederson & Shneiderman 2003) and Spence (2001). Green's Cognitive Dimensions of Notations framework has for many years provided a systematic classification of the design parameters in interactive visual representations. A brief introduction is provided in Blackwell & Green (2003).

Research on visual representation topics is regularly presented at the Diagrams conference series (which has a particular emphasis on cognitive science), the InfoDesign and Vision Plus conferences (which emphasise graphic and typographic information design), the Visual Languages and Human-Centric Computing symposia (emphasising software tools and development), and the InfoVis and Information Visualisation conferences (emphasising quantitative and scientific data visualisation).

IV - International Conference on Information Visualization

 

DIAGRAMS - International Conference on the Theory and Application of Diagrams

Next conference is coming up 02 Jul 2012 in Canterbury, UK

VL-HCC - Symposium on Visual Languages and Human Centric Computing

 
 
Next conference is coming up 30 Sep 2012 in Innsbruck, Austria

InfoVis - IEEE Symposium on Information Visualization

 
 
 

5.11 Commentary by Ben Shneiderman

How to cite this commentary in your report

Picture of Ben Shneiderman. Copyright unknown.
Ben Shneiderman is a Professor in the Department of Computer Science, Founding Director (1983-2000) of the Human-Computer Interaction Laboratory, and Member of the Institute for Advanced Computer Studies at the University of Maryland at College Park. He has taught previously at the State University of New York and at Indiana University. He was made a Fellow of the ACM in 1997, elected a ...   
Read more >> 

Since computer displays are such powerful visual appliances, careful designers devote extensive effort to getting the visual representation right. They have to balance the demands of many tasks, diverse users, and challenging requirements, such as short learning time, rapid performance, low error rates, and good retention over time. Designing esthetic interfaces that please and even delight users is a further expectation that designers must meet to be successful. For playful and discretionary tasks esthetic concerns may dominate, but for life critical tasks, rapid performance with low error rates are essential.

Alan Blackwell's competent description of many visual representation issues is a great start for newcomers with helpful reminders even for experienced designers. The videos make for a pleasant personal accompaniment that bridges visual representation for interface design with thoughtful analyses of representational art. Blackwell's approach might be enriched by more discussion of visual representations in functional product design tied to meaningful tasks. Learning from paintings of Paris is fine, but aren't there other lessons to learn from visual representations in airport kiosks, automobile dashboards, or intensive care units?

These devices as well as most graphical user interfaces and mobile devices raise additional questions of changing state visualization and interaction dynamics. Modern designers need to do more than show the right phone icon, they need to show ringing, busy, inactive, no network, conference mode, etc., which may include color changes (highlighted, grayed out), animations, and accompanying sounds. These designers also need to deal with interactive visual representations that happen with a click, double-click, right-click, drag, drag-and-drop, hover, multi-select, region-select, brushing-linking, and more.

The world of mobile devices such as phones, cameras, music players, or medical sensors is the new frontier for design, where visual representations are dynamic and tightly integrated with sound, haptics, and novel actions such as shaking, twisting, or body movements. Even more challenging is the expectation that goes beyond the solitary viewer to the collaboration in which multiple users embedded in a changing physical environment produce new visual representations.

These changing and interactive demands on designers invite creative expressions that are very different from designs for static signs, printed diagrams, or interpretive art. The adventure for visual representation designers is to create a new language of interaction that engages users, accelerates learning, provides comprehensible feedback, and offers appropriate warnings when dangers emerge. Blackwell touches on some of these issues in the closing Gapminder example, but I was thirsty for more.

 
 

5.12 Commentary by Clive Richards

How to cite this commentary in your report

Picture of Clive Richards. © Clive Richards
Clive Richards, MPhil PhD(RCA) FCSD FRSA, is a Visiting Professor to the Faculty of Arts and Architecture, University of Brighton. He has worked in a wide range of art and design fields, including commercial practice (technical illustration, information graphics, corporate identity, typography and artists' catalogue design), academic research, undergraduate and postgraduate teaching, PhD...   
Read more >> 

If I may be permitted a graphically inspired metaphor Alan Blackwell provides us with a neat pen sketch of that extensive scene called 'visual representation' (Blackwell 2011).

"Visualisation has a lot more to offer than most people are aware of today" we are told by Robert Kosara at the end of his commentary (Kosara 2010) on Stephen Few's related article on 'Data visualisation for human perception' (Few 2010). Korsara is right, and Blackwell maps out the broad territory in which many of these visualisation offerings may be located. In this commentary I offer a few observations on some prominent features in that landscape: dynamics, picturing, semiotics and metaphor.

Dynamics

Ben Shneiderman's critique of Blackwell's piece points to a lack of attention to "... additional questions of changing state visualisations and interaction dynamics" (Shneiderman 2010). Indeed the possibilities offered by these additional questions present some exciting challenges for interaction designers - opportunities to create novel and effective combinations of visual with other sensory and motor experiences in dynamic operational contexts. Shneiderman suggests that: "These changing and interactive demands on designers invite creative expressions that are very different from design for static signs, printed diagrams, or interpretive art". This may be so up to a point, but here Shneinderman and I part company a little. The focus of Blackwell's essay is properly on the visual representation side of facilities available to interaction designers, and in that context he is quite right to give prominence to highly successful but static visual representation precedents, and also to point out the various specialist fields of endeavour in which they have been developed. Some of these representational approaches have histories reaching back thousands of years and are deeply embedded within our culture. It would be foolhardy to disregard conventions established in, say, the print domain, and to try to re-invent everything afresh for the screen, even if this were a practical proposition. Others have made arguments to support looking to historical precedents. For example Michael Twyman has pointed out that when considering typographic cueing and "... the problems of the electronic age ... we have much to learn from the manuscript age" (Twyman 1987, p5). He proposes that studying the early scribes' use of colour, spacing and other graphical devices can usefully inform the design of today's screen-based texts. And as Blackwell points out in his opening section on 'Typography and text' "most information on computer screen is still presented as text".

It is also sometimes assumed that the pictorial representation of a dynamic process is best presented dynamically. However it can be argued that the comic book convention of using a sequence of static frames is sometimes superior for focusing the viewer's attention on the critical events in a process, rather than using an animated sequence in which key moments may be missed. This is of course not to deny the immense value of the moving and interactive visual image in the right context. The Gapminder charts are a case in point (http://www.gapminder.org). Blackwell usefully includes one of these, but as a static presentation. These diagrams come to life and really tell their story through the clustering of balloons that inflate or deflate as they move about the screen when driven through simulated periods of time.

While designing a tool for engineers to learn about the operation and maintenance of an oil system for an aircraft jet engine, Detlev Fischer devised a series of interactive animations, called 'Cinegrams' to display in diagrammatic form various operating procedures (Fischer and Richards 1995). He used the cinematic techniques of time compression and expansion in one animated sequence to show how the slow accumulation of debris in an oil filter, over an extended period of time, would eventually create a blockage to the oil flow and trigger the opening of a by-pass device in split seconds. Notwithstanding my earlier comment about the potential superiority of the comic strip genre for displaying some time dependant processes this particular Cinegram proved very instructive for the targeted users. There are many other examples one could cite where dynamic picturing of this sort has been deployed to similarly good effect in interactive environments.

Picturing

Shneinderman also comments that: "Blackwell's approach might be enriched by more discussion of visual representation in functional product design tied to meaningful tasks". An area I have worked in is the pictorial representation of engineering assemblies to show that which is normally hidden from view. Techniques to do this on the printed page include 'ghosting' (making occluding parts appear as if transparent), 'exploding' (showing components separately, set out in dis-assembly order along an axis) and cutting away (taking a slice out of an outer shell to reveal mechanisms beneath). All these three-dimensional picturing techniques were used by, if not actually invented by, Leonardo Da Vinci (Richards 2006). All could be enhanced by interactive viewer control - an area of further fruitful exploration for picturing purposes in technical documentation contexts.

Blackwell's section on 'Pictures' warns us that when considering picturing options to avoid the "resemblance fallacy" pointing out the role that convention plays, even in so called photo-realistic images. He also points out that viewers can be distracted from the message by incidental information in 'realistic' pictures. From my own work in the field I know that technical illustrators' synoptic black and white outline depictions are regarded as best for drawing the viewer's attention to the key features of a pictorial representation. Research in this area has shown that when using linear perspective type drawings the appropriate deployment of lines of varying 'weight', rather than of a single thickness, can have a significant effect on viewers' levels of understanding about what is depicted (Richards, Bussard and Newman 2007). This work was done specifically to determine an 'easy to read' visual representational style when manipulating on the screen images of CAD objects. The most effective convention was shown to be: thin lines for edges where both planes forming the edge are visible and thicker lines for edges where only one plane is visible - that is where an outline edge forms a kind of horizon to the object.

These line thickness conventions appear on the face of it to have little to do with how we normally perceive the world, and Blackwell tells us that: "A good pictorial representation need not simulate visual experience any more than a good painting of a unicorn need resemble an actual unicorn". And some particular representations of unicorns can aid our understanding of how to use semiotic theory to figure out how pictures may be interpreted and, importantly, sometimes misunderstood - as I shall describe in the following.

Semiotics

Blackwell mentions semiotics, almost in passing, however it can help unravel some of the complexities of visual representation. Evelyn Goldsmith uses a Charles Addams cartoon to explain the relevance of the 'syntactic', 'semantic' and 'pragmatic' levels of semiotic analysis when applied to pictures (Goldsmith 1978). The cartoon in question, like many of those by Charles Addams, has no caption. It shows two unicorns standing on a small island in the pouring rain forlornly watching the Ark sailing away into the distance. Goldsmith suggests that most viewers will have little trouble in interpreting the overlapping elements in the scene, for example that one unicorn is standing behind the other, nor any difficulty understanding that the texture gradient of the sea stands for a receding horizontal plane. These represent the syntactic level of interpretation. Most adults will correctly identify the various components of the picture at the semantic level, however Goldsmith proposes that a young child might mistake the unicorns for horses and be happy with 'boat' for the Ark. But at the pragmatic level of interpretation, unless a viewer of the picture is aware of the story of Noah's Ark, the joke will be lost  - the connection will not be made between the scene depicted in the drawing and the scarcity of unicorns. This reinforces the point that one should not assume that the understanding of pictures is straightforward. There is much more to it than a simple matter or recognition. This is especially the case when metaphor is involved in visual representation.

Metaphor

Blackwell's section on 'Visual metaphor' is essentially a critique of the use of "theories of visual metaphor" as an "approach to explaining the conventional Mackintosh/Windows 'desktop' ". His is a convincing argument but there is much more which may be said about the use of visual metaphor - especially to show that which otherwise cannot be pictured. In fact most diagrams employ a kind of spatial metaphor when not depicting physical arrangements, for example when using the branches of a tree to represent relations within a family (Richards 2002). The capability to represent the invisible is the great strength of the visual metaphor, but there are dangers, and here I refer back to semiotics and particularly the pragmatic level of analysis. One needs to know the story to get the picture.

In our parental home, one of the many books much loved by my two brothers and me, was The Practical Encyclopaedia for Children (Odhams circa 1948). In it a double page spread illustration shows the possible evolutionary phases of the elephant. These are depicted as a procession of animals in a primordial swamp cum jungle setting. Starting with a tiny fish and passing to a small aquatic creature climbing out of the water onto the bank the procession progresses on through eight phases of transformation, including the Moeritherium and the Paleomatodon, finishing up with the land-based giant of today's African Elephant. Recently one of my brothers confessed to me that through studying this graphical diorama he had believed as a child that the elephant had a life cycle akin to that of a frog. He had understood that the procession was a metaphor for time. He had just got the duration wrong - by several orders of magnitude. He also hadn't understood that each separate depiction was of a different animal. He had used the arguably more sophisticated concept that it was the same animal at different times and stages in its individual development.

Please forgive the cliché if I say that this anecdote clearly illustrates that there can be more to looking at a picture than meets the eye? Blackwell's essay provides some useful pointers for exploring the possibilities of this fascinating territory of picturing and visual representation in general.   

References

  • Blackwell A 2011 'Visual representation' Interaction-Design.org
  • Few S 2010 'Data visualisation for human perception' Interaction-Design.org
  • Fischer D and Richards CJ 1995 'The presentation of time in interactive animated systems diagrams' In: Earnshaw RA and Vince JA (eds) Multimedia Systems and Applications London: Academic Press Ltd (pp141 - 159). ISBN 0-12-227740-6
  • Goldsmith E 1978 An analysis of the elements affecting comprehensibility of illustrations intended as supportive of textPhD thesis (CNAA) Brighton Polytechnic
  • Korsa R 2010 'Commentary on Stephen Few's article: Data visualisation for human perception' Interaction-Design.org
    Odhams c. 1949 The practical encyclopaedia for children (pp 194 - 195)
  • Richards CJ 2002 'The fundamental design variables of diagramming' In: Oliver P, Anderson M and Meyer B (eds) Diagrammatic representation and reasoning London: Springer Verlag (pp 85 - 102) ISBN 1-85233-242-5
  • Richards CJ 2006 'Drawing out information - lines of communication in technical illustration' Information Design Journal 14 (2) 93 - 107
  • Richards CJ, Bussard N, Newman R 2007 'Weighing up line weights: the value of differing line thicknesses in technical illustrations' Information Design Journal 15 (2) 171 - 181
  • Shneiderman B 2011 'Commentary on Alan Blackwell's article: Visual representation' Interaction-Design.org
  • Twyman M 1982 'The graphic representation of language' Information Design Journal 3 (1) 2 - 22
 
 

5.13 Commentary by Peter C-H. Cheng

How to cite this commentary in your report

Picture of Peter C-H. Cheng. © Peter C-H. Cheng
Peter Cheng’s main research interest is in the nature of representational systems, spanning the design of external representations and the cognitive processes that deal with internal mental representations. Knowledge rich representations used for higher forms of cognition (e.g., complex problem solving, discovery, conceptual learning) in conceptually demanding domains are a particular i...   
Read more >> 

Alan Blackwell has provided us with a fine introduction to the design of visual representations. The article does a great job in motivating the novice designer of visual representations to explore some of the fundamental issues that lurk just beneath the surface of creating effective representations.  Furthermore, he gives us all quite a challenge:

Improving on the current conventions requires serious skill and understanding. Nevertheless, interaction designers should be able, when necessary, to invent new visual representations.

 

Alan, quite rightly, claims that we must consider the fundamental principles of symbolic correspondence, if we are to design new genres of visual representations beyond the common forms of displays and interfaces.  The report begins to equip the novice visual representation designer with an understanding of the nature of symbolic correspondence between the components of visual representations and the things they represent, whether objects, actions or ideas.  In particular, it gives a useful survey of how correspondence works in a range of representations and provides a systematic framework of how systems of correspondence can be applied to design. The interactive screen shot is an exemplary visual representation that vividly reveals the correspondence techniques used in each part of the example diagram.

However, suppose you really wished to rise to the challenge of creating novel visual representations, how far will a knowledge of the fundamentals of symbolic correspondence take you? Drawing on my studies of the role of diagrams in the history of science, experience of inventing novel visual representations and research on problem solving and learning with diagrams, from the perspective of Cognitive Science, my view is that such knowledge will be necessary but not sufficient for your endeavours.  So, what else should the budding visual representation designer consider? From the perspective of cognitive science there are at least three aspects that we may profitably target.

First, there is the knowledge of how human process information; specifically the nature of the human cognitive architecture. By this, I mean more than visual perception, but an understanding of how we mentally receive, store, retrieve, transform and transmit information. The way the mind deals with each of these basic types of information processing provides relevant constrains for the design of visual representations. For instance, humans often, perhaps even typically, encode concepts in the form of hierarchies of schemas, which are information structures that coordinate attributes that describe and differentiate classes of concepts. These hierarchies of schemas underpin our ability to efficiently generalize or specialize concepts. Hence, we can use this knowledge to consider whether particular forms of symbolic correspondence will assist or hinder the forms of inference that we hope the user of the representation may make. For example, are the main symbolic correspondences in a visual representation consistent with the key attributes of the schemas for the concepts being considered?

Second, it may be useful for the designer to consider the broader nature of the tasks that the user may wish to do with the designed representation.  Resource allocation, optimization, calculating quantities, inferences about of possible outcomes, classification, reasoning about extreme or special cases, and debugging: these are just a few of the many possibilities. These tasks are more generic than the information-oriented options considered in the 'design uses' column of Figure 27 in the article. They are worth addressing, because they provide constraints for the initial stages of representation design, by narrowing the search for what are likely to be effective correspondences to adopt. For example, if taxonomic classification is important, then separation and layering will be important correspondences; whereas magnitude calculations may demand scale mapping, Euclidian and metrical correspondences.

The third aspect concerns situations in which the visual representation must support not just a single task, but many diverse tasks. For example, a visual representation to help students learn about electricity will be used to explain the topology of circuits, make computations with electrical quantities, provide explanations of circuit behaviour (in terms of formal algebraic models and as qualitative causal models), facilitate fault finding or trouble shooting, among other activities. The creation of novel representations in such circumstances is perhaps one of the most challenging for designers. So, what knowledge can help? In this case, I advocate attempting to design representations on the basis of an analysis of the underlying conceptual structure of the knowledge of the target domain. Why? Because the nature of the knowledge is invariant across different classes of task. For example, for problem solving and learning of electricity, all the tasks depend upon the common fundamental conceptual structures of the domain that knit together the laws governing the physical properties of electricity and circuit topology. Hence, a representation that makes these concepts readily available through effective representation designed will probably be effective for a wide range of tasks.

In summary, it is desirable for the aspiring visual representation designer to consider symbolic correspondence, but I recommend they cast their net more widely for inspiration by learning about the human cognitive architecture, focusing on the nature of the task for which they are designing, and most critically thinking about the underlying conceptual structure of the knowledge of the target domain.

 
 

5.14 Commentary by Brad A. Myers

How to cite this commentary in your report

Picture of Brad A. Myers. © Brad A. Myers
Brad A. Myers is a Professor in the Human-Computer Interaction Institute in the School of Computer Science at Carnegie Mellon University. He is an ACM Fellow, winner of six best paper awards, and a member of the CHI Academy, an honor bestowed on the principal leaders of the field. He is the principal investigator for the Natural Programming Project and the Pebbles Handheld Computer Proje...   
Read more >> 

I have been teaching human-computer interaction to students with a wide range of backgrounds for many years. One of the most difficult areas for them to learn seems to be visual design. Students seem to quickly pick up rules like Nielsen's Heuristics for interaction (Nielsen & Molich, 1990), whereas the guidelines for visual design are much more subtle. Alan Blackwell's article presents many useful points, but a designer needs to know so much more! Whereas students can achieve competence at achieving Nielsen's "consistency and standards," for example, they struggle with selecting an appropriate representation for their information. And only a trained graphic designer is likely to be able to create an attractive and effective icon. Some people have a much better aesthetic sense, and can create much more beautiful and appropriate representations. A key goal of my introductory course, therefore, is to try to impart to the students how difficult it is to do visual design, and how wide the set of choices is. Studying the examples that Blackwell provides will give the reader a small start towards effective visual representations, but the path requires talent, study, and then iterative design and testing to evaluate and improve a design's success.

References

  • Nielsen, J., & Molich, R. (1990). Heuristic evaluation of user interfaces. Paper presented at the Proc. ACM CHI'90 Conf, Seattle, WA, 249-256.
  • See also: http://www.useit.com/papers/heuristic/heuristic_list.html
 

5.10 Behind the scenes

Interviewing Alan in his home, close to the university. ...

Cambridge is a charming town with a great atmosphere. ...

The typical tourist picture of King's College, University of...

Filming at Kettle's Yard Gallery. ...

Filming at Kettle's Yard Gallery. ...

Eating a well-deserved lunch after two days of interviewing....

Adjusting the audio of the different scenes and denoising. ...

Adjusting the audio of the different scenes and denoising. ...

Editing the video. Co...

 

5.15 User-contributed notes

Give us your opinion! Do you have any comments/additions
that you would like other visitors to see?

 
comment You (your email) say: May 7th, 2012
#1
Add a thoughtful commentary or note to this page ! 

your homepage, facebook profile, twitter, or the like
will be spam-protected
How many?
= e.g. "6"
By submitting you agree to the Site Terms
 

5.16 References

 what's this?

Anderson, Michael, Meyer, Bernd and Olivier, Patrick (2002): Diagrammatic Representation and Reasoning. London, UK,

Bederson, Benjamin B. and Shneiderman, Ben (2003): The Craft of Information Visualization: Readings and Reflections. Morgan Kaufman Publishers

Bertin, Jacques (1967): Semiology of Graphics: Diagrams, Networks, Maps (Sémiologie graphique: Les diagrammes - Les réseaux - Les cartes). English translation by W. J. Berg. Madison, WI, USA, University of Wisconsin Press

Blackwell, Alan (2002): Psychological perspectives on diagrams and their users. In: Anderson, Michael, Meyer, Bernd and Olivier, Patrick (eds.). "Diagrammatic Representation and Reasoning". London, UK: pp. 109-123

Blackwell, Alan (2006): The reification of metaphor as a design tool. In ACM Transactions on Computer-Human Interaction, 13 (4) pp. 490-530

Despite causing many debates in human-computer interaction (HCI), the term "metaphor" remains a central element of design practice. This article investigates the history of ideas behind user-interface (UI) metaphor, not only technical developments, but also less familiar perspectives from education, philosophy, and the sociology of science. The historical analysis is complemented by a study of attitudes toward metaphor among HCI researchers 30 years later. Working from these two streams of evidence, we find new insights into the way that theories in HCI are related to interface design, and offer recommendations regarding approaches to future UI design research.

© All rights reserved Blackwell and/or ACM Press

Blackwell, Alan and Engelhardt, Yuri (2002): A Meta-Taxonomy for Diagram Research. In: Anderson, Michael, Meyer, Bernd and Olivier, Patrick (eds.). "Diagrammatic Representation and Reasoning". London, UK: pp. 47-64

The recent rise of multimedia technology has turned visual communication into an everyday reality and made it necessary to achieve a better understanding of the role of diagrams and sketches in communication and in creative thought and problem-solving. Diagrammatic Representation and Reasoning is a wide-ranging, multidisciplinary overview of this area, covering relevant research in computer science, artificial intelligence, cognitive science and psychology. Key topics include: - Cognitive aspects of diagrammatic information; - Formal methods for computing with diagrams; - Applications of advanced diagrammatic systems. This book is a state-of-the-art survey that will be a valuable resource for researchers and students in the fields of cognitive science, artificial intelligence, human-computer interaction, and graphics and visualisation.

© All rights reserved Blackwell and Engelhardt and/or their publisher


Blackwell, Alan and Green, T. R. G. (2003): Notational Systems - The Cognitive Dimensions of Notations Framework. In: Carroll, John M. (ed.). "HCI Models, Theories, and Frameworks". San Francisco: Morgan Kaufman Publisherspp. 103-133

Carroll, John M. and Mazur, Sandra A. (1986): LisaLearning. In Computer, 19 (11) pp. 35-49

Engelhardt, Yuri (2002). The Language of Graphics. A framework for the analysis of syntax and meaning in maps, charts and diagrams (PhD Thesis). University of Amsterdam

Ferguson, Eugene S. (1994): Engineering and the Mind's Eye. MIT Press

Garland, Ken (1994): Mr. Beck's Underground Map. Capital Transport Publishing

Goodman, Nelson (1976): Languages of Art. Hackett Publishing Company

Gregory, Richard L. (1970): The Intelligent Eye. London, Weidenfeld and Nicolson

Horton, William (1994): The Icon Book: Visual Symbols for Computer Systems and Documentation. John Wiley and Sons

Ittelson, W. H. (1996): Visual perception of markings. In Psychonomic Bulletin & Review, 3 (2) pp. 171-187

Lowe, Ric (1992): Successful Instructional Diagram.

MacEachren, Alan M. (2004): How Maps Work: Representation, Visualization, and Design. The Guilford Press

Mccloud, Scott (1994): Understanding Comics: The Invisible Art. Harper Paperbacks

Micklewright, Keith (2005): Drawing: Mastering the Language of Visual Expression. Harry N. Abrams

Drawing is a language, a necessary skill for anyone who wants to express ideas or feelings in written images. Like all languages, it can be mastered with practice and instruction. Author Keith Micklewright distills a lifetime of hard thinking about drawing, presenting techniques-along with exercises-that help us become fluent at visual communication. The advantage of his approach is that drawing is seen as a flexible form of expression rather than a set of mechanical skills. There is no right way to draw creatively, anymore than there is one style of writing creatively. To drive this point home, Micklewright illustrates the book with marvelous drawings by great artists, from Old Masters to the present, that range from precise portraiture to ecstatic nature studies. There is no other book on the subject that combines such a deep, lucid text with such a generous collection of inspirational art.

© All rights reserved Micklewright and/or Harry N. Abrams


Napoles, Veronica (1987): Corporate Identity Design.

Norman, Donald A. (1988): The Design of Everyday Things. New York, Doubleday

Petre, Marian (1995): Why Looking Isn't Always Seeing: Readership Skills and Graphical Programming. In Communications of the ACM, 38 (6) pp. 33-44

Pérez-Gómez, Alberto and Pelletier, Louise (1997): Architectural Representation and the Perspective Hinge. MIT Press

The relationship between the architectural representation and its intended product--a building--has undergone a profound transformation over the centuries. Before the age of modern technology, the systematically predictive role of architectural drawing so taken for granted today was less dominant in the evolution from architectural idea to built work. The age of computer-aided design has brought with it a stricter standard of fidelity. However, contemporary architecture need not simply accept the inevitability of a technological imperative. This book demonstrates that representation is never a neutral tool or mere picture of a future building. Writing from inside the discipline of architecture, rather than from the more common extrapolations from the history of painting and philosophy, Alberto Pérez-Gómez and Louise Pelletier focus on the implications of the tool of perspective (and the hegemony of vision) for architectural representation. Their primary thesis is that tools of representation have a direct influence on the conceptual development of projects and generation of forms, and that there are alternatives to the reductive working methods of most contemporary practice.

© All rights reserved Pérez-Gómez and Pelletier and/or MIT Press


Resnick, Elizabeth (2003): Design for Communication: Conceptual Graphic Design Basics. Wiley

Design for Communication offers a unique approach to mastering the basic design principles, conceptual problem-solving methods, and critical-thinking skills that distinguish graphic designers from desktop technicians. This book presents forty-two basic to advanced graphic design and typography assignments collaboratively written by college educators to teach the fundamental processes, concepts, and techniques through hands-on applications. Each assignment is illustrated with actual student solutions, and each includes a process narrative and an educator's critical analysis revealing the reasoning behind the creative strategies employed by each individual student solution.

© All rights reserved Resnick and/or Wiley


Richards, Clive (1984). Diagrammatics: an investigation aimed at providing a theoretical framework for studying diagrams and for establishing a taxonomy of their fundamental modes of graphic organization. Unpublished Phd Thesis. Royal College of Art, London, UK

Sellen, Abigail and Harper, Richard H. R. (2001): The Myth of the Paperless Office. MIT Press

Over the past thirty years, many people have proclaimed the imminent arrival of the paperless office. Yet even the World Wide Web, which allows almost any computer to read and display another computer's documents, has increased the amount of printing done. The use of e-mail in an organization causes an average 40 percent increase in paper consumption. In The Myth of the Paperless Office, Abigail Sellen and Richard Harper use the study of paper as a way to understand the work that people do and the reasons they do it the way they do. Using the tools of ethnography and cognitive psychology, they look at paper use from the level of the individual up to that of organizational culture.Central to Sellen and Harper's investigation is the concept of "affordances" -- the activities that an object allows, or affords. The physical properties of paper (its being thin, light, porous, opaque, and flexible) afford the human actions of grasping, carrying, folding, writing, and so on. The concept of affordance allows them to compare the affordances of paper with those of existing digital devices. They can then ask what kinds of devices or systems would make new kinds of activities possible or better support current activities. The authors argue that paper will continue to play an important role in office life. Rather than pursue the ideal of the paperless office, we should work toward a future in which paper and electronic document tools work in concert and organizational processes make optimal use of both.

© All rights reserved Sellen and Harper and/or MIT Press


Shneiderman, Ben and Plaisant, Catherine (2009): Designing the User Interface: Strategies for Effective Human-Computer Interaction (5th ed.). Addison-Wesley

The much-anticipated fourth edition of Designing the User Interface provides a comprehensive, authoritative introduction to the dynamic field of human-computer interaction (HCI). Students and professionals learn practical principles and guidelines needed to develop high quality interface designs—ones that users can understand, predict, and control. It covers theoretical foundations, and design processes such as expert reviews and usability testing. Numerous examples of direct manipulation, menu selection, and form fill-in give readers an understanding of excellence in design. Recent innovations in collaborative interfaces, online help, and information visualization receive special attention. A major change in this edition is the integration of the World Wide Web and mobile devices throughout the book. Chapters have examples from cell phones, consumer electronics, desktop displays, and Web interfaces.

© All rights reserved Shneiderman and Plaisant and/or Addison-Wesley


Spence, Robert (2001): Information Visualization. Addison Wesley

This is the first fully integrated book on the emerging area of information visualization, incorporating dynamic examples on an accompanying website to complement the static representations within the book. Its emphasis is on real-world examples and applications of computer-generated/interactive information visualization. Readers will learn how to display information to: pick out key information from large data streams; present ideas clearly and effectively; and increase the usability and efficiency of computer systems. It takes a dynamic approach to the subject using software examples on an associated website. This book is appropriate for readers interested in information visualization, human-computer interaction, business information technology, and computer graphics

© All rights reserved Spence and/or Addison Wesley


Sutherland, Ivan E. (1963). Sketchpad, A Man-Machine Graphical Communication System. PhD Thesis at Massachusetts Institute of Technology, online version and editors' introduction by Alan Blackwell & K. Rodden. Technical Report 574. Cambridge University Computer Laboratory

The Sketchpad system uses drawing as a novel communication medium for a computer. The system contains input, output, and computation programs which enable it to interpret information drawn directly on a computer display. It has been used to draw electrical, mechanical, scientic, mathematical, and animated drawings; it is a general purpose system. Sketchpad has shown the most usefulness as an aid to the understanding of processes, such as the notion of linkages, which can be described with pictures. Sketchpad also makes it easy to draw highly repetitive or highly accurate drawings and to change drawings previously drawn with it. The many drawings in this thesis were all made with Sketchpad.

© All rights reserved Sutherland and/or his/her publisher

Tufte, Edward R. (1983): The Visual Display of Quantitative Information. Cheshire, CT, Graphics Press

Ware, Colin (2004): Information Visualization: Perception for Design, 2nd Ed. San Francisco, Morgan Kaufman

Changes to this chapter

29 Jan 2011: Added
26 Jan 2011: Modified
25 Jan 2011: Modified
24 Jan 2011: Modified

About this chapter

Author(s): Alan Blackwell
Editors: Mads Soegaard and Rikke Friis Dam
Reviewer 1: Name suppressed
Reviewer 2: Name suppressed

Peer-review is based on the reviewing guidelines and coordinated by the Reviewing Board.
Commentaries by: Ben Shneiderman, Clive Richards, Peter C-H. Cheng and Brad A. Myers
This encyclopedia chapter has undergone double-blinded peer-review by two reviewers based on the reviewing guidelines, language copy-editing, typesetting, and reference validation.

How to cite/reference this page
URL: http://www.interaction-design.org/encyclopedia/visual_representation.html

Licensed through a Creative Commons licence Open Access

We believe in Open Access and the democratization of knowledge. Unfortunately, world class educational materials are normally hidden behind payment systems or in expensive textbooks. If you want this to change, you should help us out! Kind thoughts are not enough - you need to act!
Copyright Terms
We do NOT use copyright as a restrictive instrument, but as an instrument to protect the author against misuse while encouraging redistribution and use of his/her work. As such, these copyright terms are designed for the author and the reader, not the publisher and the profit.

Except as otherwise noted, this page/work is copyright of Alan Blackwell and The Interaction-Design.org Foundation (Chr. Molbechs Vej 4, DK-8000 Aarhus C, Denmark) and is licensed under the following terms:
  1. The Creative Commons Attribution-NoDerivs Licence
  2. The Interaction-Design.org Addendum to the Creative Commons licence

...with the exception of materials described in...:

  1. "Exceptions"

Furthermore, your use of Interaction-Design.org signifies your consent to:

  1. the "Site Terms and Conditions"
i. Creative Commons Attribution-NoDerivs 3.0 Unported

The Creative Commons Attribution-NoDerivs 3.0 Unported License

THE WORK (AS DEFINED BELOW) IS PROVIDED UNDER THE TERMS OF THIS CREATIVE COMMONS PUBLIC LICENSE ("CCPL" OR "LICENSE"). THE WORK IS PROTECTED BY COPYRIGHT AND/OR OTHER APPLICABLE LAW. ANY USE OF THE WORK OTHER THAN AS AUTHORIZED UNDER THIS LICENSE OR COPYRIGHT LAW IS PROHIBITED.

BY EXERCISING ANY RIGHTS TO THE WORK PROVIDED HERE, YOU ACCEPT AND AGREE TO BE BOUND BY THE TERMS OF THIS LICENSE. TO THE EXTENT THIS LICENSE MAY BE CONSIDERED TO BE A CONTRACT, THE LICENSOR GRANTS YOU THE RIGHTS CONTAINED HERE IN CONSIDERATION OF YOUR ACCEPTANCE OF SUCH TERMS AND CONDITIONS.

1. Definitions

  1. "Adaptation" means a work based upon the Work, or upon the Work and other pre-existing works, such as a translation, adaptation, derivative work, arrangement of music or other alterations of a literary or artistic work, or phonogram or performance and includes cinematographic adaptations or any other form in which the Work may be recast, transformed, or adapted including in any form recognizably derived from the original, except that a work that constitutes a Collection will not be considered an Adaptation for the purpose of this License. For the avoidance of doubt, where the Work is a musical work, performance or phonogram, the synchronization of the Work in timed-relation with a moving image ("synching") will be considered an Adaptation for the purpose of this License.
  2. "Collection" means a collection of literary or artistic works, such as encyclopedias and anthologies, or performances, phonograms or broadcasts, or other works or subject matter other than works listed in Section 1(f) below, which, by reason of the selection and arrangement of their contents, constitute intellectual creations, in which the Work is included in its entirety in unmodified form along with one or more other contributions, each constituting separate and independent works in themselves, which together are assembled into a collective whole. A work that constitutes a Collection will not be considered an Adaptation (as defined above) for the purposes of this License.
  3. "Distribute" means to make available to the public the original and copies of the Work through sale or other transfer of ownership.
  4. "Licensor" means the individual, individuals, entity or entities that offer(s) the Work under the terms of this License.
  5. "Original Author" means, in the case of a literary or artistic work, the individual, individuals, entity or entities who created the Work or if no individual or entity can be identified, the publisher; and in addition (i) in the case of a performance the actors, singers, musicians, dancers, and other persons who act, sing, deliver, declaim, play in, interpret or otherwise perform literary or artistic works or expressions of folklore; (ii) in the case of a phonogram the producer being the person or legal entity who first fixes the sounds of a performance or other sounds; and, (iii) in the case of broadcasts, the organization that transmits the broadcast.
  6. "Work" means the literary and/or artistic work offered under the terms of this License including without limitation any production in the literary, scientific and artistic domain, whatever may be the mode or form of its expression including digital form, such as a book, pamphlet and other writing; a lecture, address, sermon or other work of the same nature; a dramatic or dramatico-musical work; a choreographic work or entertainment in dumb show; a musical composition with or without words; a cinematographic work to which are assimilated works expressed by a process analogous to cinematography; a work of drawing, painting, architecture, sculpture, engraving or lithography; a photographic work to which are assimilated works expressed by a process analogous to photography; a work of applied art; an illustration, map, plan, sketch or three-dimensional work relative to geography, topography, architecture or science; a performance; a broadcast; a phonogram; a compilation of data to the extent it is protected as a copyrightable work; or a work performed by a variety or circus performer to the extent it is not otherwise considered a literary or artistic work.
  7. "You" means an individual or entity exercising rights under this License who has not previously violated the terms of this License with respect to the Work, or who has received express permission from the Licensor to exercise rights under this License despite a previous violation.
  8. "Publicly Perform" means to perform public recitations of the Work and to communicate to the public those public recitations, by any means or process, including by wire or wireless means or public digital performances; to make available to the public Works in such a way that members of the public may access these Works from a place and at a place individually chosen by them; to perform the Work to the public by any means or process and the communication to the public of the performances of the Work, including by public digital performance; to broadcast and rebroadcast the Work by any means including signs, sounds or images.
  9. "Reproduce" means to make copies of the Work by any means including without limitation by sound or visual recordings and the right of fixation and reproducing fixations of the Work, including storage of a protected performance or phonogram in digital form or other electronic medium.

2. Fair Dealing Rights. Nothing in this License is intended to reduce, limit, or restrict any uses free from copyright or rights arising from limitations or exceptions that are provided for in connection with the copyright protection under copyright law or other applicable laws.

3. License Grant. Subject to the terms and conditions of this License, Licensor hereby grants You a worldwide, royalty-free, non-exclusive, perpetual (for the duration of the applicable copyright) license to exercise the rights in the Work as stated below:

  1. to Reproduce the Work, to incorporate the Work into one or more Collections, and to Reproduce the Work as incorporated in the Collections; and,
  2. to Distribute and Publicly Perform the Work including as incorporated in Collections.
  3. For the avoidance of doubt:

    1. Non-waivable Compulsory License Schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme cannot be waived, the Licensor reserves the exclusive right to collect such royalties for any exercise by You of the rights granted under this License;
    2. Waivable Compulsory License Schemes. In those jurisdictions in which the right to collect royalties through any statutory or compulsory licensing scheme can be waived, the Licensor waives the exclusive right to collect such royalties for any exercise by You of the rights granted under this License; and,
    3. Voluntary License Schemes. The Licensor waives the right to collect royalties, whether individually or, in the event that the Licensor is a member of a collecting society that administers voluntary licensing schemes, via that society, from any exercise by You of the rights granted under this License.

The above rights may be exercised in all media and formats whether now known or hereafter devised. The above rights include the right to make such modifications as are technically necessary to exercise the rights in other media and formats, but otherwise you have no rights to make Adaptations. Subject to Section 8(f), all rights not expressly granted by Licensor are hereby reserved.

4. Restrictions. The license granted in Section 3 above is expressly made subject to and limited by the following restrictions:

  1. You may Distribute or Publicly Perform the Work only under the terms of this License. You must include a copy of, or the Uniform Resource Identifier (URI) for, this License with every copy of the Work You Distribute or Publicly Perform. You may not offer or impose any terms on the Work that restrict the terms of this License or the ability of the recipient of the Work to exercise the rights granted to that recipient under the terms of the License. You may not sublicense the Work. You must keep intact all notices that refer to this License and to the disclaimer of warranties with every copy of the Work You Distribute or Publicly Perform. When You Distribute or Publicly Perform the Work, You may not impose any effective technological measures on the Work that restrict the ability of a recipient of the Work from You to exercise the rights granted to that recipient under the terms of the License. This Section 4(a) applies to the Work as incorporated in a Collection, but this does not require the Collection apart from the Work itself to be made subject to the terms of this License. If You create a Collection, upon notice from any Licensor You must, to the extent practicable, remove from the Collection any credit as required by Section 4(b), as requested.
  2. If You Distribute, or Publicly Perform the Work or Collections, You must, unless a request has been made pursuant to Section 4(a), keep intact all copyright notices for the Work and provide, reasonable to the medium or means You are utilizing: (i) the name of the Original Author (or pseudonym, if applicable) if supplied, and/or if the Original Author and/or Licensor designate another party or parties (e.g., a sponsor institute, publishing entity, journal) for attribution ("Attribution Parties") in Licensor's copyright notice, terms of service or by other reasonable means, the name of such party or parties; (ii) the title of the Work if supplied; (iii) to the extent reasonably practicable, the URI, if any, that Licensor specifies to be associated with the Work, unless such URI does not refer to the copyright notice or licensing information for the Work. The credit required by this Section 4(b) may be implemented in any reasonable manner; provided, however, that in the case of a Collection, at a minimum such credit will appear, if a credit for all contributing authors of the Collection appears, then as part of these credits and in a manner at least as prominent as the credits for the other contributing authors. For the avoidance of doubt, You may only use the credit required by this Section for the purpose of attribution in the manner set out above and, by exercising Your rights under this License, You may not implicitly or explicitly assert or imply any connection with, sponsorship or endorsement by the Original Author, Licensor and/or Attribution Parties, as appropriate, of You or Your use of the Work, without the separate, express prior written permission of the Original Author, Licensor and/or Attribution Parties.
  3. Except as otherwise agreed in writing by the Licensor or as may be otherwise permitted by applicable law, if You Reproduce, Distribute or Publicly Perform the Work either by itself or as part of any Collections, You must not distort, mutilate, modify or take other derogatory action in relation to the Work which would be prejudicial to the Original Author's honor or reputation.

5. Representations, Warranties and Disclaimer

UNLESS OTHERWISE MUTUALLY AGREED TO BY THE PARTIES IN WRITING, LICENSOR OFFERS THE WORK AS-IS AND MAKES NO REPRESENTATIONS OR WARRANTIES OF ANY KIND CONCERNING THE WORK, EXPRESS, IMPLIED, STATUTORY OR OTHERWISE, INCLUDING, WITHOUT LIMITATION, WARRANTIES OF TITLE, MERCHANTIBILITY, FITNESS FOR A PARTICULAR PURPOSE, NONINFRINGEMENT, OR THE ABSENCE OF LATENT OR OTHER DEFECTS, ACCURACY, OR THE PRESENCE OF ABSENCE OF ERRORS, WHETHER OR NOT DISCOVERABLE. SOME JURISDICTIONS DO NOT ALLOW THE EXCLUSION OF IMPLIED WARRANTIES, SO SUCH EXCLUSION MAY NOT APPLY TO YOU.

6. Limitation on Liability. EXCEPT TO THE EXTENT REQUIRED BY APPLICABLE LAW, IN NO EVENT WILL LICENSOR BE LIABLE TO YOU ON ANY LEGAL THEORY FOR ANY SPECIAL, INCIDENTAL, CONSEQUENTIAL, PUNITIVE OR EXEMPLARY DAMAGES ARISING OUT OF THIS LICENSE OR THE USE OF THE WORK, EVEN IF LICENSOR HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

7. Termination

  1. This License and the rights granted hereunder will terminate automatically upon any breach by You of the terms of this License. Individuals or entities who have received Collections from You under this License, however, will not have their licenses terminated provided such individuals or entities remain in full compliance with those licenses. Sections 1, 2, 5, 6, 7, and 8 will survive any termination of this License.
  2. Subject to the above terms and conditions, the license granted here is perpetual (for the duration of the applicable copyright in the Work). Notwithstanding the above, Licensor reserves the right to release the Work under different license terms or to stop distributing the Work at any time; provided, however that any such election will not serve to withdraw this License (or any other license that has been, or is required to be, granted under the terms of this License), and this License will continue in full force and effect unless terminated as stated above.

8. Miscellaneous

  1. Each time You Distribute or Publicly Perform the Work or a Collection, the Licensor offers to the recipient a license to the Work on the same terms and conditions as the license granted to You under this License.
  2. If any provision of this License is invalid or unenforceable under applicable law, it shall not affect the validity or enforceability of the remainder of the terms of this License, and without further action by the parties to this agreement, such provision shall be reformed to the minimum extent necessary to make such provision valid and enforceable.
  3. No term or provision of this License shall be deemed waived and no breach consented to unless such waiver or consent shall be in writing and signed by the party to be charged with such waiver or consent.
  4. This License constitutes the entire agreement between the parties with respect to the Work licensed here. There are no understandings, agreements or representations with respect to the Work not specified here. Licensor shall not be bound by any additional provisions that may appear in any communication from You. This License may not be modified without the mutual written agreement of the Licensor and You.
  5. The rights granted under, and the subject matter referenced, in this License were drafted utilizing the terminology of the Berne Convention for the Protection of Literary and Artistic Works (as amended on September 28, 1979), the Rome Convention of 1961, the WIPO Copyright Treaty of 1996, the WIPO Performances and Phonograms Treaty of 1996 and the Universal Copyright Convention (as revised on July 24, 1971). These rights and subject matter take effect in the relevant jurisdiction in which the License terms are sought to be enforced according to the corresponding provisions of the implementation of those treaty provisions in the applicable national law. If the standard suite of rights granted under applicable copyright law includes additional rights not granted under this License, such additional rights are deemed to be included in the License; this License is not intended to restrict the license of any rights under applicable law.
ii. The Interaction-Design.org Addendum to the Creative Commons licence

The Interaction-Design.org Addendum to the Creative Commmons licence is a placeholder for additions to the Creative Commons licence, which are deemed necessary to include in consideration of Danish law and the operation of this site and The Interaction-Design.org Foundation.

1. Attribution

Attribution must be clearly given, i.e. the author's name, the title and URL of this work/publication/web page must clearly appear. The attribution must be given in a manner appropriate to the medium in which it is given: For example, electronic copies must include a clickable URL, which does not use the nofollow attribute value.

2. Updates

The Interaction-Design.org Foundation reserves the unilateral right to update, modify, change and alter its Site Terms and Conditions as well as Copyright Terms at any time. All such updates, modifications, changes and alterations are binding on all users and browsers of Interaction-Design.org and will be posted here.

iii. Exceptions

Exceptions

In some cases, a page/work may include content, such as an image, that is not covered by the copyright terms (i.e. "The Interaction-Design.org Addendum to the Creative Commons licence" and the "Creative Commons Attribution-NoDerivs 3.0 Unported"). When this is the case, we clearly label the content. For images, we also include the copyright label inside the image file (i.e. the full-resolution version) in metadata types like EXIF, IPTC, and XMP. We only include and label content with the following copyright terms:

  1. CC-Att-SA:
        Creative Commons Attribution-ShareAlike 3.0 Unported
        Legal Code (full licence text): http://creativecommons.org/licenses/by-sa/3.0/
  2. CC-Att-SA-2:
        Creative Commons Attribution-ShareAlike 2.0 Unported
        Legal Code (full licence text): http://creativecommons.org/licenses/by-sa/2.0/
  3. CC-Att-SA-3:
        Creative Commons Attribution-ShareAlike 3.0 Unported
        Legal Code (full licence text): http://creativecommons.org/licenses/by-sa/3.0/
  4. CC-Att-SA-1:
        Creative Commons Attribution-ShareAlike 1.0 Unported
        Legal Code (full licence text): http://creativecommons.org/licenses/by-sa/1.0/
  5. CC-Att-ND:
        Creative Commons Attribution-NoDerivs 3.0 Unported
        Legal Code (full licence text): http://creativecommons.org/licenses/by-nd/3.0/
  6. CC-Att-ND-1:
        Creative Commons Attribution-NoDerivs 1.0 Unported
        Legal Code (full licence text): http://creativecommons.org/licenses/by-nd/1.0/
  7. CC-Att-ND-2:
        Creative Commons Attribution-NoDerivs 2.0 Unported
        Legal Code (full licence text): http://creativecommons.org/licenses/by-nd/2.0/
  8. CC-Att-ND-3:
        Creative Commons Attribution-NoDerivs 3.0 Unported
        Legal Code (full licence text): http://creativecommons.org/licenses/by-nd/3.0/
  9. CC-Att:
        Creative Commons Attribution 3.0 Unported
        Legal Code (full licence text): http://creativecommons.org/licenses/by/3.0/
  10. CC-Att-2:
        Creative Commons Attribution 2.0 Unported
        Legal Code (full licence text): http://creativecommons.org/licenses/by/2.0/
  11. CC-Att-3:
        Creative Commons Attribution 3.0 Unported
        Legal Code (full licence text): http://creativecommons.org/licenses/by/3.0/
  12. CC-Att-1:
        Creative Commons Attribution 1.0 Unported
        Legal Code (full licence text): http://creativecommons.org/licenses/by/1.0/
  13. AllRightsReserved:
        All Rights Reserved. Materials used with permission. Permission to use has been granted exclusively to The Interaction-Design.org Foundation and/or the author of the given work/chapter, in which the copyrighted material is used. This permission constitutes a non-transferable license and, as such, only applies to The Interaction-Design.org Foundation. Therefore, no part of this material may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, recording or otherwise without prior written permission of the copyright holder.
  14. AllRightsReservedUsedWithoutPermission:
        All Rights Reserved. Non-free, copyrighted materials used without permission. The materials are used without permission of the copyright holder because the materials meet the legal criteria for Fair Use and/or because The Interaction-Design.org Foundation has not been able to contact the copyright holder. The most common cases are: 1) Cover art: Cover art from various items, for identification only in the context of critical commentary of that item (not for identification without critical commentary). 2) Team and corporate logos: For identification. 3) Other promotional material: Posters, programs, billboards, ads: For critical commentary. 4) Film and television screen shots: For critical commentary and discussion of the cinema and television. 5) Screenshots from software products: For critical commentary. 6) Paintings and other works of visual art: For critical commentary, including images illustrative of a particular technique or school. 7) Images with iconic status or historical importance: As subjects of commentary. 8) Images that are themselves subject of commentary.
  15. Pd:
        Public Domain (information that is common property and contains no original authorship)
  16. Trademarks and logos:
        All trademarks, logos, service marks, collective marks, design rights, personality rights or similar rights that are mentioned, used or cited on Interaction-Design.org are the property of their respective owners. The use of any trademark on Interaction-Design.org does not vest in the author or The Interaction-Design.org Foundation any trademark ownership rights in such trademarks, nor does the use of such trademarks imply any affiliation with or endorsement of The Interaction-Design.org Foundation and its authors by such owners. As such The Interaction-Design.org Foundation can not grant any rights to use any otherwise protected materials. Your use of any such or similar incorporeal property is at your own risk.

In addition, content linked from a page is not covered by one of our licenses unless specifically noted. For example, pages may link to videos or slide decks that are not covered. The design of Interaction-Design.org (graphics, html, client-side scripts, etc.) is copyright of Mads Soegaard.

iv. The Site Terms and Conditions

Please read these terms and conditions (the "Terms") carefully before using Interaction-Design.org. By using Interaction-Design.org you signify your consent to these Terms. If you do not agree to the Terms please do not use Interaction-Design.org. The Terms addresses your legal rights and obligations and includes important disclaimers and choice of law and forum provisions. Please read carefully.

1. Ownership of Interaction-Design.org

Interaction-Design.org is owned and operated by The Interaction-Design.org Foundation, a privately held corporation incorporated under the laws of Denmark, with office in Aarhus, Denmark.

Address:
The Interaction-Design.org Foundation
Att: Mads Soegaard
Chr. Molbechs Vej 4
DK-8000 Aarhus C.
Denmark

2. Choice of Law and Forum Provisions

Interaction-Design.org is run by The Interaction-Design.org Foundation, a privately held corporation residing in Aarhus, Denmark. You agree that these Terms and your use of Interaction-Design.org are governed by the laws of Denmark. You hereby consent to the exclusive jurisdiction and venue of the courts, tribunals, agencies and other dispute resolution organizations in Denmark in all disputes

  1. arising out of, relating to, or concerning Interaction-Design.org, The Interaction-Design.org Foundation, and/or these Terms
  2. in which Interaction-Design.org, The Interaction-Design.org Foundation, and/or these Terms is an issue or a material fact
  3. or in which Interaction-Design.org, The Interaction-Design.org Foundation, and/or these Terms is referenced in a paper filed in a court, tribunal, agency or other dispute resolution organization.

The Interaction-Design.org Foundation has endeavoured to comply with all legal requirements known to it in creating and maintaining Interaction-Design.org, but makes no representation that materials on Interaction-Design.org are appropriate or available for use in any particular jurisdiction. You are responsible for compliance with applicable laws. Any use in contravention of this provision or any provision of these Terms is at your own risk and, if any part of these Terms is invalid or unenforceable under applicable law, the invalid or unenforceable provision will be deemed superseded by a valid, enforceable provision that most closely matches the intent of the original provision and the remainder of these Terms shall govern such use.

3. Liability

Your use of and browsing Interaction-Design.org is at your own risk. The Interaction-Design.org Foundation does not warrant that the software used for Interaction-Design.org, and the information, material, and content on it, or any other services provided by means of Interaction-Design.org are error-free, or that their use will be uninterrupted. The Interaction-Design.org Foundation expressly disclaims all warranties related to the above-mentioned subject matter, including, without limitation, those of accuracy, condition, merchantability and fitness for particular purpose. Notwithstanding anything to the contrary on Interaction-Design.org, in no event shall The Interaction-Design.org Foundation be liable for any loss of profits, revenues, indirect, special, incidental, consequential, or other similar damages arising out of or in connection with Interaction-Design.org or out of the use of any of the services proposed by means of Interaction-Design.org.

4. Updates

The Interaction-Design.org Foundation reserves the unilateral right to update, modify, change and alter its Site Terms and Conditions as well as Copyright Terms at any time. All such updates, modifications, changes and alterations are binding on all users and browsers of Interaction-Design.org and will be posted here.

5. Legal Disclaimer

The Interaction-Design.org Foundation and its authors make no representations as to accuracy, completeness, currentness, suitability, or validity of any information, material, or content on Interaction-Design.org.

THE MATERIAL AND CONTENT POSTED ON INTERACTION-DESIGN.ORG ARE PROVIDED "AS IS" WITHOUT ANY EXPRESS WARRANTY OR IMPLIED WARRANTY OF ANY KIND INCLUDING WARRANTIES OF MERCHANTABILITY, NON-INFRINGEMENT OF INTELLECTUAL PROPERTY, OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT SHALL THE INTERACTION-DESIGN.ORG FOUNDATION BE LIABLE FOR ANY DAMAGES WHATSOEVER (INCLUDING, WITHOUT LIMITATION, DAMAGES FOR LOSS OF PROFITS, BUSINESS INTERRUPTION, LOSS OF INFORMATION) ARISING OUT OF THE USE OF OR INABILITY TO USE THE MATERIALS, EVEN IF THE INTERACTION-DESIGN.ORG FOUNDATION HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Because some jurisdictions prohibit the exclusion or limitation of liability for consequential and or incidental damages, the above limitation may not apply to you. Furthermore, The Interaction-Design.org Foundation does not warrant the accuracy or completeness of information of links or other items contained within these materials that have been provided by third parties.

6. Provision regarding change in attribution of copyrighted materials

Please contact us at mads@interaction-design.org if you, or your organization, wish to correct or change attribution or presentation of any image/material used on Interaction-Design.org, which you, or your organization, are the rightful copyright holder of. We will request that you submit proof of your ownership of the copyright on this material but will act immediately on any reasonable request.

7. Notice and prodecure for claims of copyright infringement

Every effort has been made by the individual contributing authors as well as The Interaction-Design.org Foundation to discover and contact copyright holders of artwork/illustrations/content used on Interaction-Design.org. To the extent that a copyright holder could not be found or an inadvertent permissions or copyright error was made, The Interaction-Design.org Foundation stands ready to remove content upon notice and request by a copyright holder. In the case that you believe that any content or other material provided through Interaction-Design.org infringes your copyright, you should notify The Interaction-Design.org Foundation of your infringement claim in accordance with the procedure set forth below.

We will process each notice of alleged infringement which The Interaction-Design.org Foundation receives and take appropriate action in accordance with applicable intellectual property laws. A notification of claimed copyright infringement should be emailed to mads@interaction-design.org (subject: "Takedown Request"). You may also contact us by mail at:

The Interaction-Design.org Foundation
Att: Mads Soegaard
Chr. Molbechs Vej 4
DK-8000 Aarhus C.
Denmark

To be effective, the notification must be in writing and contain the following information:

  1. an electronic or physical signature of the copyright owner or the person authorized to act on behalf of the owner of the copyright interest
  2. a description of the copyrighted work that you claim has been infringed
  3. a description of where the material that you claim is infringing is located on Interaction-Design.org that is reasonably sufficient to enable us to identify and locate the material;
  4. how The Interaction-Design.org Foundation can contact you, such as your address, telephone number, and email address
  5. a written statement by you that you have a good faith belief that the disputed use is not authorized by the copyright owner, its agent, or the law
  6. if you represent a publisher, a written statement by you that you have a good faith belief that the material has not been placed in the public domain, or licenced under another licence, before you acquired the copyright as this would possibly invalidate your copyright
  7. and a statement by you, made under penalty of perjury, that the above information in your notice is accurate and that you are the copyright owner or authorized to act on the copyright owner's behalf.

8. Trademarks and other rights

All trademarks, logos, service marks, collective marks, design rights, personality rights or similar rights that are mentioned, used or cited on Interaction-Design.org are the property of their respective owners. The use of any trademark on Interaction-Design.org does not vest in the author or The Interaction-Design.org Foundation any trademark ownership rights in such trademarks, nor does the use of such trademarks imply any affiliation with or endorsement of The Interaction-Design.org Foundation and its authors by such owners. As such The Interaction-Design.org Foundation can not grant any rights to use any otherwise protected materials. Your use of any such or similar incorporeal property is at your own risk.

9. Screenshots

Screenshots of copyrighted computer software, for which the copyright is held by the author(s) or the company that created the software, is believed to fall under the fair use doctrine in the US (and similar laws in other countries). It is believed that reproduction for purposes such as criticism, comment, news reporting, teaching, or research is not copyright infringement. If you reuse screenshots, as well as any other information on Interaction-Design.org, you do so at your own risk and under the copyright laws of your country.

10. Copyright of Abstracts

Abstracts in the Wiki Bibliography (/references/) are submitted by their authors who use the wiki to make their research as accessible as possible. When a page on Interaction-Design.org cites/references/lists a work from the bibliography, its abstract is included. However, abstracts have varying copyrights depending which publisher the work is published through. You should assume that an abstract is copyright, all rights reserved, of its publisher and/or author and therefore always use/cite abstracts according to Fair Use. You may visit the publisher's website to learn about the specific copyright terms (e.g. ACM, IEEE, or Springer) or contact the author directly. Bottom line: Cite/use abstracts according to the principles of fair use as it may otherwise be construed as a copyright infringement and subject to legal action.

11. User Submissions / User Content

You understand and acknowledge that additions to the Wiki Bibliography (including article abstracts), additions the Conference Calendar (including conference descriptions), user-contributed notes on each page (including text, photographs, graphics), or other materials posted by users on Interaction-Design.org ("Content") are the sole responsibility of the person from whom such Content originated. This means that you, and not The Interaction-Design.org Foundation, are entirely responsible for all Content that you upload, post or otherwise make available to other users of Interaction-Design.org.

When submitting content to Interaction-Design.org, you agree to not:

  1. impersonate any person or entity or falsely state or otherwise misrepresent your affiliation with a person or entity;
  2. upload, post or otherwise make available any Content that you do not have a right to make available under any law or under contractual or fiduciary relationships (such as inside information, proprietary and confidential information learned or disclosed as part of employment relationships or under nondisclosure agreements);
  3. upload, post or otherwise make available any Content that infringes any patent, trademark, trade secret, copyright or other proprietary rights ("Rights") of any party;
  4. upload, post or otherwise make available any Content that is unlawful, harmful, threatening, abusive, harassing, tortious, defamatory, vulgar, obscene, libelous, invasive of another's privacy, hateful, or racially, ethnically or otherwise objectionable;

You acknowledge that The Interaction-Design.org Foundation shall have the right to remove any Content that violates these Site Terms and Conditions or is otherwise objectionable.

12. Third Party Websites

If we provide links or pointers to other websites, no inference or assumption should be made that The Interaction-Design.org Foundation operates, controls, or is otherwise connected with these websites. When you click on a link within Interaction-Design.org, we will not warn you that you have left a Site and are subject to the terms and conditions (including privacy policies) of the destination website. In some cases it may be less obvious than others that you have left a Site and reached another website. Please be careful to read the terms of use and privacy policy of any website before you provide any confidential information or engage in any transactions. You should not rely on these Terms for another website. The Interaction-Design.org Foundation is not responsible for the content or practices of any other website. By using Interaction-Design.org, you acknowledge and agree that The Interaction-Design.org Foundation is not responsible or liable to you for any content or other materials hosted and served from any third party website.

13. Email communication: Confidential and proprietary information notice

Email messages sent from members of The Interaction-Design.org Foundation, including emails generated from the use of the interaction-design.org website, are proprietary to The Interaction-Design.org Foundation, and are intended solely for the use of the individual to whom they are addressed. Such messages may contain privileged or confidential information and should not be circulated or used for any purpose other than for what they are intended. If you receive a message from a member of The Interaction-Design.org Foundation in error, please notify the sender immediately. If you are not the intended recipient, you are hereby notified that you are strictly prohibited from using, copying, altering, or disclosing the contents of the message. The Interaction-Design.org Foundation accepts no responsibility for loss or damage arising from the use of the information transmitted by email message including damage from virus.

14. Usage conditions

Please make sure that you understand that the information provided by The Interaction-Design.org Foundation is being provided freely, and that no kind of agreement or contract is created between you and the owners, partners, users, or authors of this site, the owners of the servers upon which it is housed, the individual contributors of the The Interaction-Design.org Foundation, any project administrators, sysops or anyone else who is in any way connected with this project. If you choose to use or copy anything from from this site it does not create or imply any contractual or extracontractual liability on the part of The Interaction-Design.org Foundation or any of its members, partners, sponsors, contributors or other users. Your use of any such or similar incorporeal property is at your own risk.

15. Legal Disputes

Any dispute arising from the use of Interaction-Design.org or the interpretation of the terms is governed by the laws of Denmark, and shall be settled by the courts of Denmark. All communications regarding legal matters must be made in writing to

The Interaction-Design.org Foundation
Att: Mads Soegaard
Chr. Molbechs Vej 4
DK-8000 Aarhus C.
Denmark


 

About the author

Picture of Alan Blackwell. © Alan Blackwell
I only have one big research question, but I attack it from a lot of different angles. The question is representation. How do people make, see and use things that carry meaning? The angles from which I attack my question include various ways in which representations are applied (including design processes, interacting with technology, computer programming, visualisation), vario...   
Read more >> 

Commentaries by:

 

Ben Shneiderman

Picture of Ben Shneiderman.

Ben Shneiderman is a Professor in the Department of Computer Science, Founding Director (1983-2000) of the Human-Computer Interaction Laboratory, and Member of the Institute for Advanced Computer Studies at the University of Maryland at College Park. He has ta...

More about Ben >>
 

Clive Richards

Picture of Clive Richards.

Clive Richards, MPhil PhD(RCA) FCSD FRSA, is a Visiting Professor to the Faculty of Arts and Architecture, University of Brighton. He has worked in a wide range of art and design fields, including commercial practice (technical illustration, information graphi...

More about Clive >>
 

Peter C-H. Cheng

Picture of Peter C-H. Cheng.

Peter Cheng’s main research interest is in the nature of representational systems, spanning the design of external representations and the cognitive processes that deal with internal mental representations. Knowledge rich representations used for higher forms...

More about Peter >>
 

Brad A. Myers

Picture of Brad A. Myers.

Brad A. Myers is a Professor in the Human-Computer Interaction Institute in the School of Computer Science at Carnegie Mellon University. He is an ACM Fellow, winner of six best paper awards, and a member of the CHI Academy, an honor bestowed on the principal ...

More about Brad >>
 

Editors of this chapter:

 

Mads Soegaard

Picture of Mads Soegaard. Copyright of Mads Soegaard and Interaction-Design.org through the Creative Commons Share-Alike licence.

Previously, I've worked at The Danish National Technological Institute worki...

Read more >>
 

Rikke Friis Dam

Picture of Rikke Friis Dam. Copyright of Rikke Friis Dam and Interaction-Design.org through the Creative Commons Share-Alike licence.

Rikke Dam holds a Master's Degree in philosophy from the University of Aarhus, a...

Read more >>
 

Peer Reviewers

Reviewer 1: Name suppressed
Reviewer 2: Name suppressed

Peer-review is based on the reviewing guidelines and coordinated by the Reviewing Board.

Get Notified!

Get notified when new chapters are added to the encyclopedia!

Your Email
 

Featured chapter

Authoritative overview of Social Computing by Tom Erickson - veteran researcher at IBM Research Lab. It includes 9 HD videos filmed in Copenhagen.

Read Thomas's chapter