Upcoming Courses

go to course
User Research - Methods and Best Practices
go to course
Get Your First Job as a UX or Interaction Designer
Starts tomorrow LAST CALL!

Featured chapter

Marc Hassenzahl explains the fascinating concept of User Experience and Experience Design. Commentaries by Don Norman, Eric Reiss, Mark Blythe, and Whitney Hess

User Experience and Experience Design !


Our Latest Books

The Glossary of Human Computer Interaction
by Mads Soegaard and Rikke Friis Dam
start reading
The Social Design of Technical Systems: Building technologies for communities. 2nd Edition
by Brian Whitworth and Adnan Ahmad
start reading
Gamification at Work: Designing Engaging Business Software
by Janaki Mythily Kumar and Mario Herger
start reading
The Social Design of Technical Systems: Building technologies for communities
by Brian Whitworth and Adnan Ahmad
start reading
The Encyclopedia of Human-Computer Interaction, 2nd Ed.
by Mads Soegaard and Rikke Friis Dam
start reading

Marlene Goncalves


Publications by Marlene Goncalves (bibliography)

 what's this?
Edit | Del

Goncalves, Marlene and Tineo, Leonid (2007): A web tool for web document and data source selection with SQLFI. In: Cardoso, Jorge, Cordeiro, Jos and Filipe, Joaquim (eds.) ICEIS 2007 - Proceedings of the Ninth International Conference on Enterprise Information Systems Volume DISI June 12-16, 2007, Funchal, Portugal. pp. 449-452.

Add publication
Show list on your website

23.1 Commentary by Katina Michael and M. G. Michael

How to cite this commentary in your report

Katina Michael

Picture of Katina Michael.
Katina Michael is the IEEE Technology and Society Magazine editor-in-chief. She is the author of Innovative Automatic Identification and Location-Based Services: from Bar Codes to Chip Implants (2009) and has hosted six workshops on the Social Implications of National Security. Michael (MIEEE'04, SMIEEE'06) holds a Doctor of Philosophy in Information and Communication Technology (ICT)...   
Become a member of the Interaction Design Foundation
Katina Michael
Katina Michael is a member of The Interaction Design Foundation

Katina Michael

Bio Publications

M. G. Michael

Picture of M. G. Michael.
M.G. Michael coined the term "berveillance" in 2006 to denote omnipresent electronic surveillance embedded beneath the skin. The term was entered into the official dictionary of Australia, the Macquarie Dictionary in 2008. Michael received a PhD from the School of Theology at the Australian Catholic University in 2003 in Brisbane, Queensland, and a Master of Arts Honors from the Sch...   
Become a member of the Interaction Design Foundation
M. G. Michael
M. G. Michael is a member of The Interaction Design Foundation

M. G. Michael

Bio Publications

23.1.1 About Steve Mann

In Professor Steve Mann - inventor, physicist, engineer, mathematician, scientist, designer, developer, project director, filmmaker, artist, instrumentalist, author, photographer, actor, activist - we see so much of the paradigmatic classical Greek philosopher. I recall asking Steve if technology shaped society or society shaped technology. He replied along the lines that the question was superfluous. Steve instead pointed to praxis, from which all theory, lessons or skills stem, are practiced, embodied and realized. Steve has always been preoccupied by the application of his ideas into form. In this way too, he can be considered a modern day Leonardo Da Vinci.

It is not surprising that Professor Mann was awarded the 2004 Leonardo Award for Excellence (Leonardo, 2004). In his winning article he presented “Existential Technology” as a new category of in(ter)ventions and as a new theoretical framework for understanding privacy and identity (Mann, 2003). At the time, Mann had written more than 200 research publications already, and was the keynote speaker at numerous industry symposia and conferences. His work had also been shown in museums globally, including the Museum of Modern Art in New York, the Stedelijk Museum in Amsterdam and the Triennale di Milano (Quillian, 2004).

I embarked on my PhD in 1997, the same year in which Steve graduated with a PhD in Media Arts and Sciences from MIT under the supervision of Professor Rosalind Picard and MIT Media Lab creator and director Professor Nicholas Negroponte. I remembered being amazed by the research Steve was engaged in, particularly his insights into wearable computing, and thinking all at once what incredible uses the technologies he was developing could have but also what they might mean in terms of the social implications. At that time I was working for Nortel Networks as a network planner and strategically positioning big pipes throughout the world in anticipation of the big data that was coming through IP-based applications. Few, however, could possibly have imagined that people would be willingly creating lifelogs (or cyborg logs) through the act of glogging (PCMag, 2012), another Mann discovery, and uploading them in real-time through wireless technology, every minute of every hour of every day (Mann, 1995). 4G will make glogging even easier. Presently, there are over 165,000 gloggers at http://glogger.mobi.

23.1.2 Corresponding with Professor Mann about Sousveillance in the Educational Context

At the beginning of 2009, my close collaborator Dr MG Michael and I decided to explore the idea of glogging, inspired by correspondence with Steve on his notion of existential education (ExistEd), Figure 23.1, first officially demonstrated in 1998 (Mann, 1998). We asked our class of 163 undergraduate and postgraduate students in a compulsory computer ethics course to take part in some personal field work through the use of glogger.mobi. Examples from this class can be found in Table 1. We wanted to see sousveillance acted out before us, what its limits were, if any, and we wanted to attempt it without having sought prior Human Research Ethics Committee approval.

Full quality
Full quality
Full quality
Figure 23.1 A-B-C: Professor Steve Mann’s Existential Learning = “Learn by being [a photoborg]”
Storyboard Title


Date Uploaded

Computer Ethics Sock Puppet Theatre


20 April 2009

Nanny Cams


20 April 2009

In the 'hood: Identity theft from your home


20 April 2009

Photography and privacy


20 April 2009

Cameras and Privacy


23 April 2009

Invasion of Location Privacy


23 April 2009

Nanotech Future


23 April 2009

Mobile phone privacy


19 April 2009

Identity Fraud


20 April 2009

RFID Issues


20 April 2009



20 April 2009

Health Insurance


20 April 2009

Bluetooth - The phone is mightier than the handgun


4 May 2009

Australian Government 'Cleanfeed' Internet Filtering Scheme


24 April 2009

Life inside a camera


20 April 2009

2 guys one camera


19 April 2009

Table 23.1: Storyboards Created by University of Wollongong Students during the Course IT and Citizen Rights, Session 1, 2009

It was interesting to observe that in our class of 163 there were:

We cannot claim that in all cases our students were ‘learning by being’ (a step beyond ‘learning by doing’), but some did become true photoborgs, whilst others took on the persona of a photoborg, even if it was for a few short weeks. It takes nerve for someone to actually wear a camera, not just carry it, to admit to it recording when questioned, and to cope with the responses that that kind of activity might provoke in a setting like a regional centre in Australia. But as Steve plainly emphasizes, “[w]hat really matters, much more than whether the technology is implanted, worn, carried, or non-existent, is the degree to which the educational paradigm embodies an epistemology of personal choice, and the metaphysics of personal freedom, growth, and development” (Mann, 2006). Furthermore, Mann writes about deconstructionist learning: “As a “cyborg” in the sense of long-term adaptation, body-borne technologies, etc., one encounters a new kind of existential self-determination and mastery over one's own destiny, that can be learned, in the postmodern (posthumanism) context one might think of as the “cyborg age” in which many of us now live.”

Increasingly, photoborgs are now everywhere, and as they increase in numbers over the next decade, comfort levels of photoborg presence will also likely increase as it becomes commonplace. However, there are still laws for instance, that are in direct conflict with photoborgology. See, for example, the Surveillance Devices Act in the state of Western Australia, in Australia (WA, 1998):


  • 6. Regulation of use, installation and maintenance of optical surveillance devices
    • (1)Subject to subsections (2) and (3), a person shall not install, use, or maintain, or cause to be installed, used, or maintained, an optical surveillance device
      • (a)to record visually or observe a private activity to which that person is not a party; or
      • (b)to record visually a private activity to which that person is a party.
    • Penalty:
      • (a)for an individual: $5 000 or imprisonment for 12 months, or both;
      • (b)for a body corporate: $50 000.

As Roger Clarke (2012b) has pointed out, this act: “seems to mean that, although you can audio-record your own conversations with other people, you can't video-record them... That has serious implications for sousveillance, i.e. the use of surveillance by the less powerful, when dealing with the more powerful”.

It was in preparation for our class that we discussed the ultimate trajectory of wearables and implantables with Steve, finding his work on the existentiality axis (Figure 23.2) to be critical (Mann, 2001; Mann, 1998). It just so happened at the time we were corresponding, that Steve was working on a project with the Eyeborg Man, Rob Spence (Spence, 2008) and we were in full preparation to host the International Symposium on Technology and Society (ISTAS10) at the University of Wollongong which had as one of its major themes microchip implants for humans (UOW, 2010; Michael, 2011). Researcher Mark Gasson and Mr Amal Graafstra, both bearers of chip implants spoke on their experience of being radio-frequency identification (RFID) implantees at the symposium.

Wearability/Portability versus Existentiality
Figure 23.2: Wearability/Portability versus Existentiality

It was during this time that the interface between sousveillance and überveillance began to emerge. On the one hand, you had camera technologies that people wore to conduct surveillance “from below”, and on the other we had proposed in 2006 that embedded systems, such as implantables, would one day do the surveilling “from within”. It was in the Eyeborg’s ‘implantable’ camera that sousveillance came face-to-face with überveillance (Michael and Michael, 2009). In Figure 23.3, the various veillances are depicted in a triquetra by Mr Alexander Hayes.

The überveillance Triquetra by Mr Alexander Hayes
Full quality
Figure 23.3: The überveillance Triquetra by Mr Alexander Hayes

23.1.3 Sousveillance Outside the Context of Existential Education

Taken away from the context of learning and reflection, a wearable (or implantable) camera worn by any citizen carries with it significant and deep personal and societal implications. The photoborg may feel entirely free, a master of his/her own destiny; they may even feel safe that their point of view is being noted for re-use, if needed at a later time. Indeed, the power the photoborg presumes when they put on the camera or bear the implant, can be considered even more powerful than the traditional CCTV overhead gazing in an unrestricted manner. But no matter how one looks at it, others will inevitably be in the field of view of the wearer or bearer of technology, and unless these fellow citizens also become photoborgs themselves, there will always be inequality. Professor Mann’s sousveillance carries with it huge political, educational, environmental and spiritual overtones. The narrative which informs sousveillance is more relevant today due to the proliferation of new media, than ever before. But wherein sousveillance grants the citizen the ability to combat the powerful using their own strategic game, it also grants other citizens the ability to put on the guise of the powerful. Sousveillance is here in the eye of the beholder, the one wearing the camera. In the end it comes down to lifeworld and context and stakeholder types. What we all agree on however, is the pervasiveness of the camera, that sees everything, hears everything but comes endowed with obvious limitations such as the potential for the impairment of data, through data loss, data manipulation, or misrepresentation.

MG and Steve had similar conceptions of where the surveillance capability of the powerful is going, expressed so eloquently during the Singularity Summit in San Francisco where Steve described his Ladder Theory (Mann, 2010). MG Michael likewise referred to the idea of the “axis of access” in 2010, which Steve noted would be more correct if written “axes of access”. It was unsurprising to us, in conducting research for this article on Steve’s wearable computing history, that we stumbled across the following Wired article written in 2003 by Shachtman:

The Pentagon is about to embark on a stunningly ambitious research project designed to gather every conceivable bit of information about a person's life, index all the information and make it searchable... The embryonic LifeLog program would dump everything an individual does into a giant database: every e-mail sent or received, every picture taken, every Web page surfed, every phone call made, every TV show watched, every magazine read... All of this -- and more -- would combine with information gleaned from a variety of sources: a GPS transmitter to keep tabs on where that person went, audio-visual sensors to capture what he or she sees or says, and biomedical monitors to keep track of the individual's health... This gigantic amalgamation of personal information could then be used to "trace the 'threads' of an individual's life.

It simply goes to show how any discovery can be tailored toward any ends. Glogging was meant to sustain the power of the individual, to enable growth, maturity and development in the person. Here, it has been hi-jacked by the very same stakeholder it was created to gain protection from. Many would ask are we playing into the hands of such initiatives as DARPA’s Lifelog program by researching sousveillance and überveillance. The answer to this is not difficult- the natural trajectory of these emerging technologies would have propelled us there regardless. Arthur (2009, p. 15) speaks of an evolution of technology which is “the process by which all objects of some class are related by ties of common descent from the collection of earlier objects.” If it were not Steve Mann, then someone else would have at some given point in time discovered the true capabilities of sousveillance -- better for it to have been Mann who embraces genuine discussion on issues related to privacy, identity, and human rights.

23.1.4 International Workshop in Recognition of Steve Mann's Sousveillance Research

Mid-way through 2011, MG and I decided we would host an international workshop on Sousveillance and Point of View (POV) Technologies in Law Enforcement as our sixth workshop in the Social Implications of National Security series (Michael and Michael, 2012b) which started as an initiative under the Australian Research Council’s (ARC) Research Network for a Secure Australia (RNSA). The workshop was hosted on the 22 February 2012, exactly 17 years after Mann uploaded his wearable webcam images of MIT’s east campus fire as a “roving reporter.” But rather than just focusing on sousveillance, the workshop also emphasised the rise of crowd-sourced sousveillance, citizen rights, body-worn technologies and just-in-time policing. The workshop investigated the use of sousveillance by law enforcement for evidence-based gathering as well as its use against law enforcement by everyday citizens. The ways in which we have witnessed the proliferation of overt and covert surveillance technologies has set the stage for the re-evaluation of existing laws and practices.

The International Workshop on Sousveillance and Point of View (POV) Technologies in Law Enforcement was held exactly 17 years after Mann uploaded his wearable webcam images of MIT’s east campus
Full quality
Figure 23.4: The International Workshop on Sousveillance and Point of View (POV) Technologies in Law Enforcement was held exactly 17 years after Mann uploaded his wearable webcam images of MIT’s east campus fire as a “roving reporter.”

Over 50 delegates attended (Figure 23.5) including former Privacy Commissioners of Australia, prosecutors and barristers of the high court, members of the Queensland and Victorian police, private investigators, spy equipment vendors (Figure 23.6), National Information and Communications Technologies Australia (NICTA) representatives, educational technologists (Figure 23.7), the Commissioner for Law Enforcement Data Security, members of the Australian Privacy Foundation, artists (Figure 23.8) and academics from across the country. A highlight of the workshop was the attendance of well-known Canadian sociologists Professors Kevin Haggerty and David Lyon, who gave a keynote address and invited paper (Figure 23.6). Professor Haggerty spoke on the ‘Monitoring of Police by Police’ and Professor Lyon on the concept of the ‘Omniscient Gaze’ (Bradwell, 2012) (Figure 23.9).

Delegates at the International Workshop on Sousveillance in Australia
Full quality
Figure 23.5: Delegates at the International Workshop on Sousveillance in Australia
DUSS Pty Ltd. Demonstrating the eWitness Wearable Camera
Full quality
Figure 23.6: DUSS Pty Ltd. Demonstrating the eWitness Wearable Camera
Mr Alexander Hayes speaking about Professor Steve Mann and sousveillance during his presentation on POV and Education
Full quality
Figure 23.7: Mr Alexander Hayes speaking about Professor Steve Mann and sousveillance during his presentation on POV and Education
Mr Tim Burns. Western Australian Artist speaking at the Workshop
Full quality
Figure 23.8: Mr Tim Burns. Western Australian Artist speaking at the Workshop
Professor Kevin Haggerty (keynote), Professor David Lyon (invited speaker), and Mr Mark Lyell (plenary speaker) at the International Workshop on Sousveillance
Full quality
Figure 23.9: Professor Kevin Haggerty (keynote), Professor David Lyon (invited speaker), and Mr Mark Lyell (plenary speaker) at the International Workshop on Sousveillance

In addressing the audience with opening remarks on the workshop’s conception, I began with defining sousveillance and then went on to demonstrate its use. I could think of no better example of sousveillance-at-work than to show a short five minute clip taken by Mann himself in Downtown Toronto (Figure 23.10). In this clip you will note that Steve is exercising his civil rights and pointing out to the police officer on duty that there is a risk of someone getting electrocuted because cables are exposed to pedestrians on the sidewalk. The officer on duty rejects being a subject of Mann’s visual recording. He stops Steve as he is nearing him and exclaims: “Sir, you cannot take a picture!” To this Steve questions: “Oh. Why not?” Again, the officer exhorts Steve to stop recording. To this Steve replies- “Ok, I photograph my whole life, I always have...” To this the officer says: “I don’t want to be a part of your life through a photograph. Can you erase that photo please?” Steve does not have a chance to reply at this point and again the officer interjects growing in impatience: “Did you take a picture of me?” Steve replies: “I record my life.” Again the officer extorts: “Did you take a picture of me?” To this Steve makes a correction: “I’m recording video.” The officer interjects several times: “It’s a simple question, did you take a picture of me? Answer the question, yes or no.” Steve admits to taking footage and the officer replies: “Okay, I need you to erase that.” Steve provocatively then says: “Okay, I’ll need to call my lawyer then...” The officer is disgruntled at this point and tells Steve to call his lawyer and to give him his number. The officer continues by insisting: “Do you understand the ramifications of what is going to happen here? Don’t you realise what can happen here?” Steve tells the officer to fill out an incident report about what happened.

Code violation and physical assault. Full video at http://wearcam.org/password-66-450.htm
Full quality
Figure 23.10: Code violation and physical assault. Full video at http://wearcam.org/password-66-450.htm

A struggle over the camera occurs for a good thirty seconds with Steve refusing to let go of it... Steve reassures the officer that he is just recording his whole life. To this the officer replies: “I don’t care about your entire life.” At this point Steve wishes to seek assistance from another officer but he does not wish to do so by leaving his camera behind. Some thirty seconds later after trying to reason with Steve the officer says: “You don’t get it do you... you just don’t get it... What you have on this thing here is me, do you understand that?... Doing this job, I don’t want other people to know what I look like and what I do. Now, I don’t know where these pictures are going. They can get on the Internet, they can get somewhere like that and someone can recognise me. And I’ve got a family, and so then my family is in jeopardy. That’s what I’m telling you.” At this point of the exchange, Steve points to the film set camera to try to explain why he is recording. The officer again retorts: “I don’t care about that camera, I care about this thing pointed at my face.” Steve continues to try to convince the officer about the danger of the film set wiring and that he needs the recording for evidence. He suggests going to the police station with the officer but still this does not appease the officer who by this stage is bewildered: “You just don’t get it do you, you don’t, because if you did there wouldn’t be a problem.” To this Steve replies, “I get it... I’ve been recording my life for 20 years.” Again, the officer is adamant: “I’m not a part of your life.” Steve replies, “Well everything that I pass is a part of my life.” The struggle continues and Steve is asked if he has been in trouble with the police before and whether or not he is on any medication. There is no resolution. Steve is hurt in the incident. One week prior to this he had been electrocuted by an above-ground cable exposure in a similar context while a crew was filming at as nearby locale.

This is not the first time that Mann has been the subject of investigation. On a return flight from the United States to Canada, Mann was required by security guards to turn his machine on and off and put it through the X-ray machine while they tugged on his wires and electrodes. The New York Times reported: “the guards took him to a private room for a strip-search in which, he said, the electrodes were torn from his skin, causing bleeding, and several pieces of equipment were strewn about the room” (Guernsey, 2002). Mann was quoted as saying: “We have to make sure we don't go into a police state where travel becomes impossible for certain individuals.” At the time, Mann described suffering as a result from the sudden detachment of technology he had worn for decades.

This encounter, and others like it, cut to the core of the implications of sousveillance but also about its everyday role. When we asked Steve what we would do if participants in our glogger assessment were questioned about why they were recording others without their permission, Steve pointed us to his Request for Deletion (RFD) page (Mann, 2009). This is admittedly only a part of the solution. In the future off-the-shelf products might exist to blank out the images of people and car number plates in everyday films, just like in the realm of Google StreetView but for now this is a real issue, as seen in the encounter with the Toronto police officer. No matter how one looks at it however, the increasing use of in-car video recording cameras by law enforcement, business and civilian vehicles, helmet mounted cameras used by motorbike riders, cyclists and extreme sportsmen, roads and traffic authority cameras, embedded cameras in apparatus (e.g. cricket stumps, tazer guns) and the like, mean that through the adoption of sousveillance techniques, the average citizen can reclaim at least some of that asymmetry they have lost. Steve’s RFD approach acknowledges however, that an opt-out approach is much more realistic than an opt-in approach. Expecting everyone in my field of view (FOW) to sign a consent form allowing me to film them because I decide to walk the streets wearing a camera is just impossible. But deleting an image or film based on an individual request can be satisfied although it may not always be practical.

We are certain that as social media platforms like glogger proliferate, many will ask:

These questions are particularly pertinent in the insurance industry as in-car video recording is now widely commercialised (Figure 23.11). Wearables that record like Helmet Cams are becoming plentiful, widely used in the military, extreme sports, the mining sector, and film industry. And we now even have Taser cams, perhaps predated by the stump cam in the game of cricket over a decade ago. Some of these devices, e.g. audio listening spy glasses, are even marketed to minors through school book clubs (Figure 23.12). This raises some interesting questions about how devices used for sousveillance might be misused contra to law in a given jurisdiction. Offences may in fact be committed based on the current law but the law is not yet being enforced to curb activities related to sousveillance. On the other hand, point of view technologies more broadly may even be misused in a stalking capacity or other voyeuristic manner. See for instance, online games marketed to minors that require a webcam to be switched on for play (Figure 23.13). There are also purported borderline cases where a camera is worn by an individual who decides to take footage in a store owned by another person. While it is not a private setting per se, the store owner may not wish for his/her goods and services to be filmed. What are the rights of individuals in public spaces when it comes to private activities? How do we go about a framework for the analysis of any type of surveillance (Clarke, 2012a)?

Autovision Mobile Media Van in New South Wales, specialising in in-car vehicle tracking, navigation and recording solutions
Full quality
Figure 23.11: Autovision Mobile Media Van in New South Wales, specialising in in-car vehicle tracking, navigation and recording solutions
Image not found
Figure 23.12: Image not found
“Angelina Balerina” online dancing game that requires the use of a webcam
Full quality
Figure 23.13: “Angelina Balerina” online dancing game that requires the use of a webcam

23.1.5 Sousveillance and Point of View Technologies in Law Enforcement

The reality is that those supplying point of view (POV) equipment, some specialising in spy equipment, have undergone a massive uptake in demand. This surge is witnessed by the number of organisations that are now dealing in this kind of covert and overt new media and franchising of some of these companies. The statistics indicate that many citizens are now taking matters into their own hands and most probably at the expense of existing legislation to do with Surveillance Devices and Listening Devices Acts. In addition, members of the police force are acquiring their own technology for safety related reasons and incidence/complaint handling in an attempt to reduce the on the job stress they undergo on a daily basis.

In Australia, there were accounts of police officers some 7 years ago, purchasing video camera units and using Velcro to place these cameras in police wagons that had not come equipped with high tech gadgetry to film roadside incidences. But today we are talking about new camera kits that are just not used for in-car recording but for body-worn recording by the police. Police today may wear helmet cams, ear cams, chest cams with audio capability, GPS locators, taser cams etc.

But how much evidence gathering is too much? In the last 12 months we have seen several riots take place -- e.g. the Vancouver Riots and the London Riots. For the first time crowdsourced surveillance played second fiddle to crowdsourced sousveillance. The police called for footage to be submitted for use in convicting rioters for crimes committed. So many thousands of minutes were presented to the police -- above and beyond footage they had taken. One could consider cross-correlation of sorts taking place.

It is predicted however that with time, crowdsourced surveillance will overwhelm the limited resources employed by the police to look at such video evidence -- in many cases potentially thousands of hours worth. Professor Andrew Goldsmith of the Centre for Transnational Crime Prevention at the University of Wollongong, has written about this new visibility with respect to the Tomlinson case. It is the commentators’ opinion, as recently recorded in a special guest edited issue of the Journal of Location Based Services (Michael and Michael, 2011), that the police will be moving away from intelligent led policing and toward an IT led policing in near real-time, if not real-time. It will be the ability by the police to say that if you are currently in a zone of public disturbance, or riot, that access to real-time engineering information will be used to denote your location. Additional smartphone modalities will then be harnessed to ascertain whether or not you are a potential perpetrator- for instance accelerometer information that can denote whether you are going up or down stairs or jolting around smashing windows. To borrow from Roger Clarke, this is a form of dataveillance “on the move” (Clarke, 2009).

No doubt this kind of scenario will mean that momentarily people will decide to live off the grid- leave their mobiles behind- or use mobiles on secure and secret platforms like the Blackberry device. But it is exactly this type of scenario that may herald in the age of überveillance- a tiny onboard implant that is injected into the translucent layer of the skin, and records everything as it sees it... implants cannot be left behind... implants are always with you... and implants allegedly do not allow for tampering... the iplant as we have termed it, is that ‘shock and awe’ instrument we have been waiting for to be commercialised in all its spectre. It will supposedly be the answer to all of our electronic health record problems, our social security and tax file numbers, our real-name Internet identity, and secure mobile payments (Michael and Michael, 2012a).

The pitfalls with POV, no matter how many cameras are recording, and no matter from how many perspectives and stakeholders, is that visual evidence has limitations. What is a whole incident? How can we denote past provocation or historical data not available during a given scene? How can we ensure that data on mobile transmission has not been intercepted? How can we ensure data validation? We might well be on another road similar to that of DNA as admissible evidence in a court of law in terms of “eyewitness” recording of events. The key question to ask here is whether or not we can ever achieve “omniscience” through the use of seemingly “omnipresent” new media?

23.1.6 References


23.2 Commentary by Douglas L. Baldwin

How to cite this commentary in your report

Douglas L. Baldwin

Picture of Douglas L. Baldwin.
In December, 2012, I created a blog that I call "Bugs, Blindness and the Pursuit of Happiness." The blog was set up so that I could interview presenters at the IEEE conference, as well as begin a dialogue concerning wearable computing devices for children in special education. The initial series of blog posts reflects my first interview--a three-day series of discussions with my frie...   
Become a member of the Interaction Design Foundation
Douglas L. Baldwin
Douglas L. Baldwin is a member of The Interaction Design Foundation

Douglas L. Baldwin

Bio Publications

Steve Mann provided the vision for a new kind of digital professional; a practitioner capable of prescribing alternative perception. Like any doctor, these specialists would use diagnostic tools to arrive at a diagnosis, from which tailored prescriptions would arise.

After working for over thirty years in special education, as the founder of a vision clinic for children with special needs, and as the founder of a non-profit institute that brought sophisticated navigational technologies to blind and visually impaired kids, it is clear to me that digital perception is a revolution waiting to happen in rehabilitation and special education.

In 2010, I was approached by the X-Prize Foundation to contribute to a proposal concerning portable devices that would benefit the blind, incorporating high technologies into i-pads, cell phones; handheld systems. I knew that these handheld tools were already on the market, or soon to be. I also knew that they would be short of the vision that Steve Mann laid down years ago. There would be no concept of humanistic intelligence, no Eyetap putting sensory input where the brain was expecting to receive it (on the face at eye level), and no attention would be focused toward special education where tailored solutions were the only answer to the needs of unique children.

The list of potential remediations available to the digital perception specialist is extraordinary. I will list five that were outlined for the X Prize-Foundation report. Digital perception specialists will work with the whole body, but my focus is on the potential of what Steve calls Electric Glasses; Eyetap technology that can place computer altered images on the retina in real time.

  1. The brain can be directly impacted by electromagnetic variables that alter perception and consciousness. Hemi-synchronization (a sound wave technology) combined with light wave therapies (blue spectrum for wakefulness, and to counter seasonal affective disorder), for example, has the potential to alter mood, affect energy, and assist the evolution of consciousness for individuals.
  2. Disabilities, like autism and visual impairment, could be affected (theoretically) by placing laser input on the retina that is augmented, diminished, and/or mediated. For example, stimulating central while inhibiting peripheral vision (and the opposite), or emphasizing left field stimulation while inhibiting right field (or reverse) could significantly alter behavior patterns. Cashing input and then slowing it down or speeding it up is a method for determining how altering frame rate affects individual processing and memory storage (i.e. behavior). Enriching or reducing input directly to visual quadrants, hemi-fields, or the whole retina could benefit many people with processing and sensory disabilities.
  3. The entire field of optics will eventually give way to digital image capture and realtime alteration of images reaching the retina. Steve recognizes this when he speaks of downloading visual prescriptions. This switch to digital diagnosis and remediation is a revolution for eye doctors, and eventually portends the demise of the optical industry. Prescriptions will be altered to fit environmental demands in real time, as the environment changes.
  4. The visually impaired and blind populations are handicapped not only by sensory loss, but also by a stark, silent environment. Daniel Kish, CEO of World Access for the Blind, says that blind individuals could navigate the environment as fluidly as the sighted if there was adequate signage available. Kish says that the world is designed to help the sighted navigate. If the environment was smarter (as it could be) and if this smart environment was networked with Steve Mannʼs Smart Eyeglasses, the blind could navigate without assistance.
  5. Facebook is the beginning of hive brain. Eventually, with molecular implants, hive brain (social networking) will become a reality. In the meantime, Steve Mannʼs Electric Glasses are an intermediate step, where social networking is at face level. You could look out the eyes of your friends; in a way, become them as their sounds and sights are beamed directly into your perception.

These five examples are entirely possible now. The revolution is overdue. Google's internet glasses may be the door that brings this one step closer to reality. The military has long been working on land warrior goggles. Both Google and the military (and others) have been secret for a long time as these technologies evolved. The Lion is about to be let out of the cage.

My concern remains: Who will transform these technologies into tools that will revolutionize special education and rehabilitation?


23.3 Commentary by Woodrow Barfield and Jessica Barfield

How to cite this commentary in your report

Woodrow Barfield

Woodrow Barfield served as Professor and Director of the Sensory Engineering Laboratory at the University of Washington. He has degrees in engineering and intellectual property law and has served on the editorial board of Presence and the Virtual Reality Journal....   
Become a member of the Interaction Design Foundation
Woodrow Barfield
Woodrow Barfield is a member of The Interaction Design Foundation

Woodrow Barfield

Bio Publications

Jessica Barfield

Jess Barfield is a student and ceramics artist and will be attending Dartmouth College in the fall of 2012. She can normally be found on a field hockey field or in front of a computer....   
Become a member of the Interaction Design Foundation
Jessica Barfield
Jessica Barfield is a member of The Interaction Design Foundation

Jessica Barfield

Bio Publications

Steve Mann has written a comprehensive and informative chapter on the general topic of wearable computing (which Steve describes as miniature body-borne computational and sensory devices). We use the phrase- “general topic” because Steve expands his discussion of wearable computing to include the more expansive term, “bearable” computing (essentially wearable computing technology that is on or in the body). In the chapter, Steve also discusses how wearable computers may be used to augment, mediate, or diminish reality. As background for this commentary, I first met Steve many years ago when I attended a meeting at MIT concerning the first conference to be held on wearable computers, and Steve was then a PhD student at the MIT Media Laboratory (At the conference I made the statement: “Are we wearing the computers, or are they wearing us!”). As the faculty gathered to discuss the aims and direction of the conference, I thought then that Steve had done more to develop the field of wearable computers than the faculty that had gathered to organize the conference. Since my first meeting with Steve, he has continued his innovative work on wearable computing, and he has published extensively on the subject. I particularly enjoyed reading Steve’s antidotes concerning his experiences as a “cyborg” in a book Steve wrote for the general public, “Cyborg: Digital Destiny and Human Possibility in the Age of the Wearable Computer,” 2001. While much of Steve’s current chapter is historical in content, he also discusses many of the wearable computing applications he has created, often with Steve’s insight as to the rationale behind his inventions.

When we think of the different types of computing technology that may be worn on or in the body, we can envision a continuum that starts with the most basic of wearable computing technology (Steve mentions a wearable abacus) and ends with wearable computing that is actually connected to a person’s central nervous system. In fact, as humans are becoming more-and-more equipped with wearable (and bearable) computing technology, the distinction as to what is thought of as a “prosthesis” is becoming blurred as we integrate more computing into human anatomy and physiology. On this very topic, I co-authored a chapter about the use of computing technology to control feedback systems in human physiology (“Computing Under the Skin” which was published in Barfield and Caudell, “Fundamentals of Wearable Computing and Augmented Reality,” 2001). I agree with Steve that the extension of computing integrated into a person’s brain could radically enhance human sensory and cognitive capabilities and alter the direction of human evolution; in fact, in my view, we are just now at the cusp of this development and experimental systems (computing technology integrated into a person’s brain) are in-field now that are helping those with severe physical disabilities. For example, consider people with debilitating diseases such that they are essentially “locked in” their own body. With the appropriate wearable computing technology consisting of a microchip that is implanted onto the surface of the brain (where it monitors electronic 'thought' pulses), such people may use a computer by thought alone allowing them to communicate with their family, caregivers, and through the internet, the world at large. Sadly, in the U.S. alone about 5,000 people yearly are diagnosed with just such a disease that ultimately shuts down the motor control capabilities of their body- Amyotrophic lateral sclerosis, sometimes called Lou Gehrig's disease. This disease is a rapidly progressive, invariably fatal neurological disease that attacks the nerve cells responsible for controlling voluntary muscles. Much of the work on control theory and supervisory control of remote robots, along with digital technology, is applicable to the design and use of wearable computing for such individuals.

In our view, anyone at the cutting-edge of their discipline is not only pushing their field further, but by nature of their work, is also at the forefront of other academic disciplines as well. For example, particle physicists in search of the ultimate building blocks of the Universe, often find themselves debating those who hold a nonsecular view of the origins and structure of the Universe. Similarly, Steve’s work, albeit on a less dramatic fashion, has raised many important issues of public policy and law. For example, Steve presents the idea that wearable computers can be used to film newsworthy events as they happen or people of authority as they perform their duties. This brings up the question of whether a person has a legal right to film other people in public (answer: generally they do). In the chapter, Steve refers to an interesting case on just this topic decided by the U.S. First Circuit Court of Appeals. In the case, Simon Glik was arrested for using his cell phone’s digital video camera (a wearable computer) to film several police officers arresting a young man on the Boston Common. The charges against Glik, which included violation of Massachusetts’s wiretap statute and two other state-law offenses, were subsequently judged baseless and were dismissed. Glik then brought suit under a U.S. Federal Statute (42 U.S.C. 1983), claiming that his very arrest for filming the officers constituted a violation of his rights under the First (free speech) and Fourth (unlawful arrest) Amendments to the U.S. Constitution. The court held that based on the facts alleged, that Glik was exercising clearly-established First Amendment rights in filming the officers in a public space, and that his clearly-established Fourth Amendment rights were violated by his arrest without probable cause. However, the readers of this comment should know: In the U.S. the right to film is not without limitations. It may be subject to reasonable time, place, and manner restrictions a topic in which much case law has been decided.

Steve also discusses privacy issues they may occur when an individual wearing a computer/camera films and records people in public places. While Steve emphasizes the example where state actors, or people generally in positions of authority, are filmed, we worry about the potential to abuse people’s privacy using the technology of wearable computing. For example, video voyeurism, the act of filming or disseminating images of a person’s “private areas” under circumstance in which the person had a reasonable expectation of privacy regardless of whether the person is in a private or public location, is possible using the technology of wearable computers. In the U.S. such conduct is prohibited under State and Federal law (see for example, Video Voyeurism Prevention Act of 2004, 18 U.S.C.A. 1801). And what about the privacy issues associated with other wearable computing technology such as the ability to recognize a person’s face, then search the internet for personal information about the individual (e.g., police record, or credit report), and “tack” that information on the person as they move through the environment? Could digital “scarlet letters” be far off?

Steve’s concept of “diminished reality” in which a wearable computer can be used to replace or remove clutter, say for example, an unwanted advertisement on the side of a building, is also of interest to those in law and public policy. On this topic, I published an article in the UCLA Entertainment Law Review, 2006, titled- Commercial Speech, Intellectual Property Rights, and Advertising Using Virtual Images Inserted in TV, Film, and the Real World. In the article, I discussed the legal ramifications of placing ads consisting of virtual images projected in the real world. We can think of virtual advertising as a form of digital technology that allows advertisers to insert computer-generated brand names, logos, or animated images into television programs or movies; or with Steve’s wearable computer technology, the real world. In the case of TV, a reported benefit of virtual advertising is that it allows the action on the screen to continue while displaying an ad viewable only by the home audience. What may be worrisome about the use of virtual images to replace portions of the real world is that corporations and government officials may be able to alter what people see based on political or economic considerations; an altered reality may then become the accepted norm, the consequences of which seem to bring up the dystopian society described in Huxley’s “Brave New World.”

As a final comment, one often hears people discuss the need for “theory” to provide an intellectual framework for the work done in virtual and augmented reality. When I was on the faculty at the University of Washington, my students and I built a head tracked augmented reality system that as one looked around the space of the laboratory, they saw a corresponding computer-generated image that was rendered such that it occluded real objects in that space. We noticed that some attributes of the virtual images allowed the person to more easily view the virtual object and real world in a seamless manner. Later, I became interested in the topic of how people performed cognitive operations on computer-generated images. With Jim Foley, now at Georgia Tech, I performed experiments to determine how people mentally rotated images rendered with different lighting models. This led to thinking about how virtual images could be seamlessly integrated into the real world. I asked the question of whether there was any theory to explain how different characteristics of virtual images combined to form a “seamless whole” with the environment they were projected into, or whether virtual images projected in the real world appeared separate from the surrounding space (floating and disembodied from the real world scene). I recalled a paper I had read while in college by Garner and Felfoldy, published in Cognitive Psychology, 1970, on the integrality of stimulus dimensions in various types of information processing. The authors of the paper noted that “separable” dimensions remain psychologically distinct when in combination; an example being forms varying in shape and color. A vast amount of converging evidence suggests that people are highly efficient at selectively attending to separable dimensions. By contrast, "integral" dimensions combine into relatively unanalyzable, unitary wholes; an example being colors varying in hue, brightness and saturation. Although people can selectively attend to integral dimensions to some degree, the process is far less efficient than occurs for separable-dimension stimuli (see also Shepard, R. N., Attention and the metric structure of the stimulus space, Journal of Mathematical Psychology, 1964). I think that much can be done to develop a theory of augmented, mediated, or diminished reality using the approach discussed by Garner and Felfody, and Shepard, and I encourage readers of this comment to do so. Such research would have to expand the past work which was done on single images, to virtual images projected into the real world.

Returning to Steve’s chapter, it is an excellent source for those interested in learning about the historical context of wearable computing, and about the numerous applications Steve has developed to design a world in which the humans signal processing capabilities and wearable computing system functions form a feedback loop; the thought being, two brains are better than one! We also see Steve’s work evolving in the not too distant future to the point where humans and wearable computing technology “live” in a mutually symbiotic manner, which implies of course, the primary thinker, the wearable computing, is in some way benefiting from having a human in the loop. So, returning to what I said at the first conference held on wearable computers: Are we wearing them, or are they wearing us?


23.4 Commentary by Hiroshi Ishii

How to cite this commentary in your report

Hiroshi Ishii

Picture of Hiroshi Ishii.
Hiroshi Ishii is Jerome B. Wiesner Professor of Media Arts and Sciences at the MIT Media Lab, where he is head of the Tangible Media group and co-director of the Things That Think (TTT) consortium. Ishii's research focuses upon the design of seamless interfaces between humans, digital information, and the physical environment. His group seeks to change the "painted bits" of GUIs to "tang...   
Become a member of the Interaction Design Foundation
Hiroshi Ishii
Hiroshi Ishii is a member of The Interaction Design Foundation

Hiroshi Ishii

Bio Publications

Although most people now see Steve Mann as a father of Wearable Computing, he was already far beyond "wearable" some 20 years ago when I first met this cyborg in the MIT Media Lab. His series of inventions was not just about "wearable," but a radical form of symbiosis and co-evolution of machine and human being, which he has been experimenting with for more than two decades, living in symbiosis with computations on his skin, eyes, ears, and in his soul. His vision of "Mediated Reality" has the same significance as the "Collective Intelligence" vision of Douglas Engelbart. Because of the same reason that thinking of Doug as an inventor of the Mouse is inappropriate and disrespectful, seeing Steve as merely an inventor of "wearable" is not right. Please read his visionary papers, and enjoy Steve Mann's deep universe.


Join our community and advance:




Join our community!

Page Information

Page maintainer: The Editorial Team
URL: http://www.interaction-design.org/references/authors/marlene_goncalves.html