Upcoming Courses

go to course
The Practical Guide to Usability
89% booked. Starts in 6 days
go to course
The Ultimate Guide to Visual Perception and Design
83% booked. Starts in 12 days
 
 

Featured chapter

Marc Hassenzahl explains the fascinating concept of User Experience and Experience Design. Commentaries by Don Norman, Eric Reiss, Mark Blythe, and Whitney Hess

User Experience and Experience Design !

 
 

Our Latest Books

 
 
The Social Design of Technical Systems: Building technologies for communities. 2nd Edition
by Brian Whitworth and Adnan Ahmad
start reading
 
 
 
 
Gamification at Work: Designing Engaging Business Software
by Janaki Mythily Kumar and Mario Herger
start reading
 
 
 
 
The Social Design of Technical Systems: Building technologies for communities
by Brian Whitworth and Adnan Ahmad
start reading
 
 
 
 
The Encyclopedia of Human-Computer Interaction, 2nd Ed.
by Mads Soegaard and Rikke Friis Dam
start reading
 
 

Object Orientation Redefined

From abstract to direct objects and toward a more understandable approach to understanding

Table of Contents

Table of Contents
Extended abstract
Background and motivation
Problem statement
The emergence of Object Orientation and the problem of ‘meaning alignment’ between world and code
          Object Orientation’s affinity for ‘meaning-alignment’
          The disparity of Object Orientation and the need for a unifying idea
     Scandinavian Object orientation and the emphasis on modelling and understanding
     A conceptual framework for programming (languages)
          Modelling as the central aspect of programming
          The focus on abstraction and conceptual superstructures
          Conceptual distance
          In conclusion
     A conceptual framework for class-based object orientation
          Modelling and coordinating with a future world
A critical view of object orientation
     Knowledge distillation and purification
     The epistemological and ontological assumptions of object orientation
          Dissection along two dimensions
          Ontology of ‘underlying order’ versus ‘continual ordering’
          Underlying order vis-à-vis object orientation
          Cognitive and quasi-epistemological assumptions of object orientation
     The problematic heritage and its alternatives
          Object orientation’s assumptions as a product of its historical context
          Orthodox information processing accounts of cognition and perception
          The modal/amodal distinction
          Aristotle and semantic memory research
     Object orientation as a relativistic or cultural epistemology
     Toward a redefinition of the modelling process: From physical to suggestive modelling
Theoretical foundations of a redefinition
     Mind-body and subject-object dualisms
     Phenomenology and the action-meaning complementarity
     Wittgenstein and meaning being use
          Applicability in relation to object orientation
     Ethnomethodological accountability and transparency of use
          Applicability in relation to object orientation
          Accountability in a socio-technical network (CSCW)
          Applicability of the familiar
     Gibson and Ecological Psychology
          Some themes of ecological psychology
          Subject-object reciprocity
          Phenomenological versus Euclidian level of description
          Perceiving is knowing (perception-cognition unity)
          Perception-for-action versus perception-as-recognition
          A value-laden theory
          Ecological validity and ‘longitudinal’ time
     Applicability in relation to object orientation
          A change of fundamental assumptions
          Subject-object reciprocity and ecological validity
          Phenomenal versus euclidan level of description
          Perceiving is knowing
          Perception-for-action versus perception-as-recognition
          An ecological theory is a theory of values
          Longitudinal time and space
Toward a redefinition of object orientation
     Reconceiving the act of modelling
          Software Practice Agency and the need for a ‘concrete’ theory
          Direct participation of Model System and representational circularity
          Computer science or object orientation as a behavioural science?
          Direct Objects as a design stance?
‘Direct’ Objects – a once and future paradigm
     Model-view dependency – direct object encounter and OOUI’s
          Object-Oriented User Interfaces
     Why is Model-View independence here?
          The Model-View-Controller heritage of Smalltalk
Further reading on ‘direct objects’
Summary and conclusions
Acknowledgements
References
Notes

Extended abstract

The success of object orientation is undeniable. In 2001 the success was further emphasised when the Turing award was awarded Ole-Johan Dahl and Kristen Nygaard for conceiving “ideas fundamental to the emergence of object-oriented programming”. Eight years earlier, in 1993, Kristen Nygaard and two co-authors published an explication of what they believed was the idea of object orientation, independent of programming language or system development methodology. Their “conceptual framework for class-based object orientation” draws on their work in conceiving object orientation during the previous decades.

A core assumption of their framework is Nygaard’s mantra that “programming is understanding” and consequently that object orientation is principally an epistemological matter. Therefore, their framework attempts to explain the connection between human understanding and object-oriented formalisation, and it is to my knowledge one of the only attempts to define what the philosophical/epistemological/ontological assumptions of object orientation are. Their attempt constitutes a very positive development for the object-orientated community, which, beyond their attempt, was and is plagued with unfounded allusions to an alleged structural relationship between either the world and object-oriented objects or human thinking and object-oriented objects, e.g. “we think in objects” or the “world is made of objects with properties”. The community is also plagued with unfounded accounts as to why one object design is better or more ‘correct’ than another (except when justified by technical rationality). How can one expect to provide an explanation or justification of object designs along any sensible line of reasoning without any investigation of the foundation of such an explanation/justification?

In this report, I shall argue that the aforementioned conceptual framework for class-based object orientation is not suitable to constitute such a foundation, which was Nygaard and his co-authors’ intent, as it is based on epistemological and ontological assumptions reminiscent of modernity, a hard science view applied to the social and psychological world, an approach to categorisation as central to thinking, a view of meaning and truth as ‘underlying’, and generally a hard science approach to human understanding. Such assumptions were commonplace, especially in mainstream cognitive psychology, in the decades where Madsen, Nygaard, and Møller-Pedersen pondered on the idea of object orientation. And since cognitive psychology and computer science in these years functioned as reciprocal mirrors, it must have seemed obvious to explain a connection between understanding and object orientation on the backdrop of this synergistic relationship. However, these assumptions that came to be foundational to object orientation have at present time long shown their inadequacies in both cognitive psychology and many other disciplines. In fact, to my mind it seems self-evident that object orientation, conceived as understanding, including the central act of modelling, cannot be adequately understood from a natural science position.

As a consequence, in this report I shall present an alternative theoretical framework for object orientation based on the phenomenological tradition of Husserl and Heidegger, the social theory of Garfinkel, on the linguistic theory of Wittgenstein, on the perception-cognition theories of Gibson and Barsalou, among other theories. These theoretical positions will be explained in the light of how they may constitute the new epistemological/ontological assumptions of object orientation as well as how this change of assumptions alters the idea of object orientation. This framework can thus be used as a backdrop when answering fundamental questions like, “what is meaning in relation to object orientation?”, “what things ‘out there’ do we coordinate our code with?”, “why is this (object) design better than that one?”, etc. This update of the assumptions of object orientation, subscribing to an amodern stance, entails a redefinition of the idea of object orientation that can be characterised as a refocus from abstract to direct object encounter and a more understandable approach to understanding. By ‘understandable understanding’ I mean a more immediate and perceptually-oriented understanding than is the case for present-day object orientation, which to my mind is a quintessential engineering approach to meaning and knowledge as abstract and underlying. In my opinion, an understandable approach to understanding locates meaning in the phenomenal world of an actor, not in a disenchanted world of Newtonian facts extrapolated from sense-experience and exempted from purpose and value.


Figure 1: Preview of Figure 18 that depicts object orientation, or what may in broader terms be called ’empirical computer science’, as a specialised discipline by way of results applied from component disciplines.

Crystallised from my theoretical framework, I attempt to further a view of computer science as ‘computing science’, a discipline with an accentuation of how computing is put to use as opposed to technical rationality. I propose to conceive of the discipline in a different fashion than along the lines of the traditional research communities or university curriculae but instead as a specialised discipline by way of applied results from component disciplines (see Figure 1). Therefore, the reader must prepare himself for a line of reasoning that intentionally cuts across disciplinary borders and the genre-boundaries of most publications carrying the umbrella term ‘systems development’, ‘informatics’, or ‘computer science’.

Another consequence of changing the assumptions of object orientation is a reconception of the act of modelling. Instead of viewing modelling as a from-to relation where something ‘out there’ (i.e. the Referent System) is identified and re-presented in the software domain layer (i.e. the Model System), a subject-object reciprocal stance is adopted and I propose to view the Model System as a ‘Realm of Intended Meaning’ as it constitutes the programmer’s representation of an intended future world, as opposed to a representation of something ‘already out there’. As a further consequence of abandoning a belief in a world out there, it becomes senseless to speak of a Referent System unless it is reconceived as future meaning or the future world to be modelled. And being in the future (i.e. post-deployment) and depending on the future meaning-formation processes of the users, one cannot simply represent parts of the Referent System in the Model System. Instead, the Referent System has to be thought of as ‘real’ only to the extent that its meaning will become realised as intended once the system is appropriated by users and entangled in sociological and individual meaning-formation processes. This factor makes the Model and Referent System interdependent and one cannot any longer speak of objective modelling as a from-to relation where validity is established by checking the correspondence relation (naïve realism). Instead, one has to perform what I choose to call suggestive modelling as the Model System must assume a more proactive, suggestive, and purposive role toward the ‘construction of meaning’ to ensure that its codified intended meaning will be realised as intended in the Referent System post-deployment, where real users make sense of it (and make it into sense). This makes validity of (suggestive) modelling become not a question of descriptive correspondence but of closeness of empirically realised meaning vis-à-vis intended meaning. A model for versus a model of.

The next obvious question to treat is how to conceive of suggestive modelling in a more specific way. Fortunately, the theoretical framework for object orientation that I present in this report provides an answer, more precisely in the relationship between perception and cognition and the significance of this relationship in conceiving of meaning.

Meaning, in a natural science context and mainstream information processing cognitive psychology, is usually thought of as ‘underlying’. In information processing cognitive psychology, perception is commonly relegated the status of input device or recording system, from the outputs of which meaning and knowledge are extracted by higher-order cognitive processes. In consequence, perception cannot be regarded meaning-in-itself in orthodox cognitive psychology just as the presentation of the Model System (the View layer) is not object-oriented core meaning but merely a visual presentation of underlying meaning.

As will be evident, reconceiving modelling as suggestive modelling requires relying on epistemological theories like those of Phenomenology and Gibson’s (1979) theory of Direct Perception where meaning is indeed not underlying but very immediate. In the latter, c oncepts and knowledge are not something we ‘process’, build up, or extrapolate from sense experiences but instead it is posited that perceptual neural systems underlie both perceptual and conceptual cognitive activity. Sense experience is thus meaning in itself, not something to derive meaning from.

Such a theoretical stance brings the debate of the distinction between user interface and application semantics#1 (Model-View relationship) into the heart of a redefinition of object orientation, and notably suggestive modelling, as a ‘direct and perceptual encounter’ of meaning in the form of objects is the principal way the Model System can be ‘suggestive of interpretation’ and thus ensure its validity (as a measure of the degree of match between realised and intended meaning).

Therefore, this report is brought to an end with an archaeology of how object-oriented meaning in a non-underlying sense has been conceived over the years, from the early Smalltalk tradition and the Model-View-Controller pattern to object-oriented user interfaces (OOUIs). I show that a re-interpretation/reviving of some of these concepts may help bring the user interface back from its auxiliary and secondary role as information-transmitter-and-mediator, in line with mainstream accounts of the human perceptual system, and recognise it as the very locus of object-oriented meaning.

Background and motivation

The motivation for this project comes from experiences gained from teaching a course on object-oriented programming and systems development at the Department of Information and Media Studies at the University of Aarhus. Before teaching the course I had been pondering on the problematic epistemological/cognitive foundations of object orientation for a while but having to teach object-oriented programming and systems development to ‘unspoiled’ students underlined that something was not right with regards to the relationship between understanding and object orientation.

During the one-year course, the students were taught an object-oriented development method (Unified Process) and an object-oriented programming language (Java). What the students had particular difficulty in understanding was what the essence of object orientation was, independent of a development method or a programming language. Object orientation in the Nordic countries, and especially at the Department for Computer Science at the University of Aarhus, has always accentuated Kristen Nygaard’s mantra that “programming is understanding”, i.e. that the essence or idea of object orientation is understanding. So why did all these bright students, in their own words, find the act of ‘understanding’ counter-intuitive? After all, this is what they are trained in doing.

Allow me to elaborate on this issue. During the course, the students were to find a company or real-world organisation and make a small-scale system for this ‘customer’ (not to be put into production but to function as a working prototype). During this course of time they were taught about object-oriented programming and systems development in a manner that allowed them to iterate between acquiring text-book knowledge and trying this out in their ‘real-world’ project in cooperation with the given company.

The students generally did not have much difficulty understanding a domain model as a sort of topological view of the significant vocabulary and abstractions of the domain. They observed or interviewed the users and used the way they viewed and talked about their work as a basis for the domain model. Working iteratively, they started early making a code skeleton from the domain model and did not seem to have much trouble understanding that creating alignment between the world and the object design is what modelling is about. Continuously keeping an alignment between the world and the design artifacts such as System Sequence Diagrams and Design Class Diagrams or actual Java code helped their understanding, as was the point.

However, as soon as the students started having prototyping sessions with the users, problems arose in understanding the world-code alignment. They had difficulty understanding how to integrate the insights gained from the prototype sessions with the object-oriented classifications expressed in the domain model, object design, and code skeleton. Prototypes (at least the non-cardboard ones) created a world of tabbed panes, of dialog boxes, and other Graphical User Interface components (GUI components henceforth) that were also formalised in Java classes but it seemed unclear to the students what their relation were to the domain. For example, an applicant class is the representation of the real-world applicant but what is the thing on the screen then, the applicant screen representation? The trouble is that the screen representation is also represented in a class but this time a class without the applicant-as-domain-class’s “real-world” reference.

I explained the distinction between the technical services layer, domain layer and GUI layer, but the relationship between the domain layer and the GUI layer still seemed puzzling. It became two competing versions of reality, a problem which revealed itself in a Design Class Diagram with a lot of well-designed domain layer classes and then a constellation of GUI classes which were ‘adrift’ in relation hereto (see Figure 2). The relation between these two sets of classes seemed difficult to explain to them. The students raised questions like, ‘if the domain layer is a representation of the world out there what is the interface then, a representation of that representation?’ And ‘is the domain layer not a bit out of date in comparison with the GUI classes, the users’ new meaning’? Where was this so-called object-oriented meaning located? The confusing ‘ontological relationship’ between GUI and domain layer became even worse when I introduced the ‘best practice’ of separating the domain layer (the Model component) from the GUI classes (the View component), as is captured by architectures like Model View Controller (Krasner and Pope 1988), Seeheim Model (Green 1985) and design patterns such as InterfaceOutside (Cockburn 1996).

Figure 2: A Design Class Diagram from a group of students. They had difficulty understanding what the locus of meaning was, the GUI classes or the Domain Layer. They intuitively understood the advantages of separating business logic code from GUI code but did not understand why this was not possible without completely separating GUI and Domain layer in two different worlds.

Looking at their Design Class Diagrams, some of these classes (even thought of in precompile-time) represented the world out there. However, some of them did not, namely the GUI classes. They, on the other hand, represented what the user experienced and could relate to in run-time, c.f. the prototyping sessions. So one set of classes seemed mainly to represent the old domain (although they were ‘new’ parts added to them) while the GUI classes in runtime seemed to represent how the user was to think about and interact with the domain, the new organisational reality. Why was the relation between these two sets of classes not clearer? I found it quite difficult to put this in plain words to the students and wondered why. After all, object orientation is merely ‘a small matter of understanding’ #2.

To clarify these issues, I introduced Madsen et al.’s article toward the end of the course. It captures what the authors, among them Kristen Nygaard, mean by the idea of object orientation and represents a view of viewing programming and systems development as understanding based on categorisation, abstraction, and object-oriented meaning conceived as ‘underlying’. However, this kind of understanding was according to the students not very understandable. As one female student noted in a straightforward manner at the end of my lecture, “The problem with what you’re saying is that it is too damn abstract”.

In the end, most of the students understood what was required of them but it was clear that it required substantial ‘mental training’ and that this sort of object-oriented understanding by no means seemed natural to them.

Problem statement

Since the end of the aforementioned course, the more I have thought about what object orientation is in relation to understanding and thought processes, the more abstract and counter-intuitive the whole idea seems to me. Consequently, in this report I shall investigate what is meant by ‘understanding’ in relation to object orientation, i.e. programming-as-understanding, by examining the epistemological/ontological/philosophy of science assumptions of object orientation. In addition to such an investigation, I shall propose a theoretical framework for what I believe to be a more understandable approach to object-oriented understanding, based on an update of its epistemological/cognitive and ontological assumptions (as expressed in Madsen et al. 1993, who draw on Knudsen and Thomsen 1985). I shall also provide concrete examples of architectures and coding practices that embody my theoretical framework as well as provide a very concrete redefinition of object-oriented modelling in order to underline that my ideas are not merely theoretical constructs or idealised intellectual trumpery but indeed very viable and constructive.

The emergence of Object Orientation and the problem of ‘meaning alignment’ between world and code

As mentioned, the success of object orientation is undeniable. Almost all modern-day programming languages, even scripting languages however inherently non-object-oriented they may be, have some level of object orientation to them.

Object Orientation’s affinity for ‘meaning-alignment’

Object orientation started in back in the 60s when Kristen Nygaard developed the Simula-67 programming language. For him, programming and understanding were two sides of the same coin and he thus called his programming language Simula as it was to simulate the real world, i.e. to simulate the programmer’s understanding, not translate it into ‘unnatural’ constructs.

For this reason, Simula and its antecedents became good at handling complexity: If the correspondence between the real world and the code or object design was high (i.e. a low representational gap), the programmer would have to do less mental translations between the software world (the object design) and the real world. With an object-oriented language he could ‘code the world’ the same way ‘he viewed the world’. And, once empirical approaches such as interviewing users became accepted or even encouraged, he could ‘code the world’ the same way the ‘the user viewed the world’, i.e. the user’s conceptual or mental model of a given domain. Object orientation thus became a powerful tool for the programmer with respect to deciding on his software design.


 

Figure 3: Good cognitive economy for the programmer is constituted by creating alignment between either (A) the user’s conceptual model and the code/object design or (B) the world and the code/object design.

The breakthrough of object orientation came with SmallTalk (Goldberg and Robson 1983) some decades after Simula had provided the ‘proof-of-concept’ so to speak, and the industrial breakthrough came with the C++ language (Stroustrup 1986). However, at this time, the mid 80s, computers had made their way into offices and homes and the computer scientist was not the only one interacting with software. Until this point in time object orientation had provided a convenient way for the programmer to structure and view her software, but non-expert users could of course not be expected to interact with the software the same way computer scientists did. To this end, and to enable the spread of the computer beyond the server room or the programmer’s desk, the graphical user interface was added onto the software. Whereas object orientation provided the programmer with a “view of the software” at a, to him, meaningful level of abstraction, the user interface was the user’s “view of the software”. It acted as a translator between the technically challenged user and the ‘real software’ hidden somewhere behind the GUI, deep in the belly of the machine. Thus, GUI and application semantics existed in separate realms and this abstraction or separation has along the years come to be considered a best practice.

The emergence of UIs for end users did not distort the object-oriented ideas as the mapping between the users’ conceptual model and the object design (the ‘default’ level of abstraction is the object level) was still there. Nonetheless, it left the GUI to its own devices, without a coupling to the object design. This lack of coupling came under discussion in the HCI community, which had emerged as a discipline in the early 80s and quickly had become vastly productive. It was notably in connection with direct manipulation interfaces (e.g. Hudson 1987, Hudson and King 1988) and User Interface Management Systems (UIMSs) #3 (e.g. Rosenberg et al. 1988) that the missing coupling became problematised.

Figure 4: The lack of coupling between code/object design and GUI.

In these discussions it was often claimed that in order to provide really Direct Manipulation interfaces (with a high degree of semantic feedback) a tight relationship between GUI and application semantics was needed. Such as coupling would result in using the user’s perception of the interface to guide the object design, or, conversely, using the object model to guide the interface design. This, some claimed, would have the following desirable consequence:

Figure 5: From the programmer’s point of view there is an alignment between the user’s perception of the system (the new organisational reality) and the object design.

The above alignment was, as we shall see, taken up by many researchers and practitioners but (unfortunately) seemed to fade away again, perhaps due to the clear-cut division of labour between GUI-designer/HCI people and computer scientists that emerged through the 80s. Thus, one may claim with few exceptions that object orientation concerns itself with the cognitive coupling between code at the object design level and the user’s conceptual model, whereas the HCI community concerns itself with the cognitive coupling between the user (including her conceptual model if one is supposed to exist) and the visible and tangible GUI #4 (see Figure 5). This is of course an oversimplification but it suffices for our present purposes.

The disparity of Object Orientation and the need for a unifying idea

In 2001 the success of object orientation was further emphasised when the Turing award was awarded Ole-Johan Dahl and Kristen Nygaard for conceiving “ideas fundamental to the emergence of object-oriented programming, through their design of the programming languages Simula I and Simula 67” (ACM 2003). Because of the immense popularity and prevalence of the object-oriented paradigm, referring to systems development in computer science circles usually means object-oriented systems development.

This umbrella term encompasses countless specialised communities. Some concern themselves with purely technical matters like compiler-technologies, object persistence mechanisms, performance issues, etc. and these communities will not receive much attention in the present report as they usually do not engage in epistemological discussions (due to their self-perception as mathematical or otherwise natural science disciplines). On the other hand, attention will be paid to the parts of the object-oriented community that are concerned with the integration of the technical matters with the process aspects of systems development; in other words, how to get from capturing requirements, over analysing the problem domain, designing the system, and implementing it in code. This is the community concerned with Object-Oriented Analysis and Design, OOAD in shorthand, and since there does not seem to be any branch of object orientation concerned with epistemological matters, this community seems to be one of the few places where such discussions emerge.

There is a small set of widely-used OOAD methodologies and the most well-known are Unified Process (henceforth UP) (Jacobson et al. 1999), its sister-methodology Rational Unified Process (henceforth RUP), which are somewhat rigid or heavy-weight. As recent answers to the rigidity of UP and RUP, more ‘agile’ development methods have gained popularity, for example ‘Agile UP (Larman 1998, 2001) and Extreme Programming (henceforth XP) developed by Kent Beck (Beck 1999), which rethinks some of the core assumptions of the UP-RUP sisters.

What these development methodologies share with the design and use of object-oriented programming languages is that the idea of object orientation is their cornerstone. Object orientation is thus the idea that cuts across both analysis, design, and implementation and a reasonable question would consequently be what this idea really is – independent of development methodologies, languages, persistence mechanisms, or the like. This question has been attempted answered by very few and done so in a, to my mind, somewhat confusing manner. Before I review these attempts, I shall briefly provide some background.

Scandinavian Object orientation and the emphasis on modelling and understanding

Although the literature on object orientation in general is immense (ranging from persistence mechanisms to programming, over OOAD) there is a surprising lack of literature elucidating two key aspects of object orientation that are mutually dependent:

1. A general conceptual framework explaining what object orientation meansin general terms without reference to a specific language or a process model.

2. A thorough investigation of the alleged connection between object orientation and how the world is structured or between object orientation and human understanding. This connection remains an assertion until it has been researched, not just loosely commented on. The assumption is particularly evident in Scandinavian object orientation where modelling is central, but also in the North American tradition although downplayed on account of reuse (c.f. e.g. Madsen 1995).

This, however, does not mean that the history of computing has entirely ignored soft matters of programming languages. For example, the FORTRAN language was partly motivated by a wish to provide a programming language that reflected the users’, mainly mathematicians, domain (e.g. Collins 1995 and Sebesta 2002: pp. 44-45). Hence, FORmula TRANslation. Similarly, one of the chief design criteria of the COBOL language (COmmon Business Oriented Language) was its ease-of-use in order to appeal to as large a user population as possible, even at the expense of computational efficiency and expressivity (Sebesta 2002: pp. 61-63). Because of these Human Factors motivations, the languages were to some extent designed from ‘the human side’, but opening any kind of book or conference proceedings dealing with programming language design, one discovers that human concerns only constitute a minute part and that they are usually only superficially treated.

This state of affairs may perhaps have been expectable in the early years of computing when the technical rationality of hardware played a large part of the design efforts. However, when designing a programming language nowadays, or at least from the 80s and onward, one should expect of a language designer to thoroughly consider how her language design and its use will relate to ‘the real world’ and human understanding, in other words the epistemological and ontological assumptions of the language. However, judging from the ACM Conference on the History of Programming Languages (Feldman 1993) one gets the impression that language design is almost exclusively made on the background of technical or functionality-related premises. This leads to a sort of circular argumentation for programming languages where one design justifies the other and where their relation to the critical epistemological aspects of the empirical process of producing a system from start to finish, i.e. using the programming language for programming-as-understanding, is downplayed. For example, in a summary of SmallTalk’s history (Kay 1993) in the conference proceedings of the ACM Conference on the History of Programming Languages, Alan Kay makes no substantial comments on Smalltalk’s modelling aspects, about how it relates to the world it is to formalise and represent. Other language designers in the same conference (e.g. Hansen 1993 (Pascal) and Ritchie 1993 (the C language)) are also remarkably quiet about these aspects of their language design.

While some of the aforementioned languages like FORTRAN and COBOL have partly been designed for ease-of-use and claim to reflect the user’s domain, the claim of object orientation is somewhat more radical, namely that it hangs together with our very cognitive/epistemological processes as mentioned in #2 above. To start off the investigation of what is really meant hereby, as well as what object orientation as an idea means across programming languages and development methods (#1), I shall introduce Knudsen and Thomsen (1985).

A conceptual framework for programming (languages)

Modelling as the central aspect of programming

Knudsen and Thomsen (1985) have developed a framework of language independent concepts with the aim of enabling discussion of the act of programming and programming languages in general, independent of the premises of any specific language. What makes their work particularly interesting is their, typically Scandinavian, stance toward viewing programming languages, including non-object-oriented ones, firstly as a tool for conceptual modelling and only secondarily a tool for instructing the computer (p. i). They picture the act of programming in the following way.

Figure 6: The act of programming, as adapted from Knudsen and Thomsen (1985), p. 5

First, there is a distinction between the Referent System and the Model System. The Referent System is a part of the real world, e.g. a company. The Model System is a program or computer model simulating part of the Referent System, let us say the handling of applications in an HR department. The act of modelling is thus the process “in which problem specific concepts (established in the Referent System) are given a definition in terms of realized concepts (established in the Model System)” (p. 11). For example, the concept of application in the Referent System is realised in the Java class Application , the realised concept, from which Java objects are instantiated.

The sum of the problem specific concepts identified in the Referent System is called the problem area, “the domain of the programming process” (p. 7). The problem area, often referred to as problem domain, is in other words the piece of the Referent System that the programmer decides to actually model, e.g. the HR department’s handling of applications.

The focus on abstraction and conceptual superstructures

In the Referent System a concept “is a generalised idea about a collection of phenomena, based on common properties of instances in the collection” (p. 291) and a phenomenon is “a thing that has definite, individual existence in reality or in the mind” (p. 290). For example, the application submitted by John Smith is a phenomenon that, together with other applications, constitutes the general idea of an application. And since the Model System is a model of the Referent System, we find an exact copy of this structure here: The realised concepts (e.g. the Application class) are general ideas abstracted from entities that share properties (e.g. instances of the Application class). In the Model System however, the abstraction process is somewhat the reverse as entities (e.g. Java objects) are generated from realised concepts (Java classes), a distinction they call the Entity-Descriptor distinction (p. 28). Thus, in the Model System a concept is the cradle of phenomena and as such indispensable. And since the creation of objects thus require an identification of problem-specific concepts in the problem domain, the programmer is predisposed toward viewing the world in terms of concepts, he is compelled to answer “what sort of phenomenon is it?”

Conceptual distance

Between the problem specific concepts (an application) and the realised concepts (the Application class) a conceptual distance exists. Knudsen and Thomsen make the comment that “This distance can be used as an indication of the difficulties in modelling the [required program] in terms of the realized concepts” (p. 8). They do, however, treat conceptual distance as an a priori property of the modelling process, and do not investigate or theorise about its nature; e.g. how such a distance can be measured, how to separate the real world from its description, and whether the real world was really there before the description.

Thomsen and Knudsen apply the classic terminology of Greek philosophy and intensional logic to distinguish between a concept’s extension, intension, and designation. The extension of a concept is the collection of phenomena which the concept covers, the intension of a concept is the collection of properties that characterise phenomena in the extension of a concept, and the designation of a concept is the collection of names under which the concept is known (pp. 12-13). The authors acknowledge their Aristotelian heritage and contrast this to a fuzzy view of concepts where borders between concepts are more fluid. The key definitional property that distinguishes the fuzzy view from the Aristotelian (to use an Aristotelian phrase) is that membership in the Aristotelian view is a yes-no question whereas it is a question of higher and lower degrees of membership in the fuzzy view. This causes the Aristotelian view of concepts to have a higher affinity for formalisation and, in my own view, a determinability that makes modelling more comprehensible (“Well, is it or is it not?”). The authors note that they believe that many real-world concepts and phenomena are fuzzy, and this makes a formalisation based on an Aristotelian approach problematic. Therefore, “a critical aspect of the modelling function will often be to give an Aristotelian definition of problem specific concepts that have a fuzzy nature in the referent system.” (p. 18). And further: “No techniques to avoid [imperfect modelling] can be given, but of course we can try to make the models as good as possible to meet the needs of our application”. In other words, modelling means cutting your coat according to your cloth; modelling is about finding the best possible model for the problem area, i.e. the model with the least conceptual distance.

In conclusion

From the above, it becomes very clear that Knudsen et al. are (very) preoccupied with conceptual superstructures and conceptual distance in relation to understanding, modelling and programming-as-understanding. I believe this preoccupation with concepts to be very characteristic of Western thinking dating back to classical Greek philosophy. In western thinking, especially as manifested in our scientific tradition, one focuses on deriving a conceptual superstructure for each phenomenon studied; it seems we have a predisposition toward looking for a genus. We ask “what sort of thing is this?” as opposed to appreciating the thing on its own terms or on its immediate characteristics. And while a primary focus on the generalisable is fruitful in some scientific contexts it does not necessarily reflect how we think per se.

The fact that concepts are attributed this salience in Thomsen and Knudsen (1985) is further emphasised by the fact that a conceptual distance is a distance between ‘concepts’. I shall later use this focus on abstraction and conceptual superstructures as a key point of criticism.

A conceptual framework for class-based object orientation

Madsen et al. (1993) use Knudsen and Thomsen 1985 as a basis for their conceptual framework for class-based object orientation and to my knowledge Madsen et al. represent the only attempt to provide such a framework.

Madsen et al. (1993) describe the act of programming as that of understanding (e.g. p. 3), and hold that analysis, design, and implementation are all programming activities (thus understanding activities) at different levels of abstraction. This makes understanding and modelling paramount and object orientation is seen as a perspective for understanding the world and creating a world-code correspondence (p. 284). It also makes Madsen et al.’s conceptual framework a sort of ’connective tissue’ or ‘overarching idea’ that holds analysis, design, and implementation together as the phases share this common ‘understanding-approach’. In comparison, development methods like Agile Unified Process (UP) (Larman 2001) or RUP (Jacobson et al. 1999 ) use a series of heterogeneous types of modelling. For example, in RUP and Agile UP modelling during some parts of analysis may be viewed as procedural (via so-called ‘analysis artifacts’ such as Use Cases, System Sequence Diagrams) and during analysis modelling may be viewed as object-centric (via so-called ‘design artifacts’ such as Design Class Diagrams). For my students, the heterogeneous approach of UP made the relationship between the phases and indeed the whole development process less comprehensible and difficult to ‘get a hold of conceptually’. Due to the heterogeneity of perspectives on the domain (as embedded in the various analysis/design artifacts mentioned above) they even initially had difficulty understanding the connection between a domain model and a design class diagram #5.

One of the claims of the present report is that object orientation can and should form the connecting tissue between the phases mentioned above as well as interface design. However, due to problems of their basic scientific assumptions, Madsen et al.’s conceptual framework is to my mind not conducive of such a truly overarching approach. I shall elaborate on this in the section “ A critical view of object orientation”.

Object-oriented concept according to Madsen et al. (1993)

Class-based object-oriented language, in this case Java

Concept

Class

Phenomenon

Object

Intension

State and behaviour of a class ( attributes and methods)

Extension

Set of objects instantiated from a class

Denotation

Identifier, i.e. a set of identifiers pointing to an object in memory of the same or ‘underlying’ type declaration as the identifier (according to the Liskov substitution principle)

Inheritance, specialisation, and extension

Class inheritance via the ‘extends’ keyword

Aggregation and composition

Class variables (declared public or accessible via get/set methods), usually one does not mean basic data types.

Figure 7: Terms used in Madsen et al. and how they map onto a class-based object-oriented language, in this case Java.

Modelling and coordinating with a future world

A question we must also answer is to which extent object orientation is about simulation of something that is perceived to be ‘real’ or ‘already existing’. In this connection the phrase "the dialectics between tradition and transcendence" is often used in the HCI and the systems development community. The phrase is from by Pelle Ehn’s PhD thesis (1988), and captures the idea that design is a fundamental tension between tradition and transcendence. This tension is to be understood in terms of the designer's role being somewhere in the space between extending and improving the present design context (to root the design in the present tradition) and exploring, and eventually reifying possible future worlds by transcending, going beyond, the boundaries of the existing tradition, ideas, and conceptions of the design context. If this is true, object orientation should not only enable a faithful modelling of a Referent System, an ‘IT-fying’ of something existing, but also enable reifying a future world.

Madsen et al. address this question by stating that “A program execution is regarded as a physical model simulating the behaviour of either a real or imaginary part of the world” (p. 284). This makes the programmer’s unit of analysis real and/or imaginary parts of the world from which he makes program executions. Transcending the tradition can thus be done by ‘adding new aspects’ to the real world before creating one’s Model System of this world.

“The definition of object-oriented programming implies that all program executions are models of some other existing systems. For new systems this also holds, even though the existing system in that case is a system existing in the minds(s) of the programmer(s)”. (p. 285).

“The creation of computer systems of course includes a great deal of invention. These inventions may be imagined as new phenomena and concepts with a well defined role in the problem domain. They will then subsequently be realized as objects and patterns in the program execution”. (p. 285).

Analysis (i.e. understanding) in relation to object-oriented programming is thus conceived by Madsen et al. as something very constructionist. It is not merely appreciating the current world, nor is it merely a question of perspective on it. It is also a purposeful addition of new understanding to an existing understanding of the current world. This new and future world (in the mind of the programmer) can then be reified through the creation of a Model System.

Object-oriented analysis thus both involves identifying parts of the real world as well as adding new parts to it. That way, design and implementation activities are about coordinating with this “future world”, i.e. keeping the design and the code ‘in alignment’ with it.

 

Figure 8: Modelling is coordination with real and imaginary parts of the world, jointly constituting the problem domain.

During the development lifecycle this world is iterated over and remade several times. Each time the developer finds a better way of conceiving of the Referent System (analysis), the Model System is brought into alignment (design and implementation). Once deployed, the actual computer system will replace, augment, or somehow change the original Referent System (more specifically, the real parts of it); it becomes a part of the real world. For example, the electronic patient record replaces the paper-based record and thus becomes the patient record. After deployment, object orientation’s unit of analysis thus sometimes changes, disappears, or becomes a reference to the previous state of affairs.

To ‘construct’ the problem domain during analysis we use methods such as use case writing, exploratory, experimental, or evolutionary prototypes, domain model writing, etc. These are the analysis artifacts that first represent the problem domain. From this constructed Referent System expressed in various artifacts, relevant phenomena and concepts (jointly representing the problem domain) are picked out and modelled in the Model System, e.g. by using a ‘name-category list’ to extract nouns from use cases (Larman 2001). The problem domain is what the developer coordinates his code with during design and implementation activities, and due to the closeness of his view of the world in phenomena/concepts and the programmatic, diagrammatic, and textual means of expression available to him, a cognitive economy is established. This is what Madsen et al. mean when they state that “analysis, design and implementation are programming or modelling activities, but at different abstraction levels” (Madsen et al. 1993: p. 283) since one coordinates with the Model System in the same basic way (via phenomena and concepts) during all three phases.

Figure 9: shows how the new future reality is first established through analysis artifacts such as use cases etc. Some of the artifacts are mainly used for representing the ‘real’ parts of the Referent System and some toward the ‘imaginary parts’ of the Referent System. Subsequently, one identifies relevant phenomena and concepts, and map them over into the Model System.

An important point is that the Model System is not a computer system but modelled (future) meaning. In this regard, system merely means a coherent interrelated whole (pp. 286-289). The Model System is thus the ‘core meaning’ to which the user interface, various technical services, and other pure fabrication classes is added to make up the computer system (see Figure 10). This makes the Model System the sum of what is referred to as domain classes (collectively called the domain layer, e.g. Larman 2001) and a popular rule of thumb is that there are 5 non-domain classes for every domain class.

Figure 10: The Model System is the ‘core meaning’ to which the user interface, various technical services, pure fabrication classes are added to make up the computer system.

Although the model and the user interfaces are presented as separate concerns, Madsen et al. state that “the general approach of an object-oriented user interface is that representative elements of the model should have a representation on the screen. The user should have a feeling of manipulating the physical model directly.” (p. 312) I believe this to be an immensely important statement, one that I feel has been neglected almost completely in the object-oriented community.

A critical view of object orientation

Knowledge distillation and purification

Since the Model System is ‘core meaning’ as we concluded from the previous section, it begs the question of how we in this ‘process of knowledge’, as Madsen et al. designate it, find the ‘right’ core meaning from the wealth of representations? How do we distill/purify/crystallise meaning, i.e. ‘peel off’ the unnecessary complexity to reveal the underlying core meaning during the analysis phase? In this regard, Amarel’s quote that “solving a problem simply means representing it so as to make the solution transparent” (Amarel 1966) seems a tempting choice. In RUP and UP for example, an important aspect of the various representations (called design artifacts) is their mutually constraining relationship, called traceability (i.e. Jacobson et al. 1999: p. 10). Through these constraining relationships one can regard the development process as one of representation and re-representation (through constraining design artifacts) until the solution is transparent; one has arrived at an adequate object design, i.e. the core meaning. In classical waterfall development, as epitomised by Royce (1970), this stance was more outspoken and pure. Waterfall development may be thought of as ‘knowledge engineering’ in which knowledge is something ‘out there’ that can be ‘engineered’ to fit the representational format of the code: Each successive stage of waterfall development is about changing the ‘input representations’, i.e. design or analysis artifacts, resulting from the previous stage and yielding ‘output representations’ until the ‘solution’ has been reached.

Modern development process models like UP and RUP of course embed an iterative approach but my point is that as long as meaning is thought of as ‘underlying’, it will seem tempting to consider modelling as a process of meaning deduction or distillation. To elucidate the dangers of viewing meaning this way, let me explain one context in which Amarel’s quote is often heard, namely isomorphic problem research (e.g. Zhang 1991 and 1997) with its highly representationalist assumptions.

Isomorphic problems are problems whose underlying formal structures are claimed to be the same but whose surface structure vary (iso- means same, -morph means form). By detecting the underlying structural isomorphism the problem solver is able to solve a novel problem by superimposing the structure of the known problem upon the novel problem. Thus, the solution is assumed absolute and to be found somewhere ‘behind’ the problem’s polymorphic representational gloss, which may be more or less conducive/suggestive of its solution. A classical research example is comparison between Tic-Tac-Toe and the Number Scrabble game (also called ‘15 puzzle’) (e.g. Newell and Simon 1972, Zhang 1997, 188pp).

         Tic Tac Toe              Number Scrabble

            |   |                      |   |          
            |   |                    4 | 9 | 2         
         ---+---+---                ---+---+---
            |   |                      |   |          
            |   |                    3 | 5 | 7        
            |   |                      |   |          
         ---+---+---                ---+---+---
            |   |                    8 | 1 | 6        
            |   |                      |   |   

Figure 11: The Tic Tac Toe and Number Scrabble ‘problem isomorphs’.

The Tic Tac Toe or ‘three-on-a-row’ game does not require any introduction. The Number Scrabble on the other hand is played by two players taking turns selecting a tile. When one player has selected three tiles that add up to 15, that player is the winner. As you can see, the numbers are arranged in such a way that each vertical, horizontal, and diagonal row of numbers add up to 15, thereby allowing the ‘problem’ to be solved the same way as Tic Tac Toe.

This genre of problem-solving research thus views problems as having underlying formal structures which can somehow be ‘dug up’ from the muddle or mask of representational gloss Consequently, solving a problem simply means representing it so as to make the solution transparent. For example:

“That representation makes a difference is a long-familiar point. We all believe that arithmetic has become easier since Arabic numerals and place notation replaced Roman numerals […] All mathematics exhibits in its conclusion only what is already implicit in its premises […] Hence all mathematical derivation can be viewed simply as change in representation, making evident what was previously true but obscure.” (Simon 1981: p.153)

In object-oriented literature one finds countless allusions to problem-solving research and problem isomorphs as well as Simon’s and Amarel’s stance toward ‘masking representations’ (regrettably, they remain allusions as I have not found any thorough investigation of these phenomena in relation to object orientation). To my mind, this position that the ‘underlying properties’ are the ‘real properties’ of reality as opposed to its ‘masking representation’ is clearly present in Madsen et al.’s framework (as well as how object orientation is conceived and practiced elsewhere). This becomes particularly evident through their preoccupation with the problems of giving Aristotelian definitions to ‘fuzzy’ or somehow more ‘natural’ or ‘real’ phenomena. Through this discussion, they seem to espouse the position that if one could find the right representational format to exactly match the meaning out there in the Referent System or in the minds of the people, a solution or ‘perfect Model System’ could be reached. The assumption that there is some ‘correct’ representational format of what is ‘out there’ has led to research such as Skou (2001), which is an investigation of how categorisation research in cognitive psychology hangs together with programming-as-understanding in an object-oriented language. But as Skou found out, it is a misguided research agenda to attempt to peer into the minds of people to discover a supposed format of long-term memory or categories-as-vehicles-for-inference with the aim of designing an according programming language with perfect model-world or model-mind correspondence as a result.

Thus, o ne gets the impression that a system development process is a process of meaning distillation or purification, through which complexity is ‘peeled off’ and the underlying order/truth is represented in the Model System (in a matching representational format to minimize the ‘conceptual distance’). But this misses a point in the problem-solving experiments (to continue analogising to problem-solving research). The unequivocal conclusion of such experiments (e.g. Sternberg 1996: p. 360) is that the test subjects do not necessarily experience two isomorphic problems to be ‘the same’. So although you can formally demonstrate an underlying structural isomorphism, the problem solver does not think they are ‘the same’, nor does she consciously solve two isomorphic problems by superimposing the structure of the known problem upon the novel problem. So are the problems really the same? To some scientists, maybe. But to the test subject, no.

In systems development we are in the same situation as something real to the programmer (the model) may be experienced in a completely different way by the user through the user interface. How can something be real to both parties and what then, is most real? It is quite obvious that we cannot really answer that question unless we dig a level deeper and look at some more philosophical matters.

The epistemological and ontological assumptions of object orientation

In the following I shall try to identify the epistemological and ontological assumption of class-based object orientation, which I have commenced in the above. What I intend to show is that because of Madsen et al.’s typically Scandinavian view of programming they are, in their conceptual framework, trying to answer what understanding and, more importantly, meaning are in relation to object-oriented formalisation. To my mind, this question cannot be answered from their underlying philosophy of science standpoint, as will hopefully be evident in the following. In the theoretical framework I propose later in this report I hope to be able to answer this central question from a less modernistic standpoint and, most importantly, doing so without loosing the ability to speak of formalisation and technical rationality. This should free us from a situation where we either answer what meaning and understanding are, or we answer what technical rationality and formalisation are.

Dissection along two dimensions

As mentioned, Madsen et al. believe programming to be first and formerly a question of understanding and they believe object orientation to be particularly well-suited for this. Of the three main advantages of object orientation they mention, they emphasise this aspect, Real-world Apprehension, over stability of design (being able to change an object-oriented program in the face of changing requirements with a minimum of reprogramming of the fundamental objects; the model), and reusability (being able to reuse components/objects in several contexts or programming projects). Real-world Apprehension captures the idea that “object orientation is close to our own natural perception of the real world” (Madsen et al. 1993: p. 2).

The concept of Real-world Apprehension rests on assumptions about “natural perception”, “the real world”, and lastly the connection between these two (“our own natural perception of the real world”). To analyse Madsen et al’s assumptions concerning these aspects can be analysed along several dimensions. I have chosen to use Hirschheim and Klein (1989) as inspiration, which in turn draw on Burrell and Morgan (1979) as inspiration. Hirschheim and Klein make an analytical breakdown of the assumptions of software development projects and map the assumptions onto two dimensions; an epistemological dimension called subjectivist/objectivist and an ontological dimension called order/conflict.

At one end of the epistemological spectrum, we find the objectivist position, the essence of which “is to apply models and methods derived from the natural sciences to the study of human affairs. The objectivist treats the social world as if it were the natural world” (Burrell and Morgan 1979: p. 7 as cited in Hirschheim and Klein 1989: p. 1201). The subjectivist position at the other end of the spectrum claims that methods and models from the natural sciences cannot adequately be applied to social affairs and posits that understanding must be understood from the subjective experience of individuals (Hirschheim and Klein 1989: p. 3).

Turning to the ontological dimension, we find a conception of the world as either ordered or inherently in conflict on the respective ends. In the ordered view, the world is viewed in terms of integration and functional coordination and in the conflict view, the world is viewed in terms of change and disintegration (Hirschheim and Klein 1989, p. 1201).

Figure 12: The analytical dimensions used by Hirschheim and Klein 1989 to analyse the assumptions of system development approaches/projects and their consequent outcomes. Adapted from Hirschheim and Klein 1989 p. 1202.

To exemplify two positions on this ontology/epistemology scatterplot, subscribing to an ordered ontology and an objectivist epistemology makes systems development instrumental reasoning (p. 1203). If the world is ‘out there’ and essentially agreed upon (ontology), and reasoning and understanding (epistemology) can be ‘correct’, i.e. methodically correct as in a natural science, systems development becomes a question of deducing a solution, much like an engineering task. Another position, that in many ways is the opposite of the former, subscribes to an ordered ontology, at least at the micro-sociological level (e.g. “our group/department/project is not be characterised by disintegration although we do not necessarily agree upon the world”). On the epistemological side, this position holds a subjectivist epistemology, which makes understanding both relativistic vis-à-vis the individual and at the same time non-individual in the sense that subjective experience may be culturally determined. Theorising, e.g. discerning economic laws, is one way of imposing order on the world, a sort of sense-making of what is essentially too complex and elusive. Turning to systems development, this makes the system developer become a facilitator or sense-maker. “There is no objective criterion that distinguishes good and bad systems. It all depends on what the parties come to believe to be true” (p. 1205). Good and bad is determined by the outcome of the interpretation processes whereby the subjective consciousness becomes the locus of meaning and the basis of any ontology.

The above is not only an allusion but a forthright drawing of the lines between what I believe are the assumptions of Madsen et al.’s definition of object orientation and the assumptions which I believe are necessary for a reconceptualisation of object orientation. My opinion is that while Madsen et al. may explicitly sympathise with the latter position, their backbone or tacit assumptions are still entrenched in the former position, which makes their attempt to define object orientation as programming-as-understanding impossible.

To further analyse the assumptions of object orientation, I have chosen an analytical stance similar to Hirschheim and Klein, only I have modified the order/conflict dimension and chosen to call it underlying order/continual ordering to turn the focus from political power struggles, or at least (dis)agreement about the states of world, to considering how social order and meaningfulness is sustained and reproduced, i.e. how sense on a sociological level is established.

 

Figure 13: These are the broad-brush analytical dimensions of my analysis. They are ‘broad-brush’ because they should be considered in terms of their semantic convenience rather than their absolute correctness. For example, the positions of epistemological objectivism and ontology of continual ordering seem to be mutually exclusive.

I shall now try to identify elements of Madsen et al.’s conceptual framework on these dimensions.

Ontology of ‘underlying order’ versus ‘continual ordering’

In the dimension of ‘underlying order’, both the social and the natural world is seen either as having an underlying order that we can research and thus deduce the nature of or as being something that emerges on the background of continual ordering efforts of the actors (both technical and social actors). The former is the foundation of most natural sciences and can be found in its most naïve and archetypical form in logical positivism, the early Wittgenstein, and quintessential cognitive science or Artificial Intelligence (AI) research. For example, Herbert Simon, a quintessential AI researcher sums up that…

“The central task of a natural science is to make the wonderful commonplace: to show that complexity, correctly viewed, is only a mask for simplicity”. (Simon 1981: p.3)

Within social theory, the view of underlying order is epitomised by the theory of Durkheim (e.g. 1933) and partly Weber (e.g. 1930). The latter position on the other hand, that of continual ordering, is found in the later Wittgenstein, in the social theory of Schütz (e.g. 1967) and Garfinkel (e.g. 1967) and as a basic assumption of Science, Technology, and Society Studies (STS); e.g. Latour (1987,1988,1991).

Underlying order vis-à-vis object orientation

Madsen et al. emphasise that not all aspects of reality can be modelled and propose information processes as the kind of phenomena that are particularly suitable to be modelled.

An information process is a process where the qualities considered are:
Its substance, the physical material that it transforms.
• Measurable properties of its substance, the results of measurement represented by values.
• Transformations of its substance and thus its measurable properties”. (p. 288, original emphasis)

Substance is thus physical material being transformed and is furthermore characterised by a certain volume, a unique location in time and space, and an identity (p. 288). The measurability of substance is obtained by measurements, and transformations of substance can consequently be identified by differences in measurements over time. This leads Madsen et al. to some guidelines as to the selection of phenomena from the Referent System when going about system development:

“We have to consider tangible things where the main aspect is substance, we have to consider measurable properties of substance, and finally, we have to consider the transformations on the substance” (p. 289).

This stance toward the physical and measurable is to my mind a classic attempt to de-contextualise information or somehow make it more ‘neutral’. It is assumed in the above that the physical world is somehow less controversial or subjective than the social or personal, psychological world and therefore holds some kind of ‘core data’. In this view, “Reality consists of objects, properties, and processes that are directly observable.” (Hirschheim and Klein p. 1203) and validity is thus established through some naïve realism by checking the objects’ correspondence to reality.

In my view, Madsen et al.’s approach bumps into this kind of correspondence problem; of how to conceive of the correspondence (and both ends of it) that constitutes the representational gap or conceptual distance. If one assumes underlying order, correspondence can indeed be checked, but if one agrees that social order or meaning arises from the continual ordering processes it becomes these continual and ever-changing processes that determine the validity.

In this regard, Schefe (2000) gives a thorough critique of Madsen et al.’s framework, saying that it rests on “several misconceptions, especially in the notions of ‘substance’ and ‘representation’ as a correspondence relation, and even of ‘object’.” (2000: pp. 3-4). Schefe calls Madsen et al.’s conception of modelling ‘objective modelling’ as it seems to be conceived as a correspondence theory. Schefe holds a different view and state that “Reference is not an isomorphic relationship between descriptions and their subjects” (p. 4), the abovementioned naïve realism, and consequently concludes that “The paradigm of ‘objective modelling’ has to be replaced by a paradigm of ‘purposive description’ shifting the focus of Software Engineering research to non-formal methodologies.” (p. 1).

When positing this kind of view of reference as an isomorphic relationship between descriptions and their subjects or likewise positing an underlying ontological order, a Model System can thus be highly ‘valid’ but prove to be inappropriate once used. To my mind, this yields a schism between what I choose to call information-in-use versus information-by-itself and what I mean hereby is that information ‘is’ not but emerges in use. This schism can also be found in theories of learning, where on one end of the spectrum we find Shannon and Weaver’s (1963) classical model of communication as information transmission and on the other end theories like Luhmann’s systems theory (e.g. 1995). In the latter theory, information is not transmitted but is built up inside consciousness on the background of external ‘irritation’ and from what is already in consciousness. According to the latter theory, it would be senseless to speak of what ‘information X really is’ (e.g. in the Model System) if it were only ‘correctly intercepted’ (e.g. by a user) and not ‘distorted in the transmission’ through some medium (e.g. a GUI, language in general, symbols, etc.).

Furthermore, by emphasising measurable properties of physical substance (as opposed to e.g. the ‘experience’ of users) Madsen et al. can be said to emphasise this analytical role of the system developer (i.e. peeling off the layer of complexity) as opposed to the constructive and interventionistic. This deductive or analytical stance is also evident in the choice of words of many system developers: For example, analysis phase as opposed to ‘vision’ phase, or requirements gathering as opposed to requirement construction (c.f. Beyer and Holtzblatt 1998).

According to the Wordnet dictionary (Wordnet) one of the definitions of analysis is “the abstract separation of a whole into its constituent parts in order to study the parts and their relations.” Its antonym is something ‘synthetic’ which is defined as “not genuine or natural”. This, perhaps tacit, choice of words, I believe, is indicative of the schism between, on one side, underlying order, i.e. a ‘core’ that can be deducted via analysis, and, on the other side, continual ordering, i.e. the assumption that nothing is genuine or natural but that genuineness, naturalness, and order is continually ‘constructed’ and ‘restored’.

In terms of consequences, assuming the developer’s role to be analytical may lead to a de-emphasis of participatory design practices as “users’ statements are merely one of many possible ways of deducing the underlying truth”. This position reduces the users’ experiences and opinions to variant subjective experiences of something invariant.

The independency of ‘meaning’ from the ‘subjective’ meaning-forming process of the user also corresponds well with how a system development process is normally conceived: First, the most objective part is made. This is the model, i.e. the abstract, deducted truth lying represented, but hidden, in the belly of the software. Toward the end, the interface is added onto the model which acts as the users’ interpreter vis-à-vis the software model. That the GUI is only a window to the underlying truth is emphasised by designating it as presentation layer. To use yet another analogy, think of a database with its user interface: Is the interface only a window of the ‘real’ (objective) information or does the (subjective) use of the interface and the-information-as-presented continually constitute the ‘realness’ (objectivity) of the information?

Although Madsen et al. describe object orientation as a perspective on the world (p. 284), and hence do not explicitly subscribe to a hardcore correspondence theory of truth, in my opinion they nevertheless uphold a notion of underlying order. As mentioned earlier, I believe that for their project of explaining programming-as-understanding to come to fruition one needs to change the philosophy of science assumptions of such a explanation. For example, if the assumption of underlying order is abandoned, the model of Madsen et al.’s conceptual framework may be conceived differently. Its meaning may become real in its consequence, i.e. the result of the future meaning-forming process of the users, and the developer must thus attempt to anticipate and influence this meaning-forming process. This kind of suggestive model would suggest interpretation of it and thus have a higher probability of becoming ‘real’ as opposed to what may be called a descriptive model. I thus distinguish between a descriptive model (Madsen et al.’s model that describes the real and imaginary parts of the world) versus a prescriptive or suggestive model (which is suggestive of its interpretation and whose claim of ‘realness’ is equal to the outcome of this interpretation) #6. The latter model entails re-conceiving the notion of model as a more social model, an act of communication between developer and user.

Cognitive and quasi-epistemological assumptions of object orientation

Turning now to the epistemological/cognitive assumptions of class-based object orientation, Madsen et al. state that “object orientation is close to our own natural perception of the real world” (Madsen et al. 1993: p. 2). To elaborate on this, Madsen et al. quote Krogdahl and Olsen (1986):

“The basic philosophy underlying object-oriented programming is to make the programs as far as possible reflect that part of the reality they are going to treat. It is then often easier to understand and to get an overview of what is described in programs. The reason is that human beings from the outset are used to and trained in the perception of what is going on in the real world. The closer it is possible to use this way of thinking in programming, the easier it is to write and understand programs”. (translated from Norwegian by Madsen et al. 1993)

The quote clearly illustrates the fundamental belief of the object-oriented community that ‘there is something about object orientation and thinking’, an alleged connection with cognition/epistemology. The quote unfortunately also illustrates the striking absence of any scientific basis for making such an argument. Referring to a layman’s conception of “we probably think in objects” is simply not good enough, although compelling.

One may argue that we perceive in chunks (percepts formed from sensations), that we write in words (lexical chunks), that we speak and hear in phonological and auditory chunks, etc and thus conclude that ‘phenomena’, whether lexical, perceptual, or whatever, is an adequate description of these pervasive chunks. But the mere presence of a superficial structural resemblance (the alleged ‘chunkness’ of the world and mind) does not suggest any resemblance between the phenomena in the head, in the software, and in the world (anymore than a carrot has a low representational gab vis-à-vis a triangle due to their similar perceptual structure). We must further clarify the assumptions of this thinking in objects/phenomena.

Madsen et al. elaborate on how knowledge and meaning is formed in the following paragraphs:

The process of knowledge may be split into three levels:

1) The level of empirical concreteness. At this level we conceive reality or individual phenomena as they are. We do not realize similarities between different phenomena, nor do we obtain any systematic understanding of the individual phenomena. We notice what happens, but neither understand why it happens nor the relations between the phenomena. In the programming process this corresponds to a level where we are trying to understand the single objects that constitute the system. We have little understanding of the relations between the objects, e.g. how to group them into classes.
(2) The level of abstraction. To understand the complications of the referent system, we have to analyze the phenomena and develop concepts for grasping the relevant properties of the phenomena that we consider. In the programming process this corresponds to designing the classes and their attributes and to organizing the classes into a class/sub-class hierarchy. At this level we obtain a simple and systematic understanding of the phenomena in the referent system.
(3) The level of thought concreteness. The understanding corresponding to the abstract level is further developed to obtain an understanding of the totality of the referent system. By having organized the phenomena of the referent system by means of concepts, we may be able to understand relations between phenomena that we did not understand at the level of empirical concreteness. We may also be able to explain why things happen and to predict what will happen. (Madsen et al. p. 295-296)

In other words, Madsen et al. claim that we identify phenomena (1), abstract properties from them and form concepts (2), and lastly we establish relationships between concepts and on the basis of this extrapolated knowledge we can predict, plan, and take action (3). The quote gives us something to work with in our attempt to further elucidate Madsen et al.’s epistemological assumptions.

First of all, to my mind this is a clear subject-object dualistic view of cognition/epistemology as we (the subjects) “read off” the world (the object), form a representation of it and make predictions from this model. With a stance toward more subject-object reciprocity concepts would not take such a primary role as meaning would arise in the interaction or borderland between subject and object and thus render superfluous such an elaborate representation (more on this matter shortly).

Secondly, due to the salience of concepts Madsen et al.’s conception of cognition/epistemology does not seem particularly perceptually oriented (cf. “To understand the complications of the Referent System, we have to analyze the phenomena and develop concepts for grasping the relevant properties of the phenomena that we consider”). This is further underscored by their emphasis on ‘substance’ and their advice that we should, in our modelling process, “consider tangible things where the main aspect is substance”. According to my Webster's Revised Unabridged Dictionary (1913) two possible definitions of substance are “that which underlies all outward manifestation” and “that which is real, in distinction from that which is apparent”. In other words, appearance seems to be the antonym of substance (and thus ‘underlying’)!

Madsen et al. themselves define substance as something “characterized by a certain volume and a unique location in time and space” (p. 288) and thus define it with reference to a mathematical or geometrical universe (absolute), not with reference to the phenomenal world of an actor (relative). This view clearly leans towards computational theories of cognition as the actor performs his cognising from the premises of a mathematical, geometrical or otherwise natural science universe (more on this later).

Thus, we are presented with a view of cognition, thinking, and knowledge creation that relies on (visual) perception to provide the information from which we form/distil meaning and knowledge, as well as a view of perception as not being inherent to cognition or ‘real thinking’. ‘Real thinking’ (the level of thought concreteness) as opposed to the aforementioned unreflecting (visual) perception (the level of empirical concreteness) is described by Madsen et al. to happen by way of represented concepts, using their relations and intensional properties for reasoning.

The problematic heritage and its alternatives

Object orientation’s assumptions as a product of its historical context

What I have been alluding to in the previous paragraphs are the distinctions between amodal and modal theories of knowledge (defined below), and the distinctions between information processing and non-information processing theories of cognition (both distinctions found in cognitive psychology and other disciplines concerned with understanding and epistemology). When Madsen et al. wrote their book in 1993, psychological theories had for the previous 25 years, or since 1956 to be exact, been dominated by theories (either computational or non-computational information processing theories) researched and understood from a natural science standpoint. In 1956, at a meeting at MIT critical to the discipline of cognitive psychology, Chomsky presented a paper on his transformational grammar, Miller presented his famous paper “The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information" (1956), and Newell and Simon discussed their computational General Problem Solver model. In the same period, Donald Broadbent at the Applied Psychology Research Unit in Cambridge integrated ideas from the Human Factors discipline with new ideas in Information Theory. Broadbent’s work yielded a quintessential information processing view of cognition as a sequential series of processing stages where basic stimuli (information) was processed in a series of processing stages and ended up in long-term memory storage (Broadbent 1958). This was later extended by Atkinson and Shiffrin’s (1968) influential theory of the distinction between short and long term memory.

This ‘information storage’ was in the subsequent years researched in terms of the very same Aristotelian concepts we have been treating so far (c.f. e.g. Kintsch 1980), either as storage containers, or vehicles for reasoning about ‘stored information’. And because of the above theories, notably Chomsky’s, and the advances of the computer and programming languages, cognitive representations were thought of in terms of textual representations, e.g. ((Chair=C1),(back=b1),(seat=s1),(legs=ll)). This sort of textual symbol information could be described in a programming language and be processed by a computer. Something visual could only be represented insofar as the visual could be transformed and described by the symbolic ‘format’ of the information processing system. Meaning and information was thus meaning-about-something.

This melting pot of theories provided the grounds for what we know as information processing cognitive psychology and what held the sometimes disparate theories together was the analogy between the mind and the computer as information processing devices. Computer science and cognitive psychology ‘fed off each other’ and the mind was (perhaps consequently) understood from a mathematical and natural science standpoint.

When Kristen Nygaard and others in the 60s, 70s, and 80s were pondering on how object orientation hangs together with thinking, they did so in the heyday of computer-mind analogy and on the backdrop of the aforementioned very influential theories. T he historical context thus made it very compelling to understand ‘understanding’ from a natural science standpoint as well as consider Aristotelian concepts as keystone therein. The stance of the conceptual framework for class-based object orientation as expressed by Lehrman Madsen, Thomsen, and Nygaard is to my mind therefore a very expectable perpetuation of the most prominent cognitive theories of their time.

In the following I shall review some of the aforementioned distinctions and explain how they may help us move toward a redefinition of object orientation. From observing the broad-brush directions of research in cognition and understanding for the past 40 years and redefining object orientation on the basis of theories different from the mainstream information processing approach, different from the classical amodal approach as well as on alternative underlying philosophy of science assumptions I believe we may develop both a theoretical framework as well as a base of empirical research supportive of a redefinition.

Orthodox information processing accounts of cognition and perception

In the information processing view found in cognitive psychology, humans (the subjects) pick up information about their surrounding world (the object (purposely written in the singular)), interpret that information, store it, and retrieve it later. Hence, the canonical terms are memory ‘encoding’ and ‘retrieval’ (e.g. Baddeley 1997) and the ‘content units’ of these processes are information about the world.

Such an act of ‘processing information about the outside world’ starts with perceiving, auditorily, visually, tactically etc. and perception is thus an element in the cognitive activity. To take the most principal of our senses as example, visual perception takes place when light energy hits a photoreceptor in the retina. Stimulation of the receptor is said to occur when the amount of energy exceeds a certain amount, an excitation threshold. Subsequently, simultaneous stimulations of photoreceptors, jointly constituting the retina, are merged with each other, as well as with other information like the angle between the two respective retinal images acquired from the muscles controlling the eye (binocular disparity). These basic sensations are then somewhat meaningfully arranged through processes of perceptual segregation, e.g. the gestalt principles of form perception (e.g. Köhler 1947) to form percepts. These wholes are then sent on to our higher-order cognitive processes for further ‘processing’ through a process of transduction, which is the process of converting one form of energy into another, for example from the electromagnetic energy (light) of the visual stimuli into neural energy. Where perception stops and higher-order cognition takes over is of course an open research question, but what is important is that that this percept formed from the excitation pattern of the retinal image, is said to be converted or transduced into a neural pattern in working memory (currently the most widely accepted model of memory in mainstream information-processing cognitive psychology, c.f. Baddeley 1997). Owing to the transduction processes, consciousness can treat the result as ‘meaning’ and as such, perception and cognition are conceived as independent and modular systems in the brain, as distinguishable concerns so to speak (alluding to GUI/Model separability).

Evidently, this process is likened to information processing, as (visual) information is collected, refined, modified, or otherwise processed until it appears as an object of our consciousness or in long term memory, a distillate created from ‘basic’ information as light photons or gaseous particles (e.g. by olfactory processing). Meaning are thus chunks of ‘aboutness about the world’ so that our consciousness is able to tell us that ‘that thing’ (percept formed from the retinal image) is a ‘pull-down menu from which I am to choose an item’. In this view, if the transduction did not take place we would not have consciousness and not know which subset of the perceived information to store in memory.

However, this conception of meaning gives rise to what has been called the homunculus problem which is a problem of infinite regress: How do I recognise that thing as a pull-down menu, button, or likewise? What must be occurring in my mind is an image, a retrieval of the stored ‘pull-down menu meaning chunk’, which I recognise as matching the pull-down menu (external to my mind). But who is looking at that image in my mind's eye and recognising that it matches? And what's going on in that person’s mind, and so forth? This is the homuncular regress.

There are several other problems with the information-processing view and its assumptions such as the transductional processes, e.g. the symbol grounding problem, the Chinese Box Problem, intentionality-related problems (Searle 1980) and more, but the above criticism is the most central in relation to our present discussion of object orientation.

The modal/amodal distinction

The aforementioned question of transduction is at the heart of the distinction between modal and amodal theories of knowledge and meaning. Before the twentieth century, theories of knowledge were mainly perceptual with perception, as opposed to higher-order cognitive processes or a notion of ‘semantic memory’, as the locus of meaning. Since then, perception has been relegated the status of input device or recording system, from the outputs of which meaning and knowledge is extracted, as described above and by Madsen et al.

Both a modal and amodal theory of knowledge hold that perceptual states arise in sensory-motor systems after which a subset of the states is stored in some kind of memory. However, the amodal approach hold that this subset is transduced, transformed, into a new nonperceptual representational format. These representational structures are amodal because they bear no correspondence to the perceptual states that produced them, and the two systems (the perceptual and the cognitive) are thus assumed to be different and work according to different principles. Barsalou (1999) notes that it is partly because of the development of programming languages (!) that theories have taken this amodal turn, which he finds regrettable.

Examples of such ‘meaning storage formats’ are feature lists, frames, schemata, semantic nets, production systems, connectionism (Barsalou 1999: p. 578) and some of the quintessential references for this approach are Newell and Simon (1972), Fodor (1975), Pylyshyn (1984), and Haugeland (1985).

The modal approach on the other hand holds that meaning is not abstracted or extrapolated nor semantically different from the perceptual process that gave rise to the neural activation. An amodal theory of knowledge, such as Barsalou’s (1999), posits a convergence of the orthodox distinction of perception and cognition. In fact, Barsalou grounds cognition in perception by providing evidence that neural systems common to imagery and perception underlie functions that are normally, in amodal approaches, conceived as higher-order cognitive functions, e.g. conceptual knowledge, categorial inferences, reasoning, etc. Thus, to the extent that ‘the outside world’ is represented internally, a subset of the neural activation is stored but no transduction or meaning-extraction occurs in a modal theory of knowledge. The subset is stored analogically (bearing direct correspondence to the perceptual state that produced it) and in a post-stimulation situation, it can be retrieved and ‘replayed’ or ‘revived’, so to speak, in the perceptual system (hence modally). Once revived it then can be used for reasoning.

In Barsalou’s view, meaning is inherently perceptual as meaning is the act of perceiving or ‘replaying the perceived’, re-perceiving so to speak. Semantics cannot be abstracted from perception, it lies it to be found in the perception-as-process. But more on this later.

Figure 14: In an amodal theory of knowledge, the meaning of a chair is transduced from perceptual states into distilled meaning represented in a non-perceptual representational format. Figure reproduced from Barsalou 1999 p. 579.

Figure 15: In a modal theory of knowledge, the perceptual state is stored in an analogue format that bears direct correspondence to the perceptual states that produced the neural activity. Figure reproduced from Barsalou 1999 p. 578.

Aristotle and semantic memory research

I believe that Madsen et al.’s description of “the process of knowledge” (pp. 295-296 as cited above) represents exactly such an amodal view as described above and that Aristotelian concepts is their non-perceptual representational format. This would be highly typical of their time as Aristotle’s conception of meaning was similar to virtually all attempts of explaining the relationship between language and the world, i.e. meaning, in both linguistics (e.g. Katz and Fodor 1963) and psychology (e.g. Smith, Shoben, and Ripps 1974) prior to the late 80s (c.f. Kintsch 1980). While an Aristotelian approach may be appropriate to describe a logical system, it proved, according to many researchers (e.g. Baddeley 1997, Kintsch 1980), to be inappropriate for explaining how humans deal with meaning. As Kintsch remarked, basing psychological theories on logical models of meaning” has led semantic memory research into avenues of investigation that are barren and lead nowhere” (Kintsch 1980, p. 605). Madsen et al. acknowledge that other formats such as the prototypical may be ‘better’ (however ‘better’ is to be conceived) but they remain within an amodal and information processing approach, only suggesting an alternative format.

This is the reason I have chosen to call Madsen et al’s view “quasi-epistemological” as I believe their view is in line with a classical ‘raw’ and natural science approach to thinking. I myself believe cognition to be inherently perceptual and modal/analogical and therefore consider a ‘chase of formats’ unfruitful and misguided.

Object orientation as a relativistic or cultural epistemology

Madsen et al. use the term ‘natural perception’ to describe what object orientation makes use of. To my mind ‘natural perception’ or ‘natural cognition’ is an oxymoron. I do not believe of a generic or universal ‘cognitive model’ as would be present in a naked, savage man in a natural undomesticated wilderness. Similarly, I also find Madsen et al.’s reference to ‘natural’ or otherwise unproblematic ‘kindergarten thinking’ to be problematic.

As Hutchins (1995; e.g. p. 169) and countless others (Cole 1998 to name a canonical reference) have shown, our cognitive powers are cultural, artificial or otherwise ‘unnatural’ through and through. We do not exercise our thinking in ‘natural environments’ but create our cognitive abilities by actively constructing environments in which to exercise these abilities. For Hutchins (1995) the ‘constructed environment’ was a ship bridge featuring countless artifacts taking part in the navigation of the ship and in the ‘thinking’ activities of the sailors #7. Similarly, developers exercise their cognitive powers in an environment of object-oriented tools, design patterns, and general object oriented ‘culture’ #8. Thus, object orientation can be seen as a manifestation of just such a subjectivist or cultural epistemology: Computer scientists are trained in natural science thinking, which makes the assumptions of moderate realism in object orientation, that the world is best described in terms of objects and their properties, ideal for this target group #9. If object orientation were used for analytical purposes only, in line with the tradition of using formal methods for knowledge discovery (c.f. Dix 2003), or if it were not dependant on people outside the object-oriented culture I would not necessarily consider it problematic. However, as we have found out in the above, the object-oriented Model System is ‘future meaning’, which makes it highly problematic as it consequently presupposes a coinciding of the empirically realised meaning of the users and the developer’s intended meaning (formalised in the code). Therefore, the cultural epistemology of the object-oriented community is imposed on the users and as Paul Heckel writes in The Elements of Friendly Software Design (1984): “Our instincts and training as engineers encourage us to think logically instead of visually, and this is counterproductive to friendly design.”

To be more precise, I would say that object orientation’s advantage of so-called ‘real-world apprehension’ is about the representational gap between the unit of analysis as a perspective on the world (e.g. a world of functions, procedures, phenomena and concepts or the like) and the formalisms in which the unit of analysis is realised/modelled. But perspective means the programmer’s perspective and if an objectivist ontology is abandoned, codifying future meaning (being dependant on the meaning-formation process of other people than the programmers) in a way only meaningful to the programmer is problematic. Regardless of how close this perspective may be to our education as software engineers.

Toward a redefinition of the modelling process: From physical to suggestive modelling

Hirschheim and Klein’s analysis, as well as many others, show how strongly fundamental assumptions of systems development can influence the outcome. Similarly, the ontological and epistemological assumptions of object orientation will inevitably constitute a certain ‘inherent momentum’ and thus influence the contexts in which object orientation is used.

In the above section it has been claimed that object orientation is reminiscent of ontological and epistemological assumptions of modernity and the natural sciences, and some of the consequences of changing these assumptions have been briefly suggested. Such an update of object orientation, subscribing to an amodern stance, entails a change of the idea of object orientation, the act of modelling, and the meaning of the Model System. The Model System, I have claimed in the above, must assume a more proactive, suggestive, and purposive role toward the ‘construction of meaning’ vis-à-vis the user, which makes it much less secure as it depends on interpretation and the ordering process into which it will enter. The notion that the consequences of introducing a system into an organisation are unpredictable and will change the situation that it was originally designed for is by no means a new idea (e.g. the task-artifact cycle in Carroll et al. 1991). However, object orientation has so far tried to evade the problem by building a model of more ‘stable components’, namely physicality and abstracted truth, which is as far away from the place of interpretation (e.g. the interface and its context of use) as possible. Coordinating one’s code design with the ‘core data’, the underlying order, the deducted truth, is apparently perceived to be more ‘safe’ than coordinating with something as nebulous and fuzzy as the subjective interpretation of users. However, the subjectivity of interpretation does not mean it can escape all scientific explanation or cannot be trusted or relied on. In fact, coordinating with the interpretation or ordering process is the only way, I believe, to insure that the ‘realness’ of the developer’s model is the same as the ‘realness’ of the users’. This was my reason for suggesting a redefinition of the software model as suggestive and purposive and this makes validity become not a question of descriptive correspondence but of closeness of empirically realised meaning vis-à-vis intended meaning.

The above begs the question of which ontological and epistemological material to build such a suggestive and purposive model – this is the subject of the next section.

Theoretical foundations of a redefinition

In the following I shall present an alternative theoretical framework for object orientation. The way I do this, is to present a given theoretical position (or a group of positions) and subsequently pause to account for the consequences in relation to object orientation, an object-oriented system development process, or an object-oriented programming language.

Mind-body and subject-object dualisms

The notion that mind is a non-bodily entity, a ‘soul’, has always been pervasive in Western thinking and can be found in the writings of such early philosophers as Plato. Although many philosophers thus pondered on this matter long before Descartes, he is usually credited as the father of modern mind-body dualism (e.g. Stigen 1992: 387pp ) as expressed in his writings in the 15th century. For him, ‘mind’ meant a soul without physical substance, which according to him was its definitional distinction from structures such as the body and the brain. Res cogitans versus res extensa. This dualism is today an integral and unnoticed part of our way of thinking.

The subject-object distinction is also pervasive in most Western philosophical traditions from Kant, over Hegel, to modern cognitive psychology. The subject-object distinction is a distinction between the thinker and what the thinker thinks about, although this thinking can of course be reflexive as in self-conscious thought. In this view, the world is thus divided into an objective world of physical reality and subjective mental world of thoughts. Despite the fact that in many philosophies the subject-object distinction is only a distinction used for semantic convenience, it seems to have a certain negative ‘inherent momentum’ to it, leading us to ignore the complementarity of the subject and object.

Phenomenology and the action-meaning complementarity

The German phenomenologists were the first to contend with both mind-body dualism and the subject-object dualism (e.g. Husserl 1970). Edmund Husserl (1859-1938) founded the phenomenological tradition out of dissatisfaction with the contemporary trend for ‘scientification’ of the nature of everyday experience.

“Husserl was frustrated by the idea that science and mathematics were increasingly conducted on an abstract plane that was disconnected from human experience and human understanding, independently of questions of truth and applicability. He felt that the sciences increasingly dealt with idealized entities and internal abstractions a world apart form the concrete phenomena of daily life.” (Dourish 2001: p. 104)

In broad outline, the phenomenological tradition, as started by Husserl and elaborated by people like Heidegger, contends with the distinction between epistemology (the subjective mental world of thoughts) and ontology (physical reality) that was endorsed notably by Kant. Kant, like early Wittgenstein as we shall see below, posited that ontology is pre-given. Phenomenologists like Husserl challenged this ‘ontological firmness’ with the argument that by perceiving the world out there we construct it at the same time. We co-construct both the firm, the ontology/reality out there, and the perception of it at the same time. Consequently, it is through epistemology that we create ontology and this is the reason I mentioned that my epistemology/ontology distinction in the previous chapter was only for analytical/semantic convenience.

Husserl thus rejected the idea that an object could somehow be decontextualised or viewed independently and with reference to a scientific or mathematical universe as opposed to the subject. He rejected this kind of rationalised, abstract reasoning and turned instead his attention toward the pre-theoretical understanding and experience of everyday life.

Husserl introduced the term lifeworld, Lebenswelt, to capture the idea of our everyday pre-theoretical understanding. The lifeworld is “the intersubjective, mundane world of background understandings and experiences of the world. It is the world of the natural attitude and of everyday experience”. (Dourish 2001: p. 106). Only from the background of the lifeworld can scientific thought emerge.

Husserl’s work was elaborated in the writings of his student Martin Heidegger (1889-1976) who rejected subject-object dualism in a more radical manner than had Husserl. Heidegger used the term Geworfenheit, or thrownness in English, to describe the fact that we are ‘thrown’ into the world and cannot reflect upon it before standing in the middle of it. Our situated-ness/thrownness is our point of departure which further problematises the rationalistic ideal of decontextualisation and objective thought. As Dourish (2001: p. 107) notes, Heidegger’s stance is a reversing of cogito ergo sum in that my being-in-world, the sum, comes before the cogito.

Similar to Husserl, Heidegger argued that the subject cannot engage in detached rational reflection about an object as part of the meaning of an object is the act of perceiving it. When I perceive something I not only know what I have perceived but also that I perceived it. This was the first sign of a phenomenologist unity of action and meaning.

One aspect of Heidegger’s theory that is particularly relevant is his conception of breakdowns, the distinction between present-at-hand (zuhanden) and ready-to-hand (vorhanden) #10. When hammering, the hammer itself does not exist as an entity for the person hammering. Instead, the concernful activity of hammering exists as the hammer is ready-to-hand, its use is transparent to use a popular HCI expression. Only in a situation of break-down will the person focus on the hammer to detect what is wrong with it or maybe reorient it in his hands. It is the sum-ergo-cogito from before as the hammer was there before reflection began. Additionally, we first appreciate and experience the hammer or the act of hammering ‘as-is’, and only then discover and form opinions about its properties. First, we start hammering and only in the event of bending the nail do we learn how to use the claw to straighten out the nail. The nail-straightening property of the claw arises on a background of readiness-to-hand and it is thus “[…] meaningless to talk about the existence of objects and their properties in the absence of concernful activity, with its potential for breaking down.” (Winograd and Flores 1987: p. 37)

Wittgenstein and meaning being use

Wittgenstein’s career was divided in two opposites, marked by the two books Tractatus Logico-Philosophicus (1923) and Philosphische Untersuchungen (1953). The main theme of Tractatus Logico-Philosophicus was the relationship between language and the world the language is about. He began his book by stating that “The world is the totality of facts”, thus espousing a logical positivism that reality, both the physical and social world, is ‘out there’ and its existence and true nature can be established through logical deduction. Based on this positivist ontology he argued that each unit in a language corresponds with a fact in the world (the meaning of the word). This positivist stance, not only as expressed by Wittgenstein but many other positivists, was criticised by, for example, Bertrand Russel by way of his famous paradox and by Gödel’s theorem.

In his later work, “Philosophische Untersuchungen” (1953), Wittgenstein took a radically different perspective. Instead of a positivistic correspondence theory of language, Wittgenstein now claimed that meaning was not established by a mapping of facts with words, but established through language games (e.g. Hartnack 1994: p. 67 pp). Language games resemble real games in that the linguistic activity is constituted by rules for the use of words/expressions and it is the boundaries of these rules which delimit/demarcate the meaning of a word. The rules for the use of a word do not precede the meaning of the word and the meaning the word does not precede the rules for its use. Instead, they co-exist and are inseparable. Thus, the meaning of a word is how it is used, because once one has learned to use it (the rules of its use) one will also know its meaning: “Die Bedeutung eines Wortes ist sein Gebrauch in der Sprache” (Wittgenstein 1953: §43 (p. 20)).

In broad outline, the early Wittgenstein may be said to be a Kantian due to his ontological assumptions (the world as the totality of facts). The late Wittgenstein, on the other hand, may be designated as phenomenologist due to his position that meaning is formed on a background of experience and language practice, which is a clear perpetuation of the phenomenologists’ accentuation of the complementarity of action, experience and meaning. This stance also points ‘forward’ from personal accounts of meaning to sociological accounts, to “communities of linguistic practice”, which I shall elaborate on after the next section where I turn to the applicability of the aforementioned theoretical positions to object orientation and how they may alter the idea of object orientation.

Applicability in relation to object orientation

From the late Wittgenstein we may conclude that context, or the boundaries of the language game, is what demarcates the boundaries of meaning. Meaning is not inherent to words or phenomena in the world but is established in context or language practice. In object-oriented terms, a phenomenon is a unit of meaning and defined as “a thing that has definite, individual existence in reality or in the mind” (Madsen et al. 1993: p. 290, original emphasis). With a late-Wittgensteinian approach, it becomes senseless to speak of units of meaning residing somewhere or meaning being somehow physical or otherwise ‘out there’ as opposed to established.

From the phenomenological tradition, we concluded that practical action in the world is foundation of conscious experience (Dourish 2001: p. ix), which is another way of saying that “We do not relate to things primarily through having a representation of them” (Winograd and Flores 1986: p. 33). I do not know the attributes of a hammer or somehow use them to guide my hammering but I know how to hammer.

Consequently, one does not primarily relate to objects in the GUI by deducing their underlying properties and their reference to the world, which makes it pointless to reference the ‘true meaning of systems’, i.e. asking the developer to give the authoritative account of what the GUI-object really consists of and how it really communicates with the software model in such-and-such a way. Assuming some underlying stability of representation or referencing “what really goes on” only reinforces the user conception that the inner workings of systems are voodoo magic and lack of confidence in what may be called a principle of “what-you-see-is-what-it-is”.

Since practical action is the foundation of conscious experience and a notion of ‘true meaning’ as a singular ontology must be abandoned according to the aforementioned theoretical positions, designing for intended use becomes more important than designing to reach the most truthful/correct (singular) ontology, as we are inclined to do by Madsen et al.’s notion of physical modelling. This is a further argument for our notion of a suggestive Model System as we must assume practical action is the cradle of meaning. Since the Model System is future meaning and that we aim for the intended meaning of our Model System to coincide with the user’s, phenomenology provides us with a guideline as to how this can be achieved; namely through practical pretheoretical action in concernful activity. The only way we can aspire to exert any control over how meaning is formed, i.e. to ensure the ‘validity’ of our Model System, is by being suggestive of practical action, to suggest meaning-in-use. I thus believe we need to design objects in the Model System that users encounter directly, rather than abstractly. ‘Properties’ are mainly properties insofar as they are directly encountered, as opposed to deduced or logically abstracted. They become real in their consequences (alluding to the pragmatist Thomas Theorem in Thomas and Thomas 1928)!

Another way of expressing this is that we should design commitment to use versus static meaning. This is the reason information appliances ( Norman 1999a) are said to epitomise user-friendliness: through their physicality and specialisation they show a commitment to use, which makes them easier to get “a hold of” conceptually, as opposed to the general-purpose, non-specialised desktop computer. Similarly, conceiving of objects and phenomena, in an object-oriented sense, as entities with a commitment to use would make them easier to comprehend and in turn make the coordination between the Referent and Model System easier. What I am arguing against here, is a complete separability of application semantics and user interface with the tendency of the application semantics to have the same general-purpose and abstract nature as the desktop PC in order to accommodate the changes in the supposedly instable GUI. This may result in flexibility (termed ‘stability of design’ in Madsen et al. 1993) but unfortunately also in a ‘comprehensibility muddle’. I do not find it problematic for a programmer to think in terms of abstraction, polymorphism, or layers of indirection per se but one should be weary of the inherent momentum of such mechanisms. As I shall illustrate in the section “ Toward a redefinition of object orientation” it is possible to maintain flexibility while avoiding the muddle, thereby taking seriously the stance that object-oriented programming is first and formerly understanding.

Moving on, the notion of lifeworld reminds us of the pre-theoretical lived experience upon which all else, including analytical thought, is built. Validity of meaning, when codifying meaning in object-oriented terms (i.e. deciding whether it is more ‘correct’ that a given object should have these and not those properties), should as a consequence be established on the basis of a correspondence to the users’ lifeworlds, not on the basis of a correspondence to an abstract mathematical universe #11 which is supposed to hold more ‘unproblematic’ or objective meaning (alluding to substance and physical modelling). The very same shift from viewing meaning as residing in the-thing-in-itself to viewing meaning derived from the-thing-in-context is what has happened in the HCI community. In the 80s, right/wrong was established by reference to the singular ontology of the human cognitive apparatus. Later, the HCI community took a ‘contextual turn’ #12 and right/wrong was established by reference to context, to situatedness, and thus increasingly to a sociological level of description. It was a move from Human Factors to Human Actors as Liam Bannon (1991) put it.

Lastly, what I believe one may also conclude from phenomenology is that representations are as representations do and what they do is get people to act. Representations are as representations are experienced and acted upon. In other words, we need to pay attention to “how computation participates in the world it represents” (Dourish 2001: p. 20), not merely treat it as standalone, passive representation. In this sense, representation and participation are two sides of the same coin, which is why I believe we should conceive of the object-oriented Model System in a way that includes how it participates in the world it represents, as a sort of social actor.

This notion of actor brings us further into in the direction of sociology or at least micro-sociology, from which I have chosen ethnomethodology as a further addition to the theoretical framework upon which to base an update of object orientation.

Ethnomethodological accountability and transparency of use

Alfred Schütz (1899-1959) took up phenomenology and applied it to the social world. His main theme was intersubjectivity or more specifically how we establish a common meaningful experience of the world despite the fact that our understanding and experiences are our own. Some theorists like Maximilian (Max) Weber had posited that the social world was fundamentally ordered by ‘underlying’ principles and causalities and by studying social action that appeared meaningful, sociology could extrapolate these principles and define the criteria for meaningfulness. Order/rationality thus precedes and underlies the action of individuals. Schütz, however, argued that the social world is not meaningful per se, but becomes meaningful through the action of individuals. It is this process of how social acts become meaningful through mundane activity that Schütz turned his attention to. Schütz’ reorientation of sociology and intersubjectivity toward practical everyday problems was in turn the motivation and inspiration for Garfinkel’s development of ethnomethodology.

Garfinkel’s ethnomethodology, notably as presented in Garfinkel (1967), perpetuated Schütz’ reorientation of sociological problems and thus turned against Weber’s stance toward principles of underlying social order and Durkheim’s (1858-1917) ‘objective reality of social facts’. In place of the contemporary trend for theorising, Garfinkel advocated a descent from the researcher’s ivory tower in order to empirically investigate social processes as basis of any theorising. Therefore, Garfinkel focused on how social order and reality was achieved through the mundane activities of people and consequently did not believe in any theory, claiming to be able to explain social processes, which in some way considered itself of higher precedence than the social processes themselves. This led Garfinkel to take lay theorising of social action very seriously and not attribute salience to more ‘scientific’ reasoning. Ethnomethodology thus focussed on these commonplace understandings of social action, how you and I make sense of the social world, as opposed to attempting to deduce a more authoritative account of ‘how society really works’. As such, ethnomethodology derives from phenomenological sociology.

One of the commonplace methods for achieving a sense of orderliness and meaningfulness of action is ‘accountability’. ‘Accountable’ means that a community establishes meaningfulness or a sense of what is considered as rational action through acting rationally/meaningfully, a sort of circular definition. Meaningfulness is established through perceiving others act meaningfully and one communicates to others, and thus establishes that one is a part of this community, through acting meaningfully. Accountability in this immediate sense means observable-and-reportable, “Liable to be called on to render an account” (Webster’s Unabridged Dictionary), or “Responsible for giving an account (as of one’s acts)” (Eriksén 2002: p. 178). In Garfinkel’s own words:

”Ethnometodological studies analyze everyday activities as members’ methods for making those same activities visibly-rational-and-reportable-for-all-practical-purposes, i.e. ’accountable’ as organizations of commonplace everyday activities.” Garfinkel (1967) pp. vii

In consequence, it is observable-and-reportable behaviour which can be perceived as meaningful, and in so being, helps reconstitute the commonly held conception of meaningfulness, the social orderliness. This is the reciprocality of understanding in ethnomethodology:

“Central to these accounting practices and accounts is their reflexive character, i.e. that in the constantly ongoing accomplishment of them we embody our lives and constitute ourselves as subjects” (Eriksén 2002: p. 179)

“[…] what it means to be a member of a language community (or, perhaps, an “action-community”) is to share a set of understandings of how to act, and how to understand action, within that community. In other words, “acting rationally” and “perceiving action to be rational” are reciprocal aspects of the same set of understandings”. (Dourish 2001: p. 78)

“Garfinkel’s argument is [thus] not simply that action can be found to be rational by those who understand it, but rather that the methods of understanding and making sense of action and the methods for engaging in it are the same methods […]” (Dourish 2001: p. 79, original emphasis)

Thus, members of a community provide others with an understanding of their acts and consequently also how to respond using the same rationality (by ‘setting an action-example’). Again, meaning is not static, nor somehow ‘underlying’, but woven into action and a background, a lifeworld, of unreflected meaning.

Applicability in relation to object orientation

Accountability in the above sense is part of our everyday lives and can of course also be observed in software development, as in any human praxis. One example, of a conscious use of it are visbility-oriented development practices (e.g. McConnell 1996, p. 10), where one makes the activities in a development processes visible-and-reportable in order for developers to be able to better coordinate and for the process to become more transparent vis-à-vis project managers and clients. This is a micro-sociological way of keeping meaningfulness in check by being able to observe the ‘transparent decision-machinery’ of others.

To my mind, accountability also provides part of the key to answering how to conceive the Model System as suggestive; of how to conceive it as a social actor put into the world of the user’s organisation by the system developer to establish meaningfulness, i.e. become the intended (future) meaning. A suggestive Model System may be conceived as an ‘accountable’ Model System, one that makes its causalities or ‘rules of meaningfulness’ visible-and-reportable to the user with whom it is supposed to establish common meaningfulness. This is an approach of viewing accountability as an interactional property of systems (the Model System to be more specific) – viewing it in terms of the form and content of feedback information (Dourish 1995). In this view accountability is “the story the system tells about itself” #13 (Dourish 1995: p. 148), i.e. that the system makes visible-and-reportable the inner workings or causalities of the system, thus "transparent". This type of accountability is also known as semantic feedback (e.g. Hudson and King 1988) or computational reflection. And i n the less technically oriented part of the HCI literature it is known as transparency in the sense of being able to see ‘what is going on’, as opposed to the conventional sense of that word where ‘transparent’ means being “able to apply intellect directly to the task; the tool itself seems to disappear” (Rutkowski 1982 as cited in Shneiderman 1997: p202 ). It is important to note that whereas Dourish means accountability of the system, the whole computer system, I mean accountability of the Model System as the locus of meaning in the contemporary conception of object orientation.

Accountability in this sense means that the system, viewed as an actor, provide its users with an understanding of its acts and that way also provide the rationale for meaningful response (by ‘setting an action-example’). Keeping the phenomenological action-meaning complementarity in mind this model of system accountability should be behavioural rather than simply providing a structural snapshot of the system’s inner workings. A dialog box explaining what has happened does not satisfy this criteria. It is in fact the epitome of ‘aboutness’ and it is important to accentuate that Garfinkel does not conceive of accountability as a commentary on or meta-information about action, but both action and meaning. How to conceive of system accountability that is behavioural will be introduced in the next section.

Dourish recounts a useful distinction, namely that between accuracy and precision (Dourish 2001: p. 152). Accuracy is, in this context, a measure of how correctly the system accounts for its actions (what the user sees versus what ‘really goes on’), whereas precision is a measure of how detailed, or precise, the accounts are. Obviously, it can be a good idea to relax the level of detail to protect the user from information about all the minute workings of the system.

Accountability in a socio-technical network (CSCW)

The principle of accountability does not only apply to the situation of use with two actors, the system and the user, but also in a setting of CSCW, Computer-Supported Cooporative Work. Let me give an example:

There is an inherent momentum in the word ‘workflow’ that suggests that work should be understood in pre-determinable ‘flows’ or ‘action sequences’, epitomised by Taylorism or Scientific Management thinking. While guiding users along certain paths can be useful, it has long been known that designing an anticipation of a certain temporal sequence of actions into a system is problematic. Robinson (1993) describes this as a fundamental tension between prescribing use through design versus designing for unanticipated use. Positioning oneself on this spectrum is a trade-off between guidance, possibly restriction, and open-endedness of use. Where you position yourself is a matter of choice, but the literature seems to agree that the further away from prescription you position yourself, the more important it is to provide the basis of the activity. This can be done via accountability.

“[…] the work process as a whole emerges from this sequence of locally improvised actions. If our goal is to support this character of work, then a critical focus of design must be the provision of the information or resources which support and inform the decision-making process, rather than the formalisation and encoding of the process itself.” (Dourish 1995: pp. 147-148)

Thus, designing is about the provision of information, not in the ‘about sense’ of mainstream information processing cognitive psychology but the action-meaning reciprocal sense. This sort of information enables decisions on how to nimbly adapt to local contingencies makes users problem solvers rather than process followers. And this is what accountability is about.

Dourish (1995: p. 147) remarks that in ‘traditional’ interface design (work-at-the-interface with one user, one task, one interface) the process-based view of work seems to be predominant whereas an improvisation-based view, which Robinson (1993) represents, is predominant in the CSCW and social computing communities. The improvisation-based view of work…

“[…] focuses on work as the improvised management of local contingencies, and emphasises the way in which regularisations in the conduct of activity at the interface arise out of individual moment-to-moment action (rather than the other way around).” (Dourish 1995: p. 147)

Conceiving work as such is fundamentally different from the naïve cognitivist stance that the plan comes before the action, which Suchman (1987, 2001) so thoroughly criticised through empirical investigations and criticism of which sources like Winograd and Flores (1986) provided the theoretical basis (through phenomenology). In this view, work is unpredictable until realised through situated actions – not something emerging from some hidden, underlying principles nor plans.

Concludingly, it should be clear from the above that accountability, both as a general principle and as an interactional property, are highly suitable concepts vis-à-vis a network of socio-technical actors, i.e. a situation of Computer-Supported Cooperative Work. This is important as ‘meaning’ in the Referent System is realised in a socio-technical network. Thus, applying accountability in relation to object orientation should not only be conceived of as an interactional property of the Model System where it establishes meaningfulness with the user-in-front-of-the-interface (the suggestive, directly participating Model System) but something that reaches beyond the concrete human-computer encounter and into the sociological level of description.

Applicability of the familiar

The sceptical reader may wonder why we should apply sociological concepts to the design of computer systems, even very concretely to interfaces. A possible answer is that since we construct software from a “material without properties” (Löwgren and Stolterman 1998) we cannot rely on an understanding of some generic ‘laws of nature’ of the computer upon which to base understanding (as is the case with physical artifacts). Instead, we may capitalise on knowledge of domains from outside the system that are well-known to us. What I am suggesting is not the conventional use of analogy and metaphor. Instead, I mean exploiting or imitating mechanisms that are pervasive in the physical or social world, with which we are highly familiar, in order to fill the empty representational container of the computer with structure from the real world. The best attempts of creating understandable and usable systems have indeed copied such mechanisms from well-known domains: Direct manipulation and tangible computing capitalise on our familiarity with physical objects, and ideas in the CSCW and social computing communities that are applied to systems design (e.g. accountability) capitalise on our familiarity with the social world.

Gibson and Ecological Psychology

As mentioned, we need to conceptualise the Model System as suggestive, one that users encounter directly rather than abstractly. For this reason, I have already introduced Barsalou’s theory of Perceptual Symbol Systems (1999), which problematises the distinction between perception (something direct, e.g. seeing that thing right there) and cognition (something abstract, e.g. conceptual inference, transduction of semantics) and posit that an underlying neural system, a perceptual one, underlies both realms. However, for our present purpose of operationalising perceptual theories vis-à-vis a redefinition of object orientation, we need a more elaborate account of perception. James J. Gibson will provide us with the last piece we need in this regard.

Gibson is probably one of the most misunderstood psychologists. Especially in the HCI community where the context of psychological research, in which his ideas were put forward, is not usually known. Comically, Gibson was popularised in the HCI community by Donald Norman (1988, p. 9) who introduced the term ‘affordance’, which he regrettably had misunderstood, as well as the larger theoretical framework on which it was formed. In an attempt to remedy the situation Norman published an article in ACM SIGCHI’s journal, Interactions, trying to clarify the misunderstanding ( Norman 1999b) as well as an ‘excuse’ in his book Norman (1999a: pp. 123-124). Unfortunately, he still did not clarify the concepts, nor seem to understand them himself. To bring justice to Gibson’s work others have tried to eliminate the misunderstandings (e.g. Gaver 1991, McGrenere and Ho 2000, Bærentsen and Trettvik 2002).

Gibson was a perceptual psychologist but certainly not an orthodox one. Gibson’s theories of perception, more specifically his theory of Direct Perception, cannot be compared directly to orthodox theories like those of the aforementioned Norman (e.g. the Norman and Shallice SAS model, Norman and Shallice 1980/1986) #14 firstly because he did not share their underlying philosophy of science assumptions (e.g. subject-object dualism and time-space understanding) and secondly because he does not adhere to the central assumption of cognitive psychology that perception and cognition are separate systems. He also did not agree that sensory inputs are ‘processed’ by higher-order cognitive systems, to which the perceptual system is a slave #15. Concepts and knowledge, according to Gibson, are not something we ‘process’, build up, or extrapolate from sense experiences, a from-parts-to-wholes view; “Orthodox psychology asserts that we perceive […] objects insofar as we discriminate their properties or qualities” (Gibson 1979: p. 134). This position helps him avoid the homunculus problem as he rejected separate realms of matter and consciousness, the psychophysical dualism: Information ‘about something’ is processed.

Some themes of ecological psychology

In his (last) book, The Ecological Approach to Visual Perception, James J. Gibson (1979) put forward his theory on Direct Perception, from which I have selected a number of themes that are relevant for a redefinition of object orientation.

Subject-object reciprocity

One of the many reasons Gibson’s work is interesting is that he contested some very basic theoretical assumptions of his contemporaries. One such assumption was the subjective-objective distinction. Gibson, and what he coined as ‘Ecological Psychology’ recognised the co-evolution of animal and its environment and consequently has the reciprocity of animal-environment as its guiding principle (“ecology” means something to the effect of “the study of the relationships between organisms and their environment” according to most dictionaries #16). The concept of ‘meaning’, for example, could thus not be defined with reference to something in the head of a person or animal, but a relationship between subject and object; the result of an organism adapting to the behaviour-shaping constraints of its environment. Gibson coined the term Affordance to capture the idea of meaning as a property of both subject and object.

"The affordances of the environment are what it offers the animal, what it provides or furnishes, either for good or ill. The verb to afford is found in the dictionary, but the noun affordance is not. I have made it up. I mean by it something that refers to both the environment and the animal in a way that no existing term does. It implies the complementarity of the animal and the environment. (Gibson 1979: p. 127, original emphasis).

“[..] an affordance is neither an objective property nor a subjective property; or it is both if you like. An affordance cuts across the dichotomy of subjective-objective and helps us to understand its inadequacy.” (Gibson 1979: p. 129)

Affordances are thus what the animal is offered or provided by the environment, or rather the complementarity of the properties of the environment and those of the actor. For example, for a human of an adequate height and muscular strength a door with such-and-such a height, weight, etc may afford opening. For a smaller or weaker human, it may not. The door’s affordance of openability illustrates the complementarity of the handle, the weight of the door, etc., on one side, and the motor system of the individual actor wishing to open the door on the other.

And because Gibson took the consequence of subject-object reciprocity, his theory of visual perception is just as much a theory about the environment-to-be-perceived as the perceiver and the perceiver’s perceptual system. His description of the ‘meaningful environment’ (e.g. chapter three in Gibson 1979) is thus a methodological consequence comparable to orthodox psychologists writing about the exact properties of the materials used in an experiment in the method section.

Affordances, the absolute properties of both subject and object, are specified by the information in what Gibson calls the structured ambient optic array, the flux of visual stimuli that is continuously reflected off the environment (thus having obtained structure from it) and received by the eye. If there is adequate information in the ambient optic array to specify the affordance, it can be picked up directly by the visual system, the reason Gibson calls it Direct Perception (more on directness in the next section). Affordances thus exist independently of the actor perceiving or ‘interpreting’ them, e.g. “The secret door afforded opening and entering by Mr. Søgaard, but it was too well camouflaged in the panels of the old library for him to discover it”.

Because affordances cut across the subject-object dichotomy, their ‘qualities’ cannot be measured like abstract physical properties are measured in physics or traditional experimental psychology (Gibson 1979: p. 127). That, Gibson argues, is the wrong level of description, which is the second theme that I have chosen from his research.

Phenomenological versus Euclidian level of description

It is important to understand the reasons for Gibson’s emphasis on Directness and how it hangs together with which level of description we as scientists and laypeople understand the world. Similar to affordances, the directness in Gibson’s theory reflects a contesting of the fundamental scientific assumptions of Gibson’s contemporaries, more specifically the ontological ones.

To my mind, many theories of Gibson’s contemporaries, as well as present theories, embody a modernistic stance in that they seem to posit underlying order or some sense of a singular ontology. In fact, it is not a controversial statement that much of cognitive psychological research, which does not primarily make use of qualitative methods, focuses on essentialist or otherwise ‘underlying’ truth, i.e. answering what variance in the dependent variable, e.g. reaction time, “really means”. Complexity in the subjective realm (again) seems to be conceived as a mask for underlying simplicity, and complexity can consequently be ‘weeded out’ by removing ‘confounding’ variables such as the natural environment of test subjects or their self-reported (and hence unreliable) data. Validity in most cognitive psychological research is conceived as of statistical nature and words like data and analysis suggest the modernistic conception of “knowledge as a map of an objective reality, and validity as the correspondence of the map with the reality mapped” (Kvale 1995). The language of this paradigm is that of geometry, mathematics, and the like, and visual perception is thus explained in this language. This level of description is what I call a Euclidian level of description.

Just like Husserl was tired of the contemporary trend for scientification and conducting science on an abstract plane decoupled from applicability, Gibson was not only dissatisfied with the modernistic stance (although not calling it by this name) and found it to be the root of many scientific misinterpretations of visual perception. For example, he found his contemporaries’ ontological assumption of the world best described as an abstract intellectual space highly problematic. Gibson’s argument is that the environment is meaningful as-is and thus relative to the actor, not meaningful by mandate of some inherent and abstract properties of the ‘objective’. The meaning of a stone is not its dimensions described in mathematical or geometrical terms, but what it affords or provides the animal. This stance toward taking the ecological level of description as point of departure lead researchers to ask different research questions altogether and consequently arrive at different results.

One such example is research into how we estimate time to contact of an approaching object. Taking a Euclidian level of description as point of departure we would have to estimate distance to the object in geometrical space, estimate our own speed, the speed of the object, and combine it into an estimate, which is a relatively complex task. It is coincidently also how an information processing device like a computer would do it. However, taking an ecological level of description as point of departure one would not view this ‘calculation’ with reference to an underlying intellectual universe but more directly. In fact, Lee (1976) did exactly that as he simply took the rate of expansion of the object’s retinal image as starting point, which he called Tau. Tau thus became either Tau = divide distance by speed, or Tau = 1/(rate of expansion of object’s retinal image), the latter of which is a much simpler conception resting on far fewer assumptions of visual perception, like information processing research into depth cues (e.g. Ittelson 1951), to establish distance to object. As Eysenck and Keane (2000) note, “[Lee’s] theory is in general agreement with Gibson’s approach, because it is assumed that information about time to contact is directly available.” (p. 84). Direct perception is thus direct in the sense that meaning is not ‘extrapolated’ from visual stimuli through some computation or information process with reference to an abstract and omni-valid level of description but directly ‘there’. The information, in this case time to contact, is directly there in the optic flow patterns (registered on the retinal image). Hence, direct information pickup #17.

Perceiving is knowing (perception-cognition unity)

But how do we learn to identify or ‘know about’ these high-level invariants like Tau in the first place? Learning, to Gibson, takes place through perceptual attuning or what he calls resonance. Much like a radio; the signal is there the whole time but the radio is not necessarily ‘tuned in’ to a particular station. Similarly, the optical information specifying affordances can be there all the time but the actor needs to attune his visual system to the invariant, learn to discriminate that particular invariant pattern in the optic array. It is “Improvement of perceiving with practice and the education of attention […]” (p. 254) and one can thus discover new affordances throughout one’s entire lifetime.

As a consequence, Gibson rejects the distinction between knowing and perceiving, which makes his theory a “perceptual theory of knowledge” in line with Barsalou (1999). ‘Knowing’ that certain berries are poisonous means attuning the perceptual system to the affordance of the malevolence of the berries, so to speak. ‘Knowing something’ is a learned ability to perceptually identify a certain invariance and hence…

“The theory of information pickup […] closes the supposed gap between perception and knowledge. The extracting and abstracting of invariants are what happens in both perceiving and knowing. To perceive the environment and to conceive it are different in degree but not in kind.“ (Gibson 1979 p. 258)

The unity of perception-cognition cuts across the classic form-content discussion as conceiving follows from Directly Perceiving (i.e. Direct Perception). In other words, in Gibson’s theory there can be no form-content distinction since Direct Perception posits a unity of orthodox perception and orthodox cognition #18. If Gibson’s claim were “orthodox perception is orthodox cognition” and similarly “orthodox perceiving is orthodox knowing” he would essentially engage in a discussion of form versus content. However, although the concepts ‘Direct Perception’ and ‘orthodox perception’ share the word ‘perception’ they are vastly different and distinguishing the two is crucial. Only in an information processing account of human thinking, with its stance toward meaning as aboutness, can one distinguish between form and content.

A common misconception with regard to the perception-cognition unity is that Gibson supposedly denied the existence of long term memory, which he in fact did not. But he objected to how memory was, and still is, conceived, as a separate system, holder of meaning ‘distilled’ from (visual) stimuli. To Gibson, having been perceptually attuned to an invariant is akin to what an orthodox psychologist would call ‘store an invariant in long term memory’, but Gibson did not see the need for this representationalist stance toward somehow storing the meaning of berries. According to Gibson, memory and knowledge are first and foremost perceptual and when asking a person about poisonous berries she ‘revives’ the perceptual states of her perceptual system to reason about berries, she does not retrieve a chunk of ‘essentialist meaning of berries’.

“[…] a perceptual system that has become sensitised to certain invariants and can extract them from the stimulus flux can also operate without the constraints of the stimulus flux. Information becomes further detached from stimulation. The adjustment loops for looking around, looking at, scanning, and focusing are then inoperative. The visual system visualizes. But this is still an activity of the system, not an appearance in the theatre of consciousness.” (Gibson 1979: p. 256)

And further:

“Evidently the theory of information pickup does not need memory [in the conventional sense]. It does not have to have as a basic postulate the effect of past experience on present experience by way of memory. It needs to explain learning, that is, the improvement of perceiving with practice and the education of attention, but not by an appeal to the catch-all of past experience or to the muddle of memory.” (p. 254)

This position was very visionary of his time, given that mainstream cognitive psychology at the time completely rejected any such notion of a unity of perception and cognition as an answer to the homunculus problem of ‘aboutness of information’ and that mainstream cognitive psychology only within the last decade, to my mind, have begun to take a positive interest in Gibson’s views.

As the reader has probably guessed Gibson’s views are similar to the phenomenologists’ point that we cannot abstract the process of perceiving something from the meaning of the perceived, only that Gibson is more elaborate in his account of the perceptual processes. The other, and related, point the phenomenlogists provided us with was the reciprocity of action and meaning. Once more, this theme is refound in Gibson’s work and again his work is more elaborative with regards to the perceptual system as well as the-environment-to-be-perceived, which allows us to more directly apply his findings to our present purpose.

Perception-for-action versus perception-as-recognition

Orthodox psychology mostly uses factorised, laboratory-based experiments of visual perception where subjects usually only report what they see, not take any action in any sensory-motor sense (unless one is measuring reaction time of recognition where ‘action’ would be conceived as pressing a button, not doing anything related to the subject matter of what he/she is perceptually exposed to). Confounding variables such as movement of the head is sought eliminated by headrests, and to any other visual stimuli is removed from the contextually sterile laboratory. The subject is usually only exposed to a 2D-picture and asked to report on this as the dependent variable. By 2D I mean to distinguish it from both simulated 3D, as in a computer or a 2D picture with depth, as opposed to real 3D which necessitate moving head and body. Being exposed to 2D pictures in a laboratory does not require movement and this kind of perception thus involves a minimum of proprioception, awareness of one self and about the position of our body parts.

This stance of perceptual research is by some (e.g. Eysenck and Keane 2000: p. 62) said to treat perception as perception-as-recognition. The theories I mentioned earlier – e.g. feature theories of perception such as Selfridge’s Feature Demon Model (Selfridge 1959), constructive perception theories (Helmholtz 1909/1962), and structural-description theory such as Biederman’s Recognition-By-Components (RBC) theory (Biederman 1987) – are all based on this paradigm of recognition. For example, in a structural-description theory like Biederman’s, one recognises objects by observing edges of objects and then decomposing the objects into ‘geons’, low-level or basic geometrical shapes. Perception thus becomes recognising that something matches, in line with the amodal information processing view that perception is input device, it provides the information for the matching process after which consciousness can operate on the meaning of what is recognised.

Gibson, on the other hand, sees perception as action-oriented, a visual guidance of action, emphasised by translating affordances as ‘action possibilities’ (e.g. by Gaver 1991). This conclusion is reached by considering in which circumstances the perceptual system has evolved and is usually used: In everyday purposive, life-supporting activity during locomotion and limb movement, not on a headrest in a laboratory. Such a view of perception is one of action or (physically) acting in an environment and the human or animal thus uses its perceptual system to support this kind of activity. Meaning, conceived as perception-as-process in the above, is therefore intricately interwoven with action, not recognised or deduced. Gibson sums up this point:

“[…] psychologists assume that objects are composed of their qualities. But I now suggest that what we perceive when we look at objects are their affordances, not their qualities. We can discriminate the dimensions of difference if required to do so in an experiment, but what the object affords us is what we normally pay attention to. The special combination of qualities into which an object can be analyzed is ordinarily not noticed.

[…] Phenomenal objects are not built up of qualities; it is the other way around. The affordance of an object is what the infant begins by noticing. The meaning is observed before the substance and surface, the color and form, are seen as such. An affordance is an invariant combination of variables, and one might guess that it is easier to perceive such an invariant unit that is to perceive all the variables separately. It is never necessary to distinguish all the features of an object and, in fact, it would be impossible to do so. Perception is economical.” (Gibson 1979, pp. 134-135, original emphasis).

There is a great deal of further evidence for such an enactive and action-oriented view of perception but I shall limit myself to mention Varela et al.’s book (1991), whose approach is supportive of the general stance of my framework for object orientation but not entirely compatible with Gibson views (e.g. Varela et al. 1991: pp. 203-204). As part of their argumentation they cite a classic study by Held and Hein who raised two groups of kittens in the dark and only exposed them to light under controlled conditions:

“A first group of animals was allowed to move around normally, but each of them was harnessed to a simple carriage and basket that contained a member of the second group of animals. The two groups therefore shared the same visual experience, but the second group was entirely passive. When the animals were released after a few weeks of this treatment, the first group of kittens behaved normally, but those who had been carried around behaved as if they were blind: they bumped into objects and fell over edges. This beautiful study supports the enactive view that objects are not seen by the visual extraction of features but rather by the visual guidance of action.” (Varela et al. 1991: pp. 174-175)

To sum up, from the previous section we know that perceiving is knowing, and from this section we may now conclude that knowing is knowing-to-act concretely in space-time, as opposed to ‘knowing abstractly and decontextualised from action’.

A value-laden theory

The above rejection of perception-as-recognition, where qualities of perceived objects are recognised and transduced into meaning, is a consequence of Gibson’s theory of meaning-as-affordances.

“The theory of affordances is a radical departure from existing theories of value and meaning. It begins with a new definition of what value and meaning are. The perceiving of an affordance is not a process of perceiving a value-free physical object to which meaning is somewhat added in a way no one has been able to agree upon; it is a process of perceiving a value-rich ecological object.” (Gibson 1979: p. 140).

Conceiving of subject-object neutral affordances as inherently value-laden, enables Gibson’s theory to account for, or at least speak of, affections, motivations, feelings, etc, which perceptual theories practically never do. Such theories generally posit that such things result from higher-order cognitive ‘processing’. As Heft (2004) sums up:

“Gibson’s appreciation for the place of feelings in perceiving is evident in his concept of affordances. […] With its emphasis on value-laden features of the environment, the concept of affordances implicitly carries with it recognition of the affective dimension of perceiving from the standpoint of the individual, and in doing so acknowledges the central place of feelings in acts of perceiving.“ (Heft 2004, p. 7)

Ecological validity and ‘longitudinal’ time

Another major contribution of Gibson was his focus on what he coined ecological validity which has been touched upon in the above; that generalisability of research results should not be conceived of as generalisability across laboratory contexts but vis-à-vis the subject’s ecology/everyday context, a consequence of subject-object reciprocity (for a discussion of external and ecological validity see Mook 1983 and 1989). Gibson himself had conducted much of his research in the very ecologically valid context of World War II aeroplane cockpits, and evidently did not value rigor over relevance.

As mentioned in the above, orthodox psychologists sought to isolate factors by fixating the subjects’ head and providing only momentary stimulus presentation in perceptual experiments. Gibson, however, took a different stance. His point of departure was the ecological context of man and this made him insist that visual awareness is panoramic and that visual orientation takes place during locomotion with movements of the eyes and head. Human vision is not developed for the kind of ‘snapshot vision’ that is relied on in the laboratory, he posited, but visual awareness over time during natural locomotion and limb movement. This meant rejecting another assumption, namely that of time as conceived in the Western world and scientific tradition. Here, it seems that time is normally conceived as (finite) slices of time jointly constituting a timeline, e.g. T1, T2…T10, where variance consequently can be defined as the difference between T x and T y. This is done in classic perceptual experiments where the independent variable usually is successive stimulus presentations with a difference and the dependent variable (e.g. reaction time to correct answer) is the subject’s ability to detect variance/invariance across stimulus presentations. Gibson rejected that the human perceptual systems is built to perform this kind of ‘calculation of difference’ but that it picks up a continuous stream, a flux, of sensory stimulation. A flux is a flow that consequently cannot be fixated in time and invariance can thus be conceived as ‘invariance in a flow of variance’.

“The stimulus-sequence theory of perception [i.e. classical perceptual experiments mentioned above], based on a succession of discrete eye fixations, can assume only that the way to apprehend persistence is by an act of comparison and judgment. The perception of what-it-is-now is compared with the memory of what-it-was-then, and they are judged same. The continuous pickup theory of perception can assume that the apprehension of persistence is a simple act of invariance detection. Similarly, the snapshot theory [termed ‘stimulus-sequence theory of perception’ above] must assume that the way to apprehend change is to compare what-it-is-now with what-it-was-then and judge different, whereas the pickup theory can assume an awareness of transformation. The congruence of the [optic] array with itself or the disparity of the array with itself, as the case may be, is picked up.” (Gibson 1979: p. 247, original emphasis)

In lack of a better word, I have chosen to call Gibson’s conception of time ‘longitudinal’, and as a consequence of this conception his view of invariance detection may be called “continual-awareness-of-variance-and-invariance-in-ambient-optic-array-over-time” as opposed to the mainstream conception which he himself designates “snapshot theory”. Learning then, becomes a question of learning to distinguish invariants perceptually, a trained awareness.

This stance changes a whole array of basic research questions in perception. For example, I found the following quote in Eysenck and Keane 2000, p. 30: “One of the major accomplishments of visual perception is the way in which the two-dimensional retinal image is transformed into perception of a three-dimensional world.” Clearly, the authors assume that perception is about taking successive snapshots in 2D and constructing a 3D picture in mind herefrom. However, if one’s assumptions about time are changed the very point of departure of research into visual perception consequently changes, the ‘accomplishments’ mentioned by Eysenck and Keane disappear, and a whole array of research question must be posed differently altogether.

In addition to the above, ecological validity also yielded an embodied view of human perception: “We are told that vision depends on the eye, which is connected to the brain. I shall suggest that natural vision depends on the eyes in the head on a body supported by the ground, the brain being only the central organ of a complete visual system” (Gibson 1979: p. 1) Perception when viewed outside artificial laboratory contexts cannot, according to Gibson, be abstracted from its bodily basis as the orthodox assumption of psychophysical dualism would have it. This makes him very similar to the phenomenologist Merleau-Ponty although Gibson, according to his daughter, first became acquainted with Merleau-Ponty’s work after developing his Direct Perception theory.

Applicability in relation to object orientation

The applicability of Gibson’s and related research has already been touched on to some extent. In the following I shall elaborate on the applicability vis-à-vis our present purposes.

A change of fundamental assumptions

The first thing we may learn from Gibson is that he changed some very basic assumptions (subject-object dualism, psychophysical dualism, time as slices, perceptual space as 2D scenes), which made him ask entirely different research questions than his contemporaries. His assumptions also enabled him to answer questions that were overly complex when explained with the assumptions of orthodox psychology. A similar change of fundamental assumptions, I believe, will bring about the same change of research questions in object orientation as well as enable the answering or elimination of questions that with the present understanding of object orientation seem daunting at best, impossible at worst.

I believe that to the extent that programming, and object orientation in particular, is about understanding, the research which people like Barsalou and Gibson have provided and which now is gaining popularity, partly due to the advances of Cognitive Neuroscience and brain scanning techniques, can provide parts of the epistemological assumptions of object orientation. Using such a reorientation from viewing understanding/cognition ecologically or perceptually as opposed to mechanically (information processing) to inform object orientation, would distinguish object orientation from the ‘harder’ computer science communities. I find it paradoxical that object orientation shares its fundamental epistemological and ontological outlook with communities that view themselves as applied mathematics, resting firmly on natural science assumptions, namely computer science communities concerned with such things as algorithms and data structures. The research in cognitive psychology and related ‘epistemological’ disciplines that is supportive of Madsen et al.’s conceptual framework, to my mind, long proven its inadequacies and I believe it is time for object orientation to take consequence hereof.

Subject-object reciprocity and ecological validity

Gibson’s guiding principle of subject-object reciprocity vis-à-vis perception renders many assumptions of object orientation problematic. First of all, it becomes highly problematic to think of the Model System as residing in the realm of the objective, formalised in a neutral level of description, and subsequently used and ‘interpreted’ subjectively by the user, the subject, by way of the user interface as information-transmitter-and-mediator. As noted in an earlier section, Madsen et al. attempt to conceive of objects-in-the-world as somehow more ‘elementary’, and hence unproblematic, building blocks of matter and in this regard subscribe clearly to a subject-object dualistic approach to the world as something ‘out there’, constituting the safe, un-subjective reality. In an ecological theory there are no decontextualised objective and somehow more unproblematic building blocks. Just as the predator is complimentary to its pray, i.e. that the antelope’s sensory-motor system is developed complimentary to the lion’s due to their side-by-side evolution, so is the Model System reciprocal to the Referent System, and ‘speaks back’ at it. A well-realised Model System is one that in its epistemological and ontological assumptions anticipates this ‘speaking back’ and concretely does speak back to the user. In other words, the Model System cannot not describe the Referent System as a decontextualised truth, but participate in the Referent System and attempt to impose its intended meaning on the Referent System, i.e. what I have called suggestive modelling.

Subject-object reciprocity also reminds us that context is what renders things meaningful, as well as delimits the meaningfulness. Context demarcates the boundaries of the language game, and hence meaning. Context is not a muddle of confounding variables that should be ‘weeded out’ to reveal the essence, as conceived in many natural sciences. Instead, context is that which makes or enables any talk of essence, as the schism between the early and late Wittgenstein illustrates. This sensitivity to the ‘richness of the situation’, is well-known to especially the Nordic HCI community (c.f. Jensen and Søgaard 2004), e.g. ‘contextualist’ theories such as activity theory (from the writings of Vygotsky, Leont’ev, and Marx and its import into HCI and computing sciences, e.g. Bødker 1991).

Phenomenal versus euclidan level of description

Just as we understand time to contact of an approaching object (Tau) directly (in Gibson’s terms) and without reference to an abstract, intellectual, ‘authoritative’ universe (the ‘objective’ from above), users understand interfaces as-is and without reference to what really goes on or what it ‘means’ (aboutness). Meaning does not reside in a disenchanted world of Newtonian facts exempted from purpose and value (i.e. the Model System), but in the lifeworld and experiences of the user. Consequently, if a developer wants her code to coordinate with the user’s understanding (to keep it in check so as to minimise the representational gap), she has to take this phenomenal level of description as point of departure as did Lee (1976) when realising that information was ‘right there’ on the retinal image. Over should be the days where a model can be thought of as ‘provable’ by mandate of some Euclidian or otherwise objective level of description, which relational database modelling or algorithmic information theory (as derived from Shannon and Weaver’s information theory and Turing's computability theory) incline us to think.

Construing the Model from the meaning present in the users’ phenomenal world as opposed to the decontextualised objective is in my view the way to minimise the representational gap and thus minimise the ‘translation activities’ between Model and Referent System. This would also do away with programmerisms (or programmer-isms), the inappropriate use of technical terminology in interfaces, which can be viewed as the model ‘surfacing’ in the GUI, emerging from the depths of the, to the user, mysterious software intestines.

Perceiving is knowing

Gibson proposes a perceptual theory of knowledge that is different from orthodox modal theories, reminiscent of early Artificial Intelligence research, Representationalism, the Textual Turn, etc. If our thinking is first and foremost perceptual, meaning is not somehow ‘conveyed’ to the user by the GUI. We have to stop talking of the essence of thought as “extrapolated semantic chunks”. There can be no ‘essence’ abstractable from the very process of perceptually-based thought. As long as we think of interaction in terms of what is being done and not so much how it is being done, we will continue to see the human-computer encounter in the modernistic and information-processing way: the software model (decontextualised semantic chunks a.k.a. classes and objects) is communicated through the GUI that provides the visual means from which the user ‘reconstructs’ the model (via his information-processing apparatus) as semantic chunks in consciousness in some ‘format’ (e.g. Aristotelian, Prototypical or fuzzy concepts as Madsen et al. suggest). To my mind, non-perceptual theories of knowledge, the what-is-really-being-done-view, entail excessive aboutness, the homunculus problem, and other classical problems of the information-processing paradigm.

Instead, a perceptual, action-oriented, subject-object reciprocal, ecological theory of knowledge would hold that the process of use of the GUI in extended (longitudinal) time-space is the locus of meaning; the “how it is being done”. This is the reason the relationship between the software model (the locus of meaning for the programmer) and the use of the GUI in concernful activity (the locus of meaning for the user) is of critical importance. Whether the user drags A into B as opposed to A onto B will evidently influence her conception of the causal relationship between A and B, and this should be part of the domain of investigation/coordination of the Model System.

The question of duality of perception and cognition also elucidates mental model research in relation to GUI-Model dependency. The proposition that people rely on mental models was first put forward by the Scottish psychologist Kenneth Craik in his book The Nature of Exploration (1943) in which he wrote that the mind constructs "small-scale models" of reality that it uses to reason, to anticipate events and to underlie explanation. I do not necessarily disagree with Craik to the extent that one’s mind’s eye ‘visualises’ which is natural when positing that neural systems common to imagery and perception underlie functions that are normally, in amodal approaches, conceived as higher-order cognitive functions (c.f. the section “The modal/amodal distinction"). I do however disagree with how Craik’s notion of mental models has been imported into the HCI community. For example, in “Some Observations on Mental Models" Norman (1983) notes: "Mental models are what people really have in their heads and what guides their use of things." This clearly seems to be a view of meaning as a distillate, of extrapolated semantic chunks from which to reason, which is not surprising as Norman seems to be an epitome of the information processing approach and cognitive science (c.f. e.g. Lindsay and Norman 1977, Norman 1980, 1983, Norman and Shallice 1980/1986). For a review of mental model research in the HCI community, see Payne (2000).

What the research into mental models showed, among other things, was that humans have a compulsion toward making a mental model of the 'internal logic' of the computer system. This was or is to my mind not a ‘natural’ compulsion but rather a symptom of inadequacy of ‘causality conveyance’. Having to construct some kind of lay theory of the hidden or otherwise non-apparent causalities of the system means that visualisation has failed. In fact, ‘visualisation’ in this regard seems to have problematic connotations as it implies a visualisation of something, namely meaning, that is in some other form than visual, being conveyed by visual means.

Thus, I hardly find it surprising that people form opinions on what-goes-on-behind as decoupling the GUI from the Model entails a paradigm of what-you-see-is-NOT-what-it-is. Not recognising mental models as a symptom as opposed to a natural compulsion may also be part of the reason why many efforts in the HCI community in the early 80s were wasted since it appeared that "The aim of interface design was to invent metaphors that users could understand, but which had not much to do with the internal workings of the system" (Andersen 2003, p.4). And why deny the user access to what the system really is, and instead provide are mask of mere visualisation, a decoupled GUI layer? With perception-cognition unity, this system architecture seems problematic indeed.

Perception-for-action versus perception-as-recognition

Gibson’s stance of perception-for-action shows us that perception is (inter)action, not disembodied rationality. Perception in fact requires action and cannot be abstracted from the activity. Hence, it is not neutral the same way perception-as-recognition is. It is a stance toward embodiment of interaction as has been discussed in the HCI community a great deal recently (e.g. Dourish 2001). A model thus finds it meaning through inter(action), a good argument for moving the Model System to the place “where the action is” (alluding to Dourish 2001), namely to the GUI in some form of Model-View coupling or merger. Where the action is is ‘right there’ at the pressing of a button, at the report (on the screen), at the task (on the screen), at my handling of that thing right there (on the screen), not someplace deep in the belly of the software (the model) or in the back of one’s mind, in the mental model storehouse.

An ecological theory is a theory of values

The rejection of perception-as-recognition is a consequence of Gibson’s theory of meaning-as-affordances, as acts of perceiving, and hence knowing, are acts of perceiving value-rich ecological objects. Recently, the HCI community has turned substantial attention toward the affective dimension of technology use , e.g. the September/October edition (volume 11, issue 5) of the ACM Interactions magazine was called ‘Funology’. To my mind, adequately accounting for and explaining values and affect seems a Sisyphean task in those parts of the HCI community that have come to rely on theories and theoretical frameworks like information processing psychology. I believe such a theoretical footing can only provide a view of technology as “what it really is underneath the pretty icons and bright colours” and consequently prevents us from speaking of something as ‘strange and elusive’ as computer games.

At one end of the spectrum of attempts to account for affective dimensions in technology use, we find Norman’s latest book, Emotional Design (2004). Norman uses research results from Social Psychology and an emerging discipline named Social Cognition (for a review, see Kunda 1999) that uses the hard science approach of cognitive psychology to study the social world (!). At the other end, we find an import of theories from the aesthetic disciplines and attempts to re-orient HCI as an aesthetic discipline (Bertelsen and Pold 2002). By using Gibson, or another theoretical framework with a similar approach to values and meaning (that is comfortably clear of Norman’s end of the above spectrum), affection and emotions are inherent to the theory and do consequently not have to be treated as a ‘special case’.

Longitudinal time and space

Direct Perception involves detection of variance/invariance in the stimulus flux (longitudinal time and space, consequently) and not by way of snapshot-vision (time as slices, space as frozen scenes). However, most interfaces are construed as though humans perform the latter: The user performs a comparison between what-is-now-with-what-was-then to inform her of the state changes of the model, a sort of synchronisation between the software model and the mental model of the user. This is epitomised by dialogs, information ‘about’ what goes on, the aboutness being necessary as state changes in most interfaces are not visible through continual changes as they would be in the physical world, e.g. a dog that runs from over there to over here. The WIMP #19 -interface and most (object-oriented) GUI toolkits and libraries (e.g. Sun Microsystems 1999, 2004 ) are designed for this kind of interaction. Consider the following quote.

As with objects, if we compare navigation in the functional spaces of programmed devices with the way we navigate in the real world, the difference is one of substantiality and continuity. Instantaneous jumps from one location to another are simply impossible in the real world, and when we move from one place to another, the scenery changes continuously and in a way that automatically and unambiguously informs us about our whereabouts during the transition. We rarely need a sign giving "feedback" information about our position. (Bærentsen 2000: p. 41)

Bærentsen draws our attention to the obvious fact that the difference, of what may called ‘object identity’, between the real world and most computer systems is one of substantiality and continuity. As Madsen et al. would have it and how most GUIs are conceived, substance (i.e. substantiality in the aforementioned quote) is ‘underlying’ (i.e. the antonym of that which is apparent, c.f. section “Cognitive and quasi-epistemological assumptions of object orientation") and hence ‘informed about’. If our conception of substance or the object of our thought processes, i.e. meaning, is rethought along the lines of Gibson’s theory, continuity (e.g. the stimulus flux of the dog ‘moving’ toward me, in longitudinal time) suddenly becomes important as it is meaning in itself, “the invariance/variance in the stimulus flux” as opposed to the extrapolated meaning “the dog is moving toward me”. Again, meaning is something we (should) encounter directly, not abstractly.

A longitudinal concept of time is comprehensible in relation to the GUI where we are dealing with something entirely visual. Watching oneself slowly bending a finger can help us understand that meaning is variance/invariance over time. However, it becomes difficult to comprehend the consequences of an alteration of time-conception at the code level. How do we coordinate our code with this sort of meaning as activity in extended time-space? In other words, how do we model in the light of an updated view of time? How do we conceive of code execution in longitudinal time? These questions may seem somewhat obscure, or at least intellectually overwhelming, but only because we in the Western World are used to viewing time as slices, and variance as variance between two points in sequential and sliced time. This makes us think that meaning, at the code level, is the return value of an algorithm/function/method/etc, the algorithm only being the means to get to the result. Execution time of an algorithm (conceived as ‘slices’ or individual sub-statements sequentially processed in processor clock cycles) is not something we are interested in per se, as long as it is really short, i.e. time-efficient. How, then, do we conceive of an algorithm in longitudinal time? I do not claim to have a ready answer for this question but it serves to illustrate how engrained our underlying philosophy of science assumptions are. It simply becomes impossible to think of code execution or information processing along different lines than our defaults.

Toward a redefinition of object orientation

In the above I have provided a theoretical framework for object orientation that has overcome the problems inherent in the research supportive of Madsen et al.’s conceptual framework of class-based object orientation (1993) as well as Knudsen and Thomsen’s framework of language independent concepts ( 1985). In the following, I shall provide an update of the often-cited figure from both Knudsen and Thomsen (1985) and Madsen et al. (1993) on what modelling is.

Reconceiving the act of modelling

My update of how to conceive of modelling, is, as has probably been evident in the above, a marriage between a kind of Direct Manipulation and object-oriented classifications (in an updated form), producing a Model-View Dependence. Here are Madsen et al.’s figure and mine.

Figure 16: The act of modelling as described in Madsen et al. 1993 p. 286.

Figure 17 (click for fullsize view of image): The act of modelling as crystallised from the theoretical framework laid out in the above.

To underscore the social and constructionist nature of the Referent System I designate it the Realm of Realised meaning as it describes a future world with the resulting computer system installed in it. The Model System, on the other hand, is now conceived as the Realm of Intended Meaning to underscore its subjective and intentional nature. Suggestive modelling is necessary for the intended meaning to become realised as intended, i.e. for the future world to become as the designer/developer had anticipated. Suggestive modelling underscores the fact that the main direction of Madsen et al.’s modelling was from the Referent System to the Model System; a semi-descriptive or correspondence-like approach. In my version, the main direction is from the Realm of Intended Meaning to the Realm of Realised Meaning, as Intended Meaning must be suggestive as to the meaning-formation process, trying to ensure that it is realised as intended (a prescriptive or constraint-imposing approach). The way this is done is through direct participation in the Realm of Realised Meaning. Suggestive thus means something that is suggestive of, or constraining vis-à-vis, the meaning-construction process of the socio-technical network in a principally concrete way.

In the Realm of Intended meaning I have substituted the “class --> abstraction --> object” distinction with “Descriptor --> generation --> Object”. I have been inspired by the Descriptor-Entity distinction from Knudsen and Thomsen (1985) as it seems less verbally entangled in Aristotelian and related approaches to understanding. Descriptors are mostly relevant in pre-compile time when the developer is constructing them (and of course also in runtime when Objects are generated from them) and Objects are mainly relevant in runtime when most of them are directly encountered by the user. I envision Descriptors to be, at the lexical code level, akin to classes in class-based object orientation with the difference that they must also supply their own graphical representation, as follows from the fact that most Objects are directly encountered by the user, not being ‘underlying truth’. Descriptors are therefore mainly text-based although some portion of the work put into their construction can be done in GUI builders. Objects, on the other hand are usually very visual and concrete. Some, of course, remain hidden behind the scenes as it is not all inner aspects of the system that the user can/should be exposed to, in line with the precision/accuracy distinction from Dourish (2001). Here, I am thinking of what is commonly referred to as the ‘technical services layer’.

The Objects of the Realm of Intended Meaning are represented inside the Realm of Realised Meaning as they directly participate, being suggestive of its interpretation through accountability, in the socio-technical network that realises meaning. Meaning in this context is meaning-in-its-consequence or what is commonly agreed upon and continually re-negotiated. Therefore, meaning is not something defined at a Euclidian or an otherwise ‘neutral’ level of description, but to be described at the phenomenological level. A phenomenon, used to coordinate one’s code with, cannot be defined as “a thing that has definite, individual existence in reality or in the mind” (Madsen et al. 1993: p. 290), but a phenomenological phenomenon, its meaning being a relation between the individual and her environment. As such, a phenomenon, or rather, its meaning, is both particular to the individual as well as demarcated by the sociological level of description as a consequence of subject-object reciprocity. This is in line with the phenomenologists’ stance of the unreflected lifeworld (partly individual, partly sociological) upon which all other understanding is formed and the late-Wittgensteinian approach of context-demarcated meaning (micro- and macro-sociological language games) and ethnomethodology’s stance of making societal order emanate from the everyday ordering processes of the individual, not underlying principles of inherent human behaviour or a notion of the real and neutral world.

At the interface-level, i.e. work at the very concrete interface with one or more users, I have used Gibson to provide the theoretical framework, and additionally, because of his elaborate accounts of ‘the meaningful environment’ in relation to the perceiver (e.g. chapter three in Gibson 1979), I propose to use him as a design stance in relation to computer systems, mainly the visible parts of them anyhow. This is a stance toward taking the consequence of the perception-cognition unity and designing systems with substantiality and continuity where applicable. This results in a Direct Manipulation-like approach which, however, does not necessitate a new research agenda for Direct Manipulation as I believe the empirical findings of Direct Manipulation research, not the theoretical accounts of it (e.g. Hutchins et al. 1986), agree with a Gibsonean approach.

What I, in an object-oriented sense, mean by “the Realm of Intended Meaning should participate in the Realm of Realised Meaning” is that objects in the domain layer should not only be defined in terms of state and behaviour in an abstract sense but just as importantly in a concrete, visual and sensory-motor sense. I am not entertaining the idea of making all its properties of objects visible but making each object responsible for supplying its own View (if applicable, as one does not necessarily need a View). This is the accountable aspect of an individual object, through the direct and active interaction with which (actively using the GUI-object) or through passively #20 observing the accountable behaviour of which, meaningfulness is established. By View I thus mean an object’s visual characteristics in both a static and behavioural way, according to the longitudinal view of time, space, and perception. By View I also mean an object’s interactional possibilities (provided to the user in the GUI) in line with the perception-for-action approach where action and meaning are reciprocal and where the user consequently has to do in order to know. Hence, if an object is dragable or somehow transformable (either by continuous manipulation by the user or automatically), it should define its static visual presentation (state) as well as its visual transformation (behaviour) itself. It should also define how it can be interacted with (the action language or the Controller component of MVC) as a given object’s meaning is contingent upon these aspects. To this end, the object can of course make use of regular GUI frameworks with pre-programmed functionality. Here is an example, first written in Java code and next in pseudo-Java code.

public class employee{ 
        int age; 
        String name; 
        
        public Name getName(){ 
                return this.age; 
        } 
        public void setName(name){ 
                this.name = name; 
        } 
} 

public descriptor employee{ 
********* regular stuff of a Model object ********* 
        int age; 
        String Name; 

        public Name getName(){ 
                return this.age; 
        } 

        public void setName(name){ 
                this.name = name; 
        } 
** new stuff usually located somewhere in a cluster of GUI classes ** 
        public GUIComponent getView(){ 
                 construct default view  
        }
        public GUIComponent get16By16Icon(){ 
                 return icon  
        }
        public GUIComponent get32By32Icon(){ 
                 return icon  
        } 
        public GUIComponent getTranformationFromHiredToFired(){ 
                . return visual transformation 
        } 
        this.addWindowListener(new WindowAdapter() { 
                public void windowClosing(WindowEvent e) { 
                         enable some interactional behaviour  
                }
        }); 
}

Locating the view (e.g. get’able via a get method) in the same file as the rest of the class is intended to encourage the programmer to think that something visual or something behavioural is a property of the thing itself, not merely a property of the presentation of the thing. As a consequence, it will hopefully also encourage the programmer to formalise along visual, behavioural, and interactional dimensions, making it natural to take the GUI into account at the early ‘analysis’ stage.

The above does not mean that business logic code is mixed with GUI code. I am maintaining a Model-View independency at the method level instead of at the object level. Inside an object the code for handling buttons etc is to be separated from the business logic code. I have however chosen to call my approach of method level Model View Independency for ‘Model View dependency’ to make it antagonistic with regards to ‘normal’ object level Model View Independency, the pervasive design principle.

What I intend to do with the above code example is to provide an example to ‘ground’ the sometimes abstract and theoretical discussions of this report and, most importantly, to get the reader’s imagination going. I encourage the reader to think of similar architectures or coding practices that derive from the theoretical framework outlined in this report. Evidently, a given architecture or coding practice like the above cannot be judged either good or bad but may be judged more or less useful in a given context. For example, some performance-dependent applications may choose to exclusively consider performance characteristics of hardware and of programming language and consequently model for performance, not for object design-world correspondence. Therefore, the code example above can never be more than an example that illustrates some points of mine, but from an understanding of the theoretical framework and a concrete example of an architecture or code like the above, the reader may be able appropriate ideas presented here to fit his/her programming context.

Software Practice Agency and the need for a ‘concrete’ theory

Providing the above example of a possible architecture has another motivation, namely to accentuate the importance of reifying or ‘thingifying’ a theoretical position, what may be called a ‘concrete’ theory. To embed a theoretical (re)definition of object orientation into a tool, an architecture, or a more ‘tangible’ idea will have the advantage of ‘sneaking theory in the back door’. Since the target group of a redefinition of object orientation, not necessarily this report, are computer scientists, one cannot expect them to find discussions of sociological, cognitive, or epistemological matters interesting. However, if one makes a tool/architecture/etc that has a certain inherent momentum to it, that momentum may move practice in the desired direction, instead of taking the opposite strategy of first convincing people of your opinions and subsequently trying to make them adjust their practice accordingly. This is the reason I am proposing this very concrete practice of locating the View inside the Descriptor.

As an example of embedding ideas or a notion of a best practice into some concrete software construct, i.e. ‘software agency’, consider Python’s semantic use of whitespace. In the Python programming language, an object-oriented interpreted scripting language, indentation is namely used as a programming construct, meaning that whitespace at the beginning of a line is used to determine the grouping of statements (e.g. Rossum, and Drake 2003: sec. 2.1.8). Indentation thereby takes the place of the usual {}’s of other programming languages. Consider this simple example:

// The function returns an integer less than, equal to, or greater than    
// zero if the first argument is considered to be respectively less than   
// equal to, or greater than the second. 
def compare(a, b): 
	if a > b: 
		# a is bigger than or equal to b 
		return 1 
	elif a < b: 
		# b is bigger than a 
		return -1 
	return 0

 

This clever use of whitespace ensures that the code of a python program has a perfectly structured layout (as enforced by the interpreter), which in the coding conventions of most other languages is a ‘best practice’ and by no means a syntactical consideration (e.g. Sun Microsystems 1999: section 4). Similarly, a concrete architecture like M-V dependence may help realise the ideas of the theoretical framework outlined in this report by way of its ‘inherent momentum’, e.g. being ‘conducive’ of analysing and formalising along visual or behavioural dimensions. This sort of agency is, of course, a lot less stringent way than the ‘agency’ of Python’s use of whitespace which is enforced by a compiler and consequently not so much an ‘inherent momentum’ or agency as it is a syntactical rule. But it suffices for our exemplary purposes.

Direct participation of Model System and representational circularity

The direct participation aspect of the Realm of Intended Meaning faces us with a puzzle. How can Objects of the realm of intended meaning represent something in the future and, what is more, something in the future that they themselves are a part of? The fact that this may seem paradoxical, I believe, is typical of Western representational thinking. In fact, the word ‘representation’ seems misleading as it implies a re-presentation of something, a reference of meaning from one realm to the other. Hence, it becomes a circular representation which intuitively seems problematic and this is one of the reasons Objects should not be designated ‘representations’, as I believe the representational aspect can only be used insofar as it constitutes a ‘mental construct’ of analytical convenience to a Western programmer who is used to thinking of the world in a subject-object dualism.

Instead of being representational, Objects actively participate in their own ‘creation’ or in the realisation of their meaning in their direct encounter with the user. Hence, they do not really represent anything other than the intended meaning. When I have chosen to maintain the division in two parts (two realms) despite hereof, it is because I intend to make use of existing knowledge in order not to make my theory too incomprehensible to a person who is familiar with the present theory. Thus, since thinking in representations is our default way of thinking, this semi-representational aspect of my model allows the developer to coordinate her code with something along the lines of usual representational thinking, however elusive future or intended meaning may seem. In natural science thinking, this circularity may seem tautological, but within the framework outlined above, a symbol-symbolised reciprocity is a necessary and basic condition. Only a correspondence theory of truth can completely avoid circularity.

Computer science or object orientation as a behavioural science?

As a further consequence of my theoretical framework, computer science has to be conceived differently than what seems to be the case in most computer science departments.

A computer scientist often finds herself in a precarious situation as she is trapped between two scientific traditions, the natural sciences (e.g. computer science as applied mathematics) and the behavioural sciences (e.g. computer science as making a system that is used by real people, in real organisations, and developed in a creative development cycle). In some parts of her work she must use one set of scientific assumptions and the other set in other parts of her work #21. This dual nature of computer science and how much importance to attribute to one branch or the other is an ongoing discussion (e.g Hasle and Braüner 1998), but taking a sample curricula from a computer science department (DAIMI 2001) all courses, except very few, are concerned with technical subjects such as algorithms and data structures. Only few courses deal with how to carry through a system development project and put the technical knowledge to actual use. Thus, many computer scientists have minimal formal education in the non-technical, ‘soft’ aspects of systems development and must learn this as part of their professional careers.

Perhaps this is the reason why the object-oriented community has an alarming lack of uptake of, or ‘integratability’ vis-à-vis, research results from the HCI community and the behavioural sciences in general. Perhaps this is also the reason the sharp division of labour, enabled by M-V independence, is hailed as a ‘best practice’. And lastly, perhaps this is also the reason that OOAD methodologies present themselves as extraordinarily inclusive (based on a review of RUP, UP (Jacobson et al. 1999) and Agile UP (Larman 2001)), yet basic knowledge of HCI, empirical methods as well as sensitivity to classic methodological issues from the behavioural sciences are absent.

With my reconceptualisation of object orientation, crystallised from my theoretical framework, I attempt to further a view of computer science as ‘computing science’, a discipline with an accentuation of how computing is put to use. To do this, it is necessary to think of the discipline in a different fashion than along the lines of the traditional research communities or university curriculae. I envision object orientation, or what may in broader terms be called ‘empirical computer science’, as a specialised discipline by way of applied results from component disciplines (see Figure 18). This is a way of viewing disciplinary ‘specialisation’ along other lines than those of usual disciplinary borders or the orthodox conception of disciplinary specialisation. I claim that a disciplinary generalisation across specialised areas is in fact disciplinary specialisation and this is the reason the line of reasoning of this report cuts across disciplinary borders and the genre-boundaries of most publications carrying the umbrella term ‘systems development’, ‘informatics’, or ‘computer science’. This is indeed the case of HCI which is a specialised discipline developed from general and non-specialised knowledge of component disciplines. Although these component disciplines are older and more traditional, I do not believe any discipline to be more ‘pure’ than others as well as I find the distinction between applied and basic research highly problematic.

Figure 18: Object orientation, or what may in broader terms be called ‘empirical computer science’, as a specialised discipline by way of results applied from component disciplines

.

Direct Objects as a design stance?

An obvious question to raise is to which extent my redefinition provides us with any guidelines on how to gestalt these so-called ‘direct objects’. To give some context before responding, let us consider the discussion of this subject in the HCI community.

In the 80s there were efforts in the HCI community to explore how formal methods and models (typically of cognition) could be brought to bear on HCI (Carroll 1987, 1991a, 1991b) and some researchers went as far as viewing HCI as an applied cognitive science. The quintessential example of this position is Card, Moran, and Newell (1983) and their keystroke-level GOMS method where a design can be directly derived from theory. In this regard, Card and Newell (1985) even argued that the prospects for psychological science in HCI were slim unless it could provide cognitive models with quantitatively predictive powers. Such attempts to ‘instatiate’ or extrapolate a design from theory did however fail and it furthermore proved problematic to distinguish between applied and basic research.

As part of a counter-reaction to the initial hardening of HCI, some came to hold the position that formal methods, derived from theory, could not be used in HCI at all (Whiteside and Wixon 1987). However, in the middle ground of these extremities, some (e.g. Carroll and Kellogg 1989) opted for a view of HCI design as emerging in the nexus between the designer’s theory, her experience, and the constraints of the situation, process, materials, etc. In this respect, the architect Christopher Alexander #22 (e.g. 1964) was often used as a theoretical backdrop for such a ‘holistic’ view of the design activity, which broadly speaking is still the prevalent stance in especially the Scandinavian countries.

Turning now to the essential question of the extent to which my redefinition provides us with any guidelines on how to gestalt ‘direct objects’: As the reader has probably guessed from the overall stance of this report, I do not believe in instantiating designs from theory. I do however believe there is a place for theory in design activities, whether a hard or soft theory. To take Gibson as example, we may derive a design from his theory regarding how to convey and maintain object identity across time, space, and state changes of that object, but his theory does not enable us to answer whether the given object should appear in the interface in the first place. For this purpose, we may use sociological theories or methods, but in the end we as designers, not the theory, must make the final decision. Luckily, we do so in the nexus of input from multiple theoretical positions, our past experience, prototyping sessions with users, available technologies, and other contextual factors and, as such, our decision is well founded. Thus, in some parts of design activities, theory can be brought directly to bear on design decision-making. In other parts, we are left to our own very a-theoretical devices.

‘Direct’ Objects – a once and future paradigm #23

Encountering meaning directly, as opposed to abstractly, in the form of some notion of ‘objects’ is not new in computer systems design. In the discipline of HCI, Direct Manipulation have been extensively discussed with relation to cognition and epistemology. Also, as we shall see next, discussions of Model-View independency in relation to thinking other than those of Madsen et al. (1993) and Knudsen and Thomsen (1985) have also been taking place in the object-oriented community, although they have not been very elaborate.

The aim of the following sections is to elucidate the empirical inspiration for my revision of object orientation that crystallised from the theoretical framework I outlined, and as a result also provide empirical support of the redefinition.

Model-view dependency – direct object encounter and OOUI’s

Thinking of the Model System as something that is indeed visual, as opposed to the ‘underlying’ and abstract truth Madsen et al. described, is by no means a new affair. In fact, the development of SmallTalk, which provided the breakthrough of object orientation, and the Xerox Star workstation reflected this. This section elucidates parts of the historical and social context (to some extent based on Collins 1995) in which a paradigm of ‘direct object encounter’ evolved in order to illustrate that some object-oriented ideas have reflected a more ‘direct’ conception of meaning.

In the early 70s, Alan Kay started the Learning Research Group at Xerox Parc to develop his vision of a Dynabook, the laptop computer of today. The group was a melting pot of ideas about the relation between human thinking, on one hand, and programming and interacting with computers on the other. Some of them were:

The Learning Research Group was trying to develop an interface or some expressive means to instruct the Dynabook computer, so simple that anyone could use them. In the context of the aforementioned ideas, the research group immediately recognised object orientation’s potential as a cognitive/epistemological tool and the result was a series of SmallTalk systems (that in their infancy were not only programming languages but complete systems).

“The Smalltalk language was co-designed with its user interface. Larry Tesler, one of the researchers, pointed out the strong connection between the properties of objects in object-oriented languages and the properties of corresponding objects on the screen. Dan Ingalls, one of the principal designers and implemtentors of Smalltalk, proposed the reactive principle as the basic idea behind the Smalltalk interface: ‘Every component accessible to the user should be able to present itself in a meaningful way for observation and manipulation’ “ (Collins 1995, p. 37, original emphasis; Ingalls 1981 is cited)

Objects should thus “present themselves for observation and manipulation” and were not, according to the quote by Ingalls, something hidden as seems the case today. Most likely due to the co-evolution of the Smalltalk interface and language, Smalltalk’s stance is more radical than when Madsen et al. note that “The user should have a feeling of manipulating the physical model directly.” (p. 312). Madsen et al. merely suggest a best practice but do not take any consequence hereof neither in their language design nor in their conceptual framework #24.

As reflected in the quote by Collins and Ingalls, Smalltalk had a stance of what may be called ‘direct object encounter’. Though it has not, to my knowledge, been explicated in any article, the influences of the aforementioned important contemporary ideas are quite evident in Smalltalk. Smalltalk had the idea of Engelbart’s augmentation in that it empowered the programmer – it established a cognitive economy of object design-world correspondence at the conceptual level. And because of the world-code correspondence, the programmatic means of expression were also claimed to be structurally similar to the world. Thus the “language” of the world (conceived along the lines of engineering or natural science; in objects and properties) matched the programming language of the computer and the programmer (object orientation), as Joss was perhaps the first language to demonstrate the advantages of. Furthermore, the direct encounter of objects at the interface is akin to Sutherland’s direct manipulation and Papert’s direct (turtle) object encounter. And lastly, it was assumed that object orientation was somehow based on our thought processes, as was investigated in Arfificial Intelligence-related disciplines and by way of notably the LISP language.

Object-Oriented User Interfaces

In the Smalltalk systems, objects were thought of equally in terms of screen objects as software objects. This connection between object properties of screen objects and software objects led Larry Tesler to coin the term “object-oriented user interface” (OOUI) in Tesler (1983). From the viewpoint of the user, the essence of an OOUI was the ability to refer to individual objects, and identify and invoke object behaviours directly upon those references. From an architectural viewpoint, the essence of an OOUI was the intimate relationship between software model and View and in Tesler’s article he investigates the connection between OOUIs and object-oriented programming languages with Object Pascal as an example (which was used in the Apple Lisa Toolkit).

When the Xerox Systems Development Division decided to develop an office workstation the ideas surrounding the Learning Research Group were commercialised and this led to the Xerox 8010 “Star” workstation. The Xerox Star was released in 1981 and some of the core characteristics were:

“Icons represent data objects (documents), containers (file drawers, folders), and tools (printers, in and out mailboxes). Users express object interactions and commands by moving icons and dropping them on other icons. Icons can be asked to show their properties, or their contents, in windows. Views show the state of the objects as accurately as possible […]” (Collins 1995, p. 39)

Thus, icons had a very specific role, namely to represent objects. Objects and their properties were not something hidden in the belly of the computer but something directly there in the interface. This reciprocity of the software object level and the user interface level was a by-product of the co-design and co-evolution of the Smalltalk language and user interface.

Steve Jobs, cofounder of Apple, also visited Xerox Parc and hired Larry Tesler, among others, and subsequently developed the Apple Lisa (released in 1983), and later the Macintosh. Thus, the Macintosh interface, which is widely considered a hallmark of usability, was also founded on the ideas of OOUIs.

The principle that the user first selects an object and subsequently chooses an action to perform upon the object was dubbed the Object-Action principle. The principle entered GUI discussions in the HCI community in especially the 80s and also inspired others to extend the idea, e.g. Shneiderman’s Object-Action-Interface Model (Shneiderman 1997: 61pp). One of the advantages of this Object-Action principle is its complexity-reducing effect in that the GUI only has to display the actions applicable to the selected object. In SmallTalk this was implemented as icons representing objects and the context menu of an object (the right-click menu) presenting the object’s invokable behaviour, a design that was also meant to encourage exploration (Kay 1993: pp. 84-85). According to Kay and his research group, this also corresponded well with our understanding of language. Nouns in a sentence correspond with objects, of which verbs, or commands/methods/functions, are properties, e.g. “the ball bounces” and “myBall.bounce( )”.

To sum up, object orientation as a epistemological tool not only for programmers but mainly for users pervaded the way of thinking in the Learning Research Group and directly or indirectly spawned such important concepts as icons, context menus, OOUIs, the Object Action principle, and indeed SmallTalk itself. Today, there is little connection between objects and icons in the GUI as the idea of separating the View from the Model are being done in a more radical fashion than in the days of Smalltalk. In this light, it is paradoxical that object orientation nowadays only seems to be conceived as a tool for the programmer.

From the above, we may conclude that ‘object-oriented thinking’ in a very less abstract sense than that of Madsen et al. (1993) seems to have been foundational of both SmallTalk and modern user interfaces like Apple and Windows. Thus, the way object orientation was practiced in the North American Smalltalk approach was, to my mind, preferable to the Scandinavian stance of objective modelling, naïve realism, and underlying truth as expressed in Madsen et al. (1993) and as seems to be the predominant approach today. However, the epistemological and ontological assumptions of the North American Smalltalk tradition have never been thoroughly explicated which leaves us guessing as to what they really meant by the idea of object orientation. Scandinavian object orientation on the other hand, did not leave us guessing, but left us with the view that modelling human understanding and experience can be done on the basis of philosophy of science assumptions from the natural sciences. Neither position can be said to be satisfactory.

Why is Model-View independence here?

It is important to maintain that M-V independence is a fundamentally recommendable practice, which in fact was pioneered by Smalltalk as we shall see below. However, I believe that its inherent momentum and the way it has been implemented in parts of the object-oriented community are conducive of some of the pitfalls outlined throughout this report. To clarify what this means, we need to answer why M-V independency is here.

One answer is that developers seem to be particularly distrustful of the user interface. A prevalent attitude is that the interface is the most instable part of the system (perhaps due to what is sometimes described as a ‘muddle’ of user requirements and ‘contradicting’ statements) and instability in an application must be sought contained: If not shielded against, changes in the instable component may spread along dependencies like ripples in water and the developer may find herself chasing these knock-on effects around in the code. When guarding against these forces, developers aim for the instable component to have as low coupling as possible to the rest of the system, or put differently, maximum abstraction or independence of that component. Thereby, a component may change as much as it needs without interrupting the rest of the system. This stance toward decoupling and abstraction of the GUI is embodied in popular design patterns such as Variation Behind Interface (Cockburn 1996: pp. 41-42), User Interface Outside (Cockburn 1996: pp. 42-43), and architectures like the Model View Controller (Krasner and Pope 1988), the Higgens UIMS, and the Seeheim Model #25 (Green 1985). While these practices are generally useful I claim that this separability often results in ignoring that one needs constraints between the two levels in order to prevent a ‘drifting’ of the GUI objects vis-à-vis the domain layer objects (the Model System) thereby failing to benefit from a possible cognitive economy(this was what had happened to the students who produced Figure 2). In the example model I have outlined above (getting the GUI via getPresentation-methods of the primary domain objects) one maintains a constraining model-view dependency while allowing a separation of business logic code and code specific to the handling of GUI widgets.

Another possible answer to why developers are distrustful of interfaces is a more ‘fundamentalist’ one. Alistair Cockburn, a prominent figure in the object-oriented community, captures the idea:

“The point is, the User Interface is not really part of the application. It is the first client of the application. Put the UI outside the application and your application will live a happier, less stressful life (and so will you).” Alistair Cockburn, quoted from http://c2.com/cgi/wiki?UserInterfaceOutside March 28th, 2004

Cockburn interestingly posits that the application is the underlying model as opposed to its representational gloss, the GUI (in this case explained in Client-Server terminology). This is the type of stance I am principally arguing against in this report.

The Model-View-Controller heritage of Smalltalk

In the enthusiasm of the early years of interface design, the people (usually computer scientists) who developed the ideas were also the ones who implemented them. Only later, interface design and interface development became separated and eventually diversified in different research communities, made possible by the sharp distinction between Model and GUI. Although Smalltalk, as mentioned above, embodied the ideas of OOUIs and a strong connection between objects in the Model and in the User Interface (Tesler 1983), Smalltalk in fact pioneered a kind of Model View separability (Ingalls 1978), namely the Model View Controller (MVC) architecture.

Today, the MVC architecture, elaborately presented in Krasner and Pope (1988), is the predominant way of understanding the layering of software and the abstraction of user interface from application semantics. The MVC was developed as a part of Smalltalk-80 (Goldberg and Robson 1983) and was based on a three-way factoring of the elements of the user’s interaction with the system.

1) The user’s conceptual model (corresponding with the Model (M)).

2) The user’s perception of the concrete interface presenting the objects of the Model (the presentation language or ‘look’ of the interface (V)).

3) The user’s interactional possibilities for acting on the objects of the Model as presented through the interface (the action language or ‘feel’ (C))

The three-way factoring is a ‘conceptual’ explanation of the MVC. Viewed at object or code level, the View handles all GUI functionality and event management (if not handled by the operating system or middleware) and the View sends messages (events) to the Controller when the user performs an action. The Controller receives and handles user input (from the events generated in the View component) and notifies the Model to change in accordance with the user’s actions. The Model, once changed, broadcasts change notifications to its dependent Views (through what is popularly called the O bserver Design Pattern, c.f. Gamma et al. 1995) and the Views in turn update the screen.

Based on the conceptual and code-level explanations of the MVC, Collins 1995 pictures the MVC in the following way:

Figure 19: The MVC as a factoring of the elements of the user’s interaction with a computer system. From Collins 1995, p. 305.

One has to be careful in the interpretation of the MVC, as it may evoke an idea of isomorphism between the conceptual model and the Model System (c.f. Collins 1995. p. 305), thereby positing that users hold ‘meaning about the Model’ in their long-term memory (or some notion of mental or conceptual models) derived from interaction with the View. This kind of approach relegates the View the same role as the human perceptual system in mainstream information processing cognitive psychology, namely a transmitter, or an I/O device of ‘aboutness-information’, unless there is a very tight coupling (not programmatic but structural) between the View and Model. In Collins’ model there is one such strong coupling but if this very intimate structural relationship does not exist, the View, and interaction with it, may not be regarded as real in itself. This view unfortunately opens the possibility of distorting the powerful ideas of the MVC in a direction where too much salience is attributed to the Model and the View is deemphasised (at least by the programmer not usually bothered with GUI design). This is what I claim has happened in today’s object orientation.

It should also be noted that a very strong decoupling between Model and View was perhaps sensible in the 80s and maybe as late as the early 90s as the View component was much heavier back then than nowadays. Today, much functionality like Window Management is ‘pushed down’ in the APIs and the programmer of today is free from much of the GUI work of earlier times. This means that the View code is not enormous, thereby allowing a close relationship between View and Model without it resulting in spaghetti code.

Further reading on ‘direct objects’

Lastly, I would like to direct the reader’s attention to some further reading which embodies parts of the ideas of my theoretical framework. In addition to the Smalltalk references already provided, they are IBM’s Systems Application Architecture (SAA), launched in 1987, as described in IBM Corp. (1991) and Martin et al. (1991), and the NakedObjects framework as described in Pawson and Matthews (2002).

For the developer the IBM SAA meant a standard set of API’s, called the Common Programming Interface, that were usable across IBM’s computing environments, and for the end user the SAA meant a standard OOUI, called the Common User Access (CUA) across IBM products. What is interesting about the SAA is that it implements an OOUI and a resulting interweaving and interdependence of the Common Programming Interface (for the programmer) and the Common User Access GUI (for the user), which in effect is a Model-View dependency similar to Smalltalk’s (cf. e.g. IBM Corp. 1991: pp. 1-5, 8, 98, 113-114 and Martin et al. 1991 e.g. p. 41, 93).

The Naked Objects Framework, on the other hand, has recently generated both commercial as well as research interests. Just like the IBM SAA, the early Smalltalk tradition, and the Scandinavian tradition of object orientation, the Naked Objects Framework (nakedobjects.org and Pawson and Matthews 2002) takes an ‘undestanding approach’ to object orientation as opposed to emphasis on reuse and modularisation. It is a Java-based framework that has some interesting features such as a 1:1 relationship between business objects (Model) and the View component, (semi-)auto-generation of user interfaces, OOUI interaction style, radical direct manipulation and hence elimination of semantic feedback problems, and an architecture that discourages programmers from ‘procedural modelling’. The framework is built on the background of its creators’ dissatisfaction with how object orientation is practiced and with the commonplace ignoring of the “true essence of object orientation” which they define as behavioural completeness of objects. Their definition of the essence of object orientation as behavioural completeness is interesting as it constitutes a stance of ‘objects with completeness of meaning’ in line with the arguments put forth in this report.

The most intriguing further reading is probably articles on the Self programming language (most notably Chang et al. 1995, Smith et al. 1995). Self is a prototype-based programming which does away with the duality of classes and instances. In lack of classes, inheritance is achieved by cloning existing objects (hence ‘prototypes’ for the clones). This produces are more immediate sense of object identity without reference to conceptual superstructures.

Summary and conclusions

Madsen et al.’s “conceptual framework for class-based object orientation” (1993) captures what the authors believe to be the idea of object orientation, independent of programming language or system development methodology. A core assumption of their framework is Nygaard’s mantra that “programming is understanding” and consequently that object orientation is principally an epistemological matter. Therefore, their framework attempts to explain the connection between human understanding and object-oriented formalisation, and it is to my knowledge one of the only attempts to define the philosophical/epistemological/ontological assumptions of object orientation.

On the background of having trouble understanding the idea of object orientation myself as well as teaching it to students, I set out to redefine object orientation, the result of which became a more ‘understandable’ approach to understanding. To do this, I investigated the epistemological and ontological assumptions of object orientation and on this basis I suggested an alternative framework for understanding the connection between human understanding and object-oriented formalisation. To concretise my positions, I have provided amble exemplifications in the form of architectures, design patterns, and code examples.

I started my investigation with Knudsen and Thomsen (1985), on which Madsen et al. (1993) draw. I claimed that Knudsen and Thomsen posit a strong subject-object dualism, for example by defining a phenomenon as “a thing that has definite, individual existence in reality or in the mind” and that modelling consequently becomes ‘objective modelling’, i.e. a view of reference as an isomorphic relationship between descriptions and their subjects. Thus, the ‘rightness’ or ‘validity’ of a software model (the Model System) becomes a product of the correspondence relation between that thing ‘out there’ (in the Referent System or in the mind of the programmer) and the software model ‘in here’. We saw how Knudsen and Thomsen apply the classic terminology of Greek philosophy and intensional logic to distinguish between a concept’s extension, intension, and designation and how they make these distinctions cornerstone in establishing meaning. They posit that the abstraction process is an important element in meaning-deduction and accentuate conceptual superstructures in that meaning is somehow found in the ‘abstracted invariance across phenomena’, i.e. the intension of a concept. I claimed this preoccupation with concepts to be characteristic of Western thinking dating back to classical Greek philosophy, especially as manifested in our scientific tradition where we seem to have a predisposition toward looking for a genus.

Both Knudsen and Thomsen (1985) and Madsen et al. (1993) engage in a discussion of ‘formats’, i.e. whether many real-world concepts and phenomena are fuzzy or Aristotelian and note that “a critical aspect of the modelling function will often be to give an Aristotelian definition of problem specific concepts that have a fuzzy nature in the referent system”. I claimed that this sort of (modernistic) discussion of formats has, in many a discipline, led to a misguided hunt for a holy grail in the form of a ‘format’ of semantic memory or a ‘format’ of the physical world.

Next, I showed how Madsen et al.’s stance of ‘objective modelling’ or ‘singular ontology’ spawns the classic problems of correspondence and representation discussed in many disciplines. Modelling, resting on such ontological assumptions, may be appropriate for description of something static or entirely uncontroversial as an agreed-upon biological taxonomy. But even if ‘correct’ modelling of more ‘elusive’ things as the social domain were possible, Madsen et al. note that object orientation should not only enable a faithful modelling of a Referent System, an ‘IT-fying’ of something existing, but also enable reifying a future world (transcending the tradition). Object-oriented analysis thus both involves identifying parts of the real world as well as adding new parts to it. That way, design and implementation activities are about coordinating with this “future world”, i.e. keeping the design and the code ‘in alignment’ with future meaning.

Another important point of Madsen et al’s framework is that the Model System is ‘core meaning’, to which the user interface and various technical services are added to make up the computer system. In positing that object-oriented meaning is underlying, Madsen et al. seem to perpetuate the engineering or hard science view of meaning which is, within computing-related disciplines, epitomised by Simon’s stance that “that complexity, correctly viewed, is only a mask for simplicity", i.e. that something perceptual or representational is merely a mask for underlying order/meaning/simplicity. Thus, ‘above’ the phenomena of the referent system, their meaning, or at least much of it, exists by way of the concepts they belong to (a concept’s intension). Similarly, ‘above’ an object instantiated from its class in the model system, there is a layer of representation (the User Interface) as a mask for the ‘real’ application.

To dig further and more systematically into the ontological/epistemological assumptions of class-based object orientation I analysed Madsen et al.’s framework along two dimensions extracted from broad-brush currents of 20 th century scientific thought; a principally ontological dimension, from ‘Underlying Order’ to ‘Continual Ordering’, and a principally epistemological dimension, from subjectivist to objectivist. In the ontological dimension, both the social and the natural world is seen either as having an underlying order that we can research and thus deduce the nature of or as being something that emerges on the background of continual ordering efforts of the actors. I positioned some theories from psychology, linguistics, and sociology on this spectrum to illustrate the characteristics of the two ends of it.

Turning again to object orientation, I concluded that the hard science assumptions of Underlying Order are easily identifiable in Madsen et al.’s framework, by way of their stance toward physical modelling, i.e. modelling things with substance. Their stance toward physicality and measurability is in line with classic hard science tradition of attempting to de-contextualise information or somehow make it more ‘neutral’. Madsen et al. thus seem to try to define object-oriented understanding on the background of something less controversial or subjective than the social or personal, psychological world. However, when positing an underlying ontological order, a Model System may be claimed highly ‘valid’ but still prove inappropriate once used by real users of the computer system. This gave rise to the paradox that users may ‘deduce the wrong meaning’, i.e. misinterpret the system, where ‘wrong’ is established from the degree of meaning-deviation from the singular ontology.

Furthermore, I claimed that the stance of Underlying Order is characteristic of object orientation in general, and Madsen et al.’s framework in particular, judging from the emphasis on system developer’s analytical role (i.e. peeling off the layer of complexity to reveal underlying order along the lines of Herbert Simon’s views), as opposed to a constructive and interventionistic role. This deductive or analytical stance is also evident in the choice of words, for example, analysis phase as opposed to ‘vision’ phase, or requirements gathering as opposed to requirements construction. In this regard, it seems very indicative that analysis according to dictionaries is “the abstract separation of a whole into its constituent parts in order to study the parts and their relations.” Its antonym is something ‘synthetic’ which is defined as “not genuine or natural”. Meaning, in this quintessential engineering sense, is thus somehow thought to reside in the constituent parts and their relations more so than in the experience and use of something. Applied to object orientation and computer system design, the consequence of such a position is a view of the interface as an add-on to the model, acting as the users’ interpreter vis-à-vis the software model. That the GUI is only a window to the underlying truth is emphasised by designating it as presentation layer.

Turning to the principally epistemological dimension of Madsen et al.’s framework, I analysed the foundation of Madsen et al.’s claim that “object orientation is close to our own natural perception of the real world” or “kindergarten thinking”. In their theory on the ‘process of knowledge’ as they designate it, I claimed that Madsen et al. espouse the archetypal assumptions of an information processing view of human cognition with its hard science assumptions. This is hardly surprising as computer science and cognitive psychology have a record of ‘feeding off each other’ and what has held the sometimes disparate theories of cognitive psychology together has been the analogy between the mind and the computer as information processing devices. When Kristen Nygaard and others in the 60s, 70s, and 80s were pondering on how object orientation hangs together with thinking, they did so in the heyday of computer-mind analogy and on the backdrop of some very influential information-processing theories. Therefore, I claimed that the historical context has made it very compelling to understand object-oriented understanding from a hard science standpoint. In this connection, I elucidated how information processing cognitive psychology and modal accounts of perception have long shown their inadequacies.

Partly due to the salience of concepts and in line with orthodox information processing cognitive psychology, Madsen et al.’s conception of cognition/epistemology does not seem particularly perceptually oriented. We are presented with a view of cognition, thinking, and knowledge creation which relies on (visual) perception to provide information from which to distil meaning/knowledge, as well as a view of perception as not being inherent to cognition or ‘real thinking’. In this regard, I provided some background for understanding object orientation’s problematic heritage from cognitive psychology. Firstly, I showed how the classic duality of perception and cognition is enabled by positing that transductional processes transduce sensory information into meaning. Owing to the transductional processes, consciousness can treat the result as ‘meaning’ and as such, perception and cognition are conceived as independent and modular systems in the brain, as distinguishable concerns in line with GUI/Model separability. As I illustrated, the perception/cognition distinction is also the determining factor in the distinction between modal and amodal theories of knowledge, which became cornerstone in this report. An amodal approach holds that perceptual activation patterns, e.g. the retinal image, are transduced into a nonperceptual representational format, i.e. meaning, bearing no correspondence to the perceptual states that produced them. Such theories have historically often used Aristotelian concepts as representational format of semantic memory or as a way of otherwise explaining epistemological or ontological structuring in these theories. In fact, Aristotle’s conception of meaning was similar to virtually all attempts of explaining the relationship between language and the world, i.e. meaning, in both linguistics and psychology prior to the late 80s. And as I provided evidence to show, an Aristotelian approach may be appropriate to describe a logical system but inappropriate for explaining how humans deal with meaning. Aristotelian accounts of meaning has “led semantic memory research into avenues of investigation that are barren and lead nowhere” to quote one researcher and it should consequently be evident that object orientation’s approach to meaning may be regarded problematic.

An amodal theory of knowledge on the other hand, such as Barsalou’s (1999), posits a convergence of the orthodox distinction of perception and cognition. In fact, Barsalou grounds cognition in perception by providing evidence that perceptual neural systems underlie functions that are normally, in amodal approaches, conceived as higher-order cognitive functions. In Barsalou’s view, meaning is inherently perceptual as meaning is the act of perceiving or ‘replaying the perceived’, re-perceiving so to speak. Consequently, semantics cannot be abstracted from perception, it is to be found in the perception-as-process, and the consequences hereof for object-orientation are substantial. If meaning is inherently perceptual, the model system (codified future meaning) has to be rethought along perceptual lines. Also, meaning is not deduced or extracted by users but also created at the perceptual level.

This raised the question whether it is problematic to rely on an Aristotelian approach to meaning on the premise that computer scientists are comfortable with this approach (i.e. a cultural epistemology). My line of reasoning was that we do not exercise our thinking in ‘natural environments’ but create our cognitive abilities by actively constructing environments in which to exercise these abilities. Similarly, developers exercise their cognitive powers in an environment of object-oriented tools, design patterns, and general object oriented ‘culture’ and since developers are usually trained in natural science thinking, the assumptions of moderate realism in object orientation could perhaps, as a cultural epistemology, be regarded ideal for this target group. According to the line of reasoning presented in this report as well as what Madsen et al. write, this cannot be the case. To be precise, I claimed that object orientation, regarded as programming-as-understanding, should be conceived as to minimise the representational gap between the unit of analysis as a perspective on the world (e.g. a world of functions, procedures, phenomena and concepts or the like) and the formalisms in which the unit of analysis is realised/modelled. However, perspective in the above means the programmer’s perspective and if an objectivist ontology is abandoned, codifying future meaning dependant on the meaning-formation process of other people than the programmers in a way only meaningful to the programmer is problematic. Therefore, the assumptions of object-orientation, as presently conceived, exclude the possibility of an objection like: “If the assumptions are close to our education as engineers and our way of thinking as computer scientists, why change anything?”

As became evident in this report, fundamental assumptions of systems development inevitably influence the outcome of a system development process. Similarly, the ontological and epistemological assumptions of object orientation will inevitably constitute a certain ‘inherent momentum’ and thus influence the contexts in which object orientation is used. With the above analysis of the epistemological and ontological assumptions of object orientation, we became ready to exchange ‘the weak parts’ of object orientation and propose alternatives to get rid of its harmful momentum.From observing the broad-brush directions of research in cognition, understanding, and meaning for the past 40 years and redefining object orientation on the basis of theories different from the mainstream information processing approach, different from the classical amodal approach as well as on alternative underlying philosophy of science assumptions I proposed both a theoretical framework for, as well as a base of empirical research supportive of, a redefinition of object orientation.

The requirement specification for such an update were a framework that enables an understanding of the Model System with a more proactive, suggestive, and purposive role toward the ‘construction of meaning’ vis-à-vis the user. Such a framework may not evade ‘uncertainty problems’ by building a model from ‘stable components’ like physicality and abstracted truth or from components far away from the place of interpretation (e.g. the interface and the use of it). The framework should provide us with an understanding of how we may coordinate our code design with something as nebulous as users’ subjective interpretation or ordering processes. It should also provide us with a better understanding of what I mean by suggestive modelling and how this makes validity become not a question of descriptive correspondence but of closeness of empirically realised meaning vis-à-vis intended meaning. It should also provide a theoretical background on which to answer fundamental questions like “what is meaning in relation to object orientation?”, “what things ‘out there’ do we coordinate our code with?”, “why is this (object) design better than that one?”,

Turning to the theoretical framework, in the first part I contended with the subject-object distinction and explained how we may conceive of representation and meaning without the comprehensibility muddle of Kant’s sharp distinction between ontology and epistemology inherent to object orientation. As an alternative, I suggested to adopt the stance of p henomenologists like Husserl who challenged Kant’s ‘ontological firmness’ with the argument that by perceiving the world out there we construct it at the same time. We co-construct both the firm, the ontology/reality out there, and the perception of it at the same time and it is consequently through epistemology that we create ontology. Following Husserl, I introduced Heidegger who use the term Geworfenheit, or thrownness in English, to describe the fact that we are ‘thrown’ into the world and cannot reflect upon it before standing in the middle of it, which problematises the rationalistic/modernistic ideal of decontextualisation and objective thought (objective modelling, physicality). A subject cannot engage in detached rational reflection about an object since part of the meaning of an object is the subject’s act of perceiving it. In this way, the phenomenologists provided us with a stance toward unity of action and meaning.

From a comparison between the early and late Wittgenstein I proposed to use the late Wittgenstein’s notion that the meaning of a word is how it is used. The late Wittgenstein is compatible with the phenomenological tradition due to his position that meaning is formed on a background of experience and language practice, which perpetuates the phenomenologists’ accentuation of the complementarity of action, experience and meaning. Meaning, according to the late Wittgenstein, is not inherent to words/phenomena, nor established by reference to the outside world, but established in context or language practice. I used this a a further argument against Madsen et al.’s subject-object dualistic and representionalist stance that a phenomenon, as a unit of meaning, is “a thing that has definite, individual existence in reality or in the mind”. With a late-Wittgensteinian approach, it becomes senseless to speak of units of meaning residing somewhere or meaning being somehow physical or otherwise ‘out there’ as opposed to established.

From the phenomenological and late Wittgenstenian tradition, we furthermore concluded that practical action in the world is the foundation of conscious experience and meaning. Consequently, one does not primarily relate to objects in the GUI by deducing their underlying properties and their reference to the world, which makes it pointless to reference the ‘true meaning of systems’, i.e. asking the developer to give the authoritative account of what the GUI-object really consists of and how it really communicates with the software model in such-and-such a way.

Since practical action is the foundation of conscious experience and a notion of ‘true meaning’ as a singular ontology must be abandoned, designing for intended use becomes more important than designing to reach the most truthful/correct (singular) ontology, as we are inclined to do by Madsen et al.’s notion of physical modelling. I used this, together with the observation that practical action is the cradle of meaning, as a further argument for my notion of a suggestive Model System along the following line of reasoning: Since the Model System is future meaning and that we aim for the intended meaning of our Model System to coincide with the user’s, phenomenology provides us with a guideline as to how this can be achieved; namely through practical pretheoretical action in concernful activity. In other words, the only way we can aspire to exert any control over how meaning is formed, i.e. to ensure the ‘validity’ of our Model System, is by being suggestive of practical action, to suggest meaning-in-use. Therefore, I emphasised the need to design objects in the Model System that users encounter directly, rather than abstractly as ‘Properties’ of objects are mainly properties insofar as they are directly encountered, as opposed to deduced or logically abstracted. They become real in their consequences.

Lastly, I concluded from phenomenology that representations are as representations do and what they do is get people to act. Representations are as representations are experienced and acted upon.

Since meaning is realised in a socio-technical network, not in a rationalistic universe of Newtonian facts, a further requirement of a theoretical framework for object orientation is a sociological account of meaning and meaning-establishment. Firstly, I set out drawing a distinction between two branches of sociology. One branch, epitomised by the theories of Weber and Durkheim, posit that the social world is fundamentally ordered by ‘underlying’ principles and causalities and they believe that by studying social action that appears meaningful, sociology may extrapolate these principles and define the criteria for meaningfulness. This is in effect a sociological stance of ‘underlying meaning’ as I explained it in relation to the ontological assumptions of object orientation. The other branch of sociology is among others represented by Schütz, who argued that the social world is not meaningful per se, but becomes meaningful through the action of individuals. Perpetuating the tradition of Schütz, Garfinkel proposed accountability as a central concept, which describes the mechanisms whereby a community establishes meaningfulness or a sense of what is considered rational action. In Garfinkel’s theory, meaningfulness is established through perceiving others act meaningfully and one thus co-establishes meaningfulness through acting meaningfully (setting an action-example). Taking Garfinkel’s stance and being inspired by Madsen et al.‘s problems of correspondence and representation, I argued that we need to pay attention to how computation participates in the world it represents. In this sense, I argued that representation and participation are two sides of the same coin, which was one of the reasons I proposed to conceive of the object-oriented Model System in a way that includes how it participates in the world it represents, as a sort of social actor. For this purpose, I used Garfinkel’s principle of accountability in proposing to conceive of a suggestive Model System as an accountable one, one that makes its causalities or ‘rules of meaningfulness’ visible-and-reportable to the user with whom it is supposed to establish common meaningfulness. A computer system whose inner workings and causalities (i.e. the Model System) are visible-and-reportable once again means a dissociation from the stance of underlying meaning but, conversely, it represents a conception of meaning compatible with a modal theory of knowledge, i.e. perceptually-based thinking.

The aforementioned sociological accounts of meaning showed us how meaning emerges at the sociological level of description and through the principle of accountability we found a basis for direct encounter of the Model System (a stance of what-you-see-is-what-it-is). My aim has however also been to provide a framework on the background of which one can answer very low-level questions with regards to the direct encounter of objects, i.e. how meaning is established at the very low-level perceptual experience of a concrete interface. Therefore, I provided an operationalising of Gibson’s Direct Perception theory.

I showed how Gibson’s research is compatible with the phenomenologists’ in that it takes a subject-object reciprocal stance to meaning which locates meaning at a phenomenal level of description between the actor and his environment (exemplified by Tau, the time to impact of approaching object), as opposed to a ‘neutral’ level of description.

At the visual-perceptual level, Gibson drew a distinction between the kind of ‘snapshot vision’ that is relied on in laboratory tests and visual awareness over time, which takes place during natural locomotion and limb movement. This meant rejecting the prevailing concept of time as conceived in the Western scientific tradition as (finite) slices of time jointly constituting a timeline, e.g. T1, T2…T10, where variance consequently can be defined as the difference between T x and T y. Gibson rejected that the human perceptual systems is built to perform this kind of ‘calculation of difference’ but that it picks up a continuous stream, a flux, of sensory stimulation, which cannot be fixated in time and invariance can thus be conceived as ‘invariance in flowing variance’. Applying this to computing in general and object orientation in particular, I showed how Gibson’s stance of what I chose to call ‘longitudinal time’ are contrary to how most interfaces, e.g. WIMP, and most (object-oriented) GUI toolkits and libraries are construed: It seems that the user is expected to perform a comparison between what-is-now-with-what-was-then to inform her of the state changes of the software model, a sort of synchronisation between the software model and the mental model of the user (epitomised by dialogs, information ‘about’ what goes on). From Gibson’s notion of longitudinal time-space, I concluded that the difference between the functional spaces of information technology and the real world seems to be a difference of visual object substantiality and continuity.

In consequence, if our thinking is first and foremost perceptual, meaning is not somehow ‘conveyed’ to the user by the GUI and we have to stop talking of the essence of thought as “extrapolated semantic chunks”. In the framework I have laid out, there can be no ‘essence’ abstractable from the very process of perceptually-based thought. I argued, that as long as we think of interaction in terms of what is being done and not so much how it is being done, we will continue to see the human-computer encounter in a modernistic and information-processing way. Instead, a perceptual, action-oriented, subject-object reciprocal, ecological theory of knowledge would hold that the process of use of the GUI in extended (longitudinal) time-space is the locus of meaning; the “how it is being done”. Therefore, I argued that the relationship between the software model (the locus of meaning for the programmer) and the use of the GUI in concernful activity (the locus of meaning for the user) is of critical importance. Whether the user drags A into B as opposed to A onto B will evidently influence her conception of the causal relationship between A and B, and I claimed that this should be part of the domain of coordination of the Model System.

Next, Gibson’s stance of perception-for-action showed us that perception is (inter)action, not disembodied rationality. We saw how perception in fact requires action and cannot be abstracted from the activity. Hence, it is not neutral the same way perception-as-recognition of orthodox cognitive psychology is. In object-oriented terms, the Model System thus finds its meaning through inter(action), a good argument for moving the Model System to the place “where the action is”, namely to the GUI in some form of Model-View coupling or merger.

Furthermore, as a consequence of Gibson’s theory of meaning-as-affordances, acts of perceiving, and hence knowing, are acts of perceiving value-rich ecological objects. This provided us with a theory where affection and emotions are inherent to the theory and do not have to be treated as a ‘special case’ as is often the case in computing-related disciplines that have come to rely on theories and theoretical frameworks like information processing psychology. Such theoretical footing can, I claimed, only provide a view of technology as “what it really is underneath the pretty icons and bright colours”.

I also claimed that using a reorientation from mechanical (information processing) to ecological/perceptual understanding to inform object orientation would distinguish it from the ‘harder’ computer science communities. I pointed out the paradox that object orientation shares its fundamental epistemological and ontological outlook with communities that view themselves as applied mathematics, resting firmly on natural science assumptions.

In all in all, I showed that the research in cognitive psychology and related ‘epistemological’ disciplines that is supportive of Madsen et al.’s conceptual framework has long proven its inadequacies and the framework I have presented has taken the consequence hereof. This framework may help further a view of computer science as ‘computing science’, an accentuation of how computing is put to use.

Having provided a theoretical framework for object orientation, I crystallised an updated conception of the act of modelling on this theoretical background. The main components of my reconception of modelling were:

In addition to the above, I provided a concrete code example from a potential architecture that realises the general outlook of the theoretical framework. My intention was to provide an example to ‘ground’ the sometimes abstract and theoretical discussions of this report, to get the reader’s imagination going, and lastly to accentuate the importance of reifying or ‘thingifying’ a theoretical position. I argued that to embed a theoretical (re)definition of object orientation into a tool, an architecture, or a more ‘tangible’ idea will have the advantage of ‘sneaking theory in the back door’. If one makes a tool/architecture/etc that has a certain inherent momentum to it, that momentum may move practice in the desired direction, instead of taking the opposite strategy of first educating computer scientists of epistemological matters and subsequently trying to make them adjust their practice accordingly.

To further illustrate that my ideas are not merely theoretical constructs or idealised intellectual trumpery, I elucidated the empirical inspiration for my revision of object orientation that crystallised from the theoretical framework I outlined, which also provided empirical support of the viability of my redefinition. This took the form of an archaeology of ‘direct objects’ as it turned out that thinking of the Model System as something very visual, as opposed to the ‘underlying’ and abstract truth Madsen et al. described, is by no means a new affair. This ‘once and future paradigm for object orientation’ where objects are encountered directly was the paradigm of the SmallTalk tradition and what became known as object-oriented user interface (OOUIs). As I showed, Smalltalk was the melting pot of some contemporary trends of computing that were thought out from the human side of the human-computer encounter. These original ideas of ‘direct object orientation’ spawned such important concepts as icons (as well as much of the core WIMP widgets), the Object-Action principle , the Macintosh interface, the Windows interface and more. This core principle of ‘directness of meaning’ has unfortunately been diluted since then.

As final evidence of the viability of my approach I showed that pursuing a principle of Model-View dependence can be done while maintaining the many merits of Model-View independence. I showed that Smalltalk’s featured a reciprocity of the software object level and the user interface level although at the same time it was Smalltalk that pioneered Model-View separability, namely through the Model View Controller (MVC) architecture. Since the MVC architecture has been foundational to MV-independence, I explained the Model-View architecture and argued that the contemporary trend for complete separation of M and V often results in ignoring the need for constraints between the two levels in order to prevent a ‘drifting’ of the GUI objects vis-à-vis the domain layer objects and a consequent failing to benefit from a possible cognitive economy. I argued that an interpretation of MVC along the lines of the original stance of ‘directness of meaning’ may allow for separation of GUI and business logic code while maintaining a close relationship between View and Model without resulting in spaghetti code.

Taken as a whole, I have provided a redefinition of object orientation that can be characterised as a refocus from abstract to direct objects and a more understandable approach to understanding.

Notes

  1. By application semantics I mean the Model System (in the rhetoric of Madsen et al.), the Model component (in rhetoric of the Model-View-Controller pattern) or simply the ‘intestines’ of a system. By semantics I do not intend to connote such things as denotational, axiomatic or operational semantics which are computer science terms with very specific meaning.
  2. Alluding to the research results on end-user programming presented in Nardi’s book “A small matter of programming” (1993)
  3. A ‘UIMS’ (User Interface Management System) became the HCI-word to designate a software architecture in which the implementation of the user interface is clearly separated from that of the application's underlying functionality.
  4. at least until the Contextual Turn of HCI, c.f. Jensen and Søgaard 2004
  5. a domain model is a type of model of the problem domain in the Referent System and a design class di agram is a type of model of the Model System. Therefore, they should be, at least from a Scandinavian perspective, be regarded as the most important models
  6. I have been inspired by Schefe’s (2000) distinction between model of and model for.
  7. Einstein is often quoted for saying that his pen was smarter than him. By ‘smartness’ he was referring to his pencil's ability to fixate temporary results on paper, freeing up his working memory and thereby enabling him to solve an arithmetic problem in increments. Without the pencil and paper, he would have to store the results of the increments in short term memory, quickly exceeding its limits. Artifacts like Einstein’s pen and their relation to our cognitive powers have long been of interest in communities studying cognition
  8. For one of the first and best expositions of ‘nerd’ culture and man-tool reciprocity, see Turkle 1984
  9. Given that much of the world is not best described in objects and properties, as the emergence of postmodernity has shown, the natural science stance of viewing the world in objects is mainly metaphorical and consequently very unnatural as “the essence of metaphor is understanding and experiencing one kind of thing in terms of another” (Lakoff and Johnson 1983: p. 5)
  10. This phenomenon of focus shift is well-known in many disciplines, although its nature and the conclusions drawn from it are different. For example, Polanyi (1966) speaks of the dual activity of subsidiary and focal awareness. Cognitive psychologists speak of controlled and automatic processes in connection with consciousness and learning and use it to speak of performance characteristics of certain cognitive tasks. Activity theorists speak of focus shifts between operations and actions (e.g.Bødker 1991).
  11. Unless one is programming for performance or from other considerations of technical rationality as opposed to programming-as-understanding.
  12. as Jensen and Søgaard (2004) have called it
  13. It seems to me that Dourish’s designation of accountability as an interactional property as “the story the system tells about itself” connotes ‘aboutness’. I would have chosen something like “transparency of the causality of the system”, which, however, is less catchy.
  14. In addition, in a previous section I mentioned some theories which may be called orthodox.
  15. e.g. the relationship between the visio-spatial sketchpad and the central executive in Baddeley’s working memory model, see Baddeley 1997.
  16. Source: The dictionary databases at dict.org
  17. Not surprisingly, cognitive scientists such as Fodor and Pylyshyn have extensively criticised Gibson (e.g. Fodor and Pylyshyn 1981)
  18. I mean a unity of orthodox perception and orthodox cognition in broad outline only, as Gibson rejected many basic assumptions of orthodox perception and cognition altogether.
  19. WIMP stands for Windows, Icons, Menus and Pointers (or maybe Windows, Icons, Mouse, Pull-down menus). WIMP is the style of graphical user interface that uses the above-mentioned common widgets. It was invented at Xerox PARC, popularised by the Apple Macintosh, and now available in other varieties such as the Microsoft Windows operating system, the X Window System, OSF/Motif, NeWS and RISC OS.
  20. e.g. passively observing how the email object is received and visually and continually moves/morphs itself into a folder
  21. A self-imposed multiple personality disorder seems a basic condition for computer scientists.
  22. Now that I mention Alexander I can simply not resist to comment on how his concept of patterns has been misconceived in the computer science community. To Alexander, a pattern is a way to create what he called ‘the quality without a name’, i.e. the perfect fit between form (i.e. a design) and its context. By quality, he meant quality for people, not quality measured as some universal objective phenomenon. Yet the way his idea of patterns is used in the computer science community is usually codified best practices with regards to a technical design, i.e. design patterns, which Alexander himself has explicitly argued against ( Alexander 1964: Foreword).
  23. The title is inspired by Michael Cole’s “Cultural Psychology – a once and future discipline” (1998)
  24. In this regard, it should also be noted that the difference between my model outlined above and Madsen et al.’s is that the Model System in my model, called Realm of Intended Meaning, presents itself by (sometimes) graphical means whereas Madsen et al.’s model is presented via a GUI. I hope the reader does not think of this as a subtle difference but a very crucial one. It is the difference between a descriptive underlying model, and a more proactive, suggestive, direct model
  25. So called because it was developed at a conference in Seeheim.

Acknowledgements

I would like to extend many thanks…
- to my thesis adviser Per Hasle and to Susanne Bødker.
- to Jonas Kroustrup, for proof-reading and commenting on the sociological sections as well as the many professional talks over coffee.
- to Ole Villumsen for hair-splitting proof-reading and feedback on the object-oriented sections.
- to Sanne Fjord Jensen at the DAIMI library for providing me with a steady stream of books and going as far as getting hold of old IBM manuals from German and Austrian libraries.
- to the students of the course ‘Programmering og Systemudvikling’ at IMV (fall 2003) for pointing out the counter-intuitiveness of object orientation.
- to some of the students of the course ‘Eksperimentel Systemudvikling’ at DAIMI for passionately debating the fundamental question of what computer science really is.

 

References (bibliography)

ACM (2001). ACM Turing Award Page for 2001. Retrieved September 5, 2003 from ACM: http://www.acm.org/awards/turing_citations/dahl-nygaard.html 

Alexander, Christopher (1970): Notes on the Synthesis of Form. Harvard University Press

Amarel, Saul (1966): On the mechanization of creative processes. In IEEE Spectrum, 3 p. 112-114

Andersen, Peter B. (year unknown). What semiotics can and cannot do for HCI. Retrieved August 7, 2003 from University of Aalborg, Denmark: http://www.cs.auc.dk/~pba/Preprints/WhatSemioticsCan.pdf 

Atkinson, Richard C., Shiffrin, Richard M. (1968): Human memory: A proposed system and its control processes. In: Spence, K. W., Spence, J. T. (Eds.) "The Psychology of Learning and Motivation: Advances in Research and Theory (Vol. 2)". Academic Press

Baddeley, Alan D. (1997): Human Memory: Theory and Practice (Rev. Ed.). Hove, Psychology Press

Baddeley, Alan D., Hitch, G. J. (1974): Working Memory. In: Bower, G. H. (Ed.) "Recent Advances in Learning and Motivation (Vol 8)". Academic Press

Bannon, Liam (1991): From human factors to human actors: the role of psychology and human-computer interaction studies in system design. In: Greenbaum, Joan, Kyng, Morten (Eds.) "Design at work: cooperative design of computer systems". Lawrence Erlbaum Associates

Barsalou, Lawrence (1999): Perceptual Symbol Systems. In Behavioral and Brain Sciences, 22 p. 577-660

Beaudouin-Lafon, Michel (2000): Instrumental Interaction: An Interaction Model for Designing Post-WIMP User Interfaces. In: Turner, Thea, Szwillus, Gerd, Czerwinski, Mary, Peterno, Fabio, Pemberton, Steven (ed.): Proceedings of the ACM CHI 2000 Human Factors in Computing Systems Conference. April 1-6, 2000, The Hague, The Netherlands. p.446-453

Beck, Kent (1999): Extreme Programming Explained: Embrace Change. Reading, MA, Addison-Wesley Publishing

Bertelsen, Olav, Bødker, Susanne, Kuutti, Kari (ed.): Proceedings of the Second Nordic Conference on Human-Computer Interaction - Nordichi 2002. October 19-23, 2002, Aarhus, Denmark.

Bertelsen, Olav, Pold, Søren ( 2002): Towards the aesthetics of Human-computer interaction. In: Frøkjær, Erik, Hornbæk, Kasper (ed.): Proceedings of the 2nd Danish Human-Computer Interaction Research Symposium. November 7, 2002, Aarhus, Denmark. p.

Beyer, Hugh, Holtzblatt, Karen (1998): Contextual Design: Defining Customer-Centered Systems. San Francisco, Morgan Kaufman Publishers

Biederman, Irving (1987): Recognition-by-components: A theory of human image understanding. In Psychological Review, 94 p. 115-147

Booch, Grady, Rumbaugh, James, Jacobson, Ivar (1998): The Unified Modeling Language User Guide. Addison-Wesley Publishing

Broadbent, Donald (1958): Perception and Communication. Oxford, England, Pergamon

Burrell, Gibson, Morgan, Gareth (1979): Sociological Paradigms and Organizational Analysis: Elements of the Sociology of Corporate Life. London, Heinemann

Button, Graham, Dourish, Paul (1996): Technomethodology: Paradoxes and Possibilities. In: Tauber, Michael J., Bellotti, Victoria, Jeffries, Robin, Mackinlay, Jock D., Nielsen, Jakob (ed.): Proceedings of the ACM CHI 96 Human Factors in Computing Systems Conference. April 14-18, 1996, Vancouver, Canada. p.19-26

Bærentsen, Klaus B. (2000): Intuitive User Interfaces. In Scandinavian Journal of Information Systems, 12 p. 29-60

Bærentsen, Klaus B., Trettvik, Johan (2002): An Activity Theory Approach to Affordance. In: Bertelsen, Olav, Bødker, Susanne, Kuutti, Kari (ed.): Proceedings of the Second Nordic Conference on Human-Computer Interaction - Nordichi 2002. October 19-23, 2002, Aarhus, Denmark. p.51-60

Bødker, Susanne (1991): Through the Interface - A Human Activity Approach to User Interface Design. Hillsdale, NJ, Lawrence Erlbaum Associates

Card, Stuart K., Mackinlay, Jock D., Shneiderman, Ben (1999): Information Visualization. In: Card, Stuart K., Mackinlay, Jock D., Shneiderman, Ben (Eds.) "Readings in Information Visualization: Using Vision to Think". Academic Press

Card, Stuart K., Moran, Thomas P., Newell, Allen (1983): The Psychology of Human-Computer Interaction. New Jersey, Lawrence Erlbaum Associates

Carroll, John M. (1991): Introduction: The Kittle House manifesto. In: Carroll, John M. (Ed.) "Designing Interaction: Psychology at the Human-Computer Interface". Cambridge University Press

Carroll, John M. (Ed.) (1987): Interfacing Thought. Cambridge, MIT Press

Carroll, John M. (Ed.) (1991): Designing Interaction: Psychology at the Human-Computer Interface. Cambridge University Press

Carroll, John M., Kellogg, Wendy A. (1989): Artifact as Theory-Nexus: Hermeneutics Meet Theory-Based Design. In: Bice, Ken, Lewis, Clayton H. (ed.): Proceedings of the ACM CHI 89 Human Factors in Computing Systems Conference. April 30 - June 4, 1989, Austin, Texas. p.7-14

Carroll, John M., Kellogg, Wendy A., Rosson, Mary Beth (1991): The Task-Artifact Cycle. In: Carroll, John M. (Ed.) "Designing Interaction: Psychology at the Human-Computer Interface". Cambridge University Press

Carroll, John M., Rosson, Mary Beth (2003): Design Rationale as Theory. In: Carroll, John M. (Ed.) "HCI Models, Theories, and Frameworks". Morgan Kaufman Publishers

Chang, Bay-Wei, Ungar, David, Smith, Randall B. (1995): Getting Close to Objects: Object-Focused Programming Environments. In: Burnett, Margaret, Goldberg, Adele, Lewis, Ted (Eds.) "Visual Object-Oriented Programming: Concepts and Environments". Prentice Hall

Cockburn, Alistair (1996): The interaction of social issues and software architecture. In Communications of the ACM, 39 (10) p. 40-46

Cole, Michael (1998): Cultural Psychology: A Once and Future Discipline. Cambridge, MA, The Belknap Press of Harvard University Press

Collins, Dave (1995): Designing Object-Oriented User Interfaces. Addison-Wesley Publishing

Craik, Kenneth (1943): The Nature of Exploration. Cambridge, England, Cambridge University Press

Daimi (2001). Studieordning for Datalogi.

Dix, Alan (2003): Upside-down A's and Algorithms - Computational Formalisms and Theory. In: Carroll, John M. (Ed.) "HCI Models, Theories, and Frameworks". Morgan Kaufman Publishers

Dourish, Paul (1995): Accounting for System Behaviour: Representation, Reflection and Resourceful Action. In: Proceedings of the third decennial conference Computers In Context CIC-95. August 14-18, 1995, Aarhus, Denmark. p.147-156

Dourish, Paul (2001): Where the Action Is: The Foundations of Embodied Interaction. MIT Press

Durkheim, Emile (1933): The Division of Labor in Society (Translated with an introduction by George Simpson). New York, Free Press

Edmonds, Ernest (Ed.) (1992): The Separable User Interface. Academic Press

Ehn, Pelle (1988): Work-Oriented Design of Computer Artifacts. Stockholm, Arbetslivscentrum

Engelbart, Douglas C. (1962). Augmenting human intellect: A conceptual framework (AFOSR-3233 Summary Report, SRI Project No. 3578). AFOSR, Stanford Research Institute http://www.liquidinformation.org/engelbart/62_paper_full.pdf

Eysenck, Michael W., Keane, Mark T. (2000): Cognitive Psychology. A Student's Handbook. (Fourth edition). London, Psychology Press

Feldman, Stuart I. (ed.): The second ACM SIGPLAN conference on History of programming languages. April 20 - 23, 1993, 1993, Cambridge, Massachusetts, United States.

Fodor, Jerry A. (1975): The Language of Thought. Cambridge, MA, USA, Harvard University Press

Fodor, Jerry A., Pylyshyn, Zenon (1981): How direct is visual perception? Some reflections in Gibson's 'Ecological Approach'. In Cognition, 9 p. 139-196

Gamma, Erich, Helm, Richard, Johnson, Ralph, Vlissides, John (1995): Design Patterns: Elements of reusable object-oriented software. Addison-Wesley Publishing

Garfinkel, Harold (1967): Studies in ethnomethodology. Englewood Cliffs, NJ, USA, Prentice Hall

Gaver, William W. (1991): Technology Affordances. In: Robertson, Scott P., Olson, Gary M., Olson, Judith S. (ed.): Proceedings of the ACM CHI 91 Human Factors in Computing Systems Conference. April 28 - June 5, 1991, New Orleans, Louisiana. p.79-84

Gentner, Donald R., Nielsen, Jakob (1996): The Anti-Mac Interface. In Communications of the ACM, 8 p. 70-82

Gibson, James J. (1979): The Ecological Approach to Visual Perception. New Jersey, USA, Lawrence Erlbaum Associates

Goldberg, Adele, Robson, David (1983): SmallTalk-80: The Language and Its Implementation. Addison-Wesley Publishing

Green, Mark (1985): Report on Dialogue Specification Tools. In: Pfaff, Günther E. (Ed.) "User Interface Management Systems". Springer Verlag

Hansen, Per Brinch (1993): Monitors and concurrent Pascal: a personal history. In: Feldman, Stuart I. (ed.): The second ACM SIGPLAN conference on History of programming languages. April 20 - 23, 1993, 1993, Cambridge, Massachusetts, United States. p.1-35

Hartnack, Justus (1994): Wittgenstein og den moderne filosofi. Copenhagen, Denmark, C. A. Reitzels Forlag

Hasle, Per, Braüner, Torben (1998): Systemudvikling - Logos eller Mythos?. In Rhetorica Scandinavica, 6 p. 33-45

Heckel, Paul (1984): The Elements of Friendly Software Design. Warner Books

Heft, Harry (2004). The Place of Feelings in Ecological Psychology: A Jamesian Excursion. In: Program for the mini-conference on Ecological Psychology. August 26-27. Aarhus, Denmark.

Helmholtz, Hermann Ludwig Ferdinand Von (1962): Treatise on Physiological Optics (translated and edited by J. P. C. Southhall). New York, USA, Dover

Hirschheim, Rudy, Klein, Heinz K. (1989): Four paradigms of information systems development. In Communications of the ACM, 32 (10) p. 1199-1216

Hudson, Scott E. (1987): UIMS Support for Direct Manipulation Interfaces. In ACM SIGGRAPH Computer Graphics, 21 (2) p. 120-124

Hudson, Scott E., King, Roger (1988): Semantic Feedback in the Higgens UIMS. In IEEE Transactions on Systems, Man, and Cybernetics, 14 (8) p. 1188-1206

Husserl, Edmund (1970): The Crisis of European Sciences and Transcendental Phenomenology: An Introduction to Phenomenological Philosophy. Evanston, Northwestern University Press

Hutchins, Edwin, Hollan, James D., Norman, Donald A. (1986): Direct Manipulation Interfaces. In: Norman, Donald A., Draper, Steven (Eds.) "User Centered System Design: New Perspectives on Human-Computer Interaction". Lawrence Erlbaum Associates

IBM Corporation (1991). Systems Application Architecture: Common User Access Advanced Interface Design Reference (SC34-4290-00). IBM Corporation

Ingalls, Daniel H. (1978): The Smalltalk-76 Programming System: Design and Implementation. In: Conference Record of the Fifth Annual Symposium on Principles of Programming Languages. 1978, Tuscon, Arizona, USA. p.9-15

Ingalls, Daniel H. (1981): Design Principles Behind Smalltalk. In Byte, 6 (8) p. 286-298

Ittelson, W. H. (1951): Size as a cue to distance: Static localisation. In American Journal of Psychology, 64 p. 54-67

Jacobson, Ivar (1987): Object-oriented development in an industrial environment. In SIGPLAN Notices, 22 (12) p. 183-191

Jacobson, Ivar, Booch, Grady, Rumbaugh, James (1999): The Unified Software Development Process. Reading, MA, Addison-Wesley Publishing

Jacobson, Ivar, Christerson, Magnus, Jonsson, Patrik, Övergaard, Gunnar (1992): Object-oriented software engineering: A use-case driven approach. Reading, MA, Addison-Wesley Publishing

Jensen, Mikkel, Soegaard, Mads (2004): Refocusing the Contextual Turn: The Forgotten Construction of Meaning at the Interface. In: Hyrskykari, Aulikki (ed.): Proceedings of the Third Nordic Conference on Human-Computer Interaction. October 23-27, 2004, Tampere, Finland. p.185-188

Katz, Jerrold, Fodor, Jerry A. (1963): The structure of a semantic theory. In Language, 39 p. 170-210

Kay, Alan C. (1993): The Early History of Smalltalk. In SIGPLAN Notices, 28 (3) p. 69-95

Kintsch, Walter (1980): Semantic Memory: A tutorial. In: Nickerson, R. S. (Ed.) "Attention and performance VIII". Lawrence Erlbaum Associates

Knudsen, Jørgen L., Thomsen, Kristine S. (1985). A Conceptual Framework for Programming Languages. DAIMI-PB, Daimi, Aarhus University

Krasner, G. E., Pope, S. T. (1988): A Cookbook for Using the Model-View-Controller User Interface Paradigm In Smalltalk-80. In Journal of Object-Oriented Programming, 1 (3) p. 26-49

Krippendorf, Klaus (1995): On the essential contexts of artifacts or on the proposition that “design is making sense (of things)?. In: Buchanan, Richard, Margolin, Victor (Eds.) "The Idea of Design". MIT Press

Krogdahl, S., Olsen, K. A. (1986): Modular and Object-Oriented Programming. In Datatid, 9 p.

Kunda, Ziva (1999): Social cognition: Making sense of people. Cambridge, MA, MIT Press

Kvale, Steinar (1995): The Social Construction of Validity. In Qualitative Inquiry, 1 (1) p. 19-40

Köhler, Wolfgang (1947): Gestalt Psychology. New York, Liveright

Lakoff, George, Johnson, Mark (1983): Metaphors We Live by. New York, Basic Books

Larkin, Jill H., Simon, Herbert A. (1987): Why a diagram is (sometimes) worth 10,000 words. In Cognitive Science, 11 p. 65-100

Larman, Graig (2001): Applying UML and Patterns: An Introduction to Object-Oriented Analysis and Design and the Unified Process (2nd Ed.). Prentice Hall

Latour, Bruno (1987): Science in Action: How to Follow Scientists and Engineers Through Society. Cambridge, MA, Harvard University Press

Latour, Bruno (1988): Mixing humans and non-humans together: the sociology of a door-closer. In Social Problems, 35 (3) p. 298–310

Latour, Bruno (1991): We Have Never Been Modern. Cambridge, MA, Harvard University Press

Laurel, Brenda K. (1986): Interface as Mimesis. In: Norman, Donald A., Draper, Steven (Eds.) "User Centered System Design: New Perspectives on Human-Computer Interaction". Lawrence Erlbaum Associates

Lee, David N. (1976): Visuo-motor coordination in space-time. In: Stemach, G.E., Requin, J. (Eds.) "Tutorials in motor behaviour". North-Holland

Lewis, Clayton H., Olson, Gary M. (1987): Can Principles of Cognition Lower the Barriers to Programming?. In: Olson, Gary M., Sheppard, Sylvia B., Soloway, Elliot (Eds.) "Empirical Studies of Programmers: Second Workshop". Ablex Publishing

Lindsay, P. H., Norman, Donald A. (1977): Human Information Processing: An Introduction to Psychology. New York, Academic Press

Löwgren, Jonas, Stolterman, Erik (1998): Design av informationsteknik - materialet utan egenskaper. Lund, Sweden, Studenterlitteratur

Madsen, Ole L. (1995): Open Issues in Object-Oriented Programming - A Scandinavian Perspective. In Software - Practice and Experience, 25 p.

Madsen, Ole L., Møller-Pedersen, Birger, Nygaard, Kristen (1993): Object-Oriented Programming in the BETA Programming Language. Addison-Wesley Publishing

Martin, James, Chapman, Kathleen Kavanagh, Leben, Joe (1991): Systems Application Architecture. Englewood Cliffs, New Jersey, Prentice Hall

McConnell, Steve (1996): Rapid Development: Taming Wild Software Schedules. Redmond, Wa., USA, Microsoft Press

McGrenere, Joanna, Ho, Wayne (2000): Affordances: Clarifying and Evolving a Concept. In: Proceedings of Graphics Interface 2000. 2000, Montreal, Canada. p.179-186

Miller, George A. (1956): The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information. In Psychological Review, 63 p. 81-97

Mook, D. G. (1983): In defence of external invalidity. In American Psychologist, 38 p. 379 - 387

Mook, D. G. (1989): The myth of external validity. In: Poon, L. W., Rubin, D. C., Wilson, B. A (Eds.) "Everyday cognition in adulthood and late life". Cambridge University Press

Nardi, Bonnie A. (1993): A small matter of programming: Perspectives on end user computing. Cambridge, MA, MIT Press

Naur, Peter (1992): Computing: A Human Activity. New York, ACM Press

Neisser, Ulric (1989): Concepts and Conceptual Development : Ecological and Intellectual Factors in Categorization. Cambridge University Press

Newell, Allen, Card, Stuart K. (1985): The prospects for psychological science in human-computer interaction. In Human-Computer Interaction, 1 (3) p. 209-242

Nielsen, Jakob (1994): Heuristic evaluation. In: Nielsen, Jakob, Mack, Robert L. (Eds.) "Usability Inspection Methods". John Wiley and Sons

Norman, Donald A. (1980): Twelve Issues for Cognitive Science. In Cognitive Science, 4 p. 1-32

Norman, Donald A. (1983): Some Observations on Mental Models. In: Gentner, Donald R., Stevens, A. L. (Eds.) "Mental Models". Lawrence Erlbaum Associates

Norman, Donald A. (1986): Cognitive engineering. In: Norman, Donald A., Draper, Steven (Eds.) "User Centered System Design: New Perspectives on Human-Computer Interaction". Lawrence Erlbaum Associates

Norman, Donald A. (1988): The Design of Everyday Things. New York, Doubleday

Norman, Donald A. (1991): Cognitive artifacts. In: Carroll, John M. (Ed.) "Designing Interaction: Psychology at the Human-Computer Interface". Cambridge University Press

Norman, Donald A. (1999): Affordances, Conventions, and Design. In Interactions, 6 (3) p. 38-41

Norman, Donald A. (1999): Invisible Computer: Why Good Products Can Fail, the Personal Computer Is So Complex and Information Appliances Are the Solution. London, MIT Press

Norman, Donald A. (2004): Emotional Design: Why We Love (Or Hate) Everyday Things. Basic Books

Norman, Donald A., Shallice, Tim (1980). Attention to action: Willed and automatic control of behaviour. Centre for Human Information Processing (Technical Report #99). Centre for Human Information Processing, University of California, San Diego, CA, USA

Norman, Donald A., Shallice, Tim (1986): Attention to action: Willed and automatic control of behaviour (Revised reprint of Norman and Shallice (1980)). In: Davidson, Richard J., Schwartz, Gary E., Shapiro, David (Eds.) "Consciousness and Self-Regulation: Advances in Research and Theory". Plenum Press

Pawson, Richard, Matthews, Robert (2002): Naked Objects. John Wiley and Sons

Payne, Stephen J. (2003): Users' Mental Models: The Very Ideas. In: Carroll, John M. (Ed.) "HCI Models, Theories, and Frameworks". Morgan Kaufman Publishers

Polanyi, Michael (1966): The Tacit Dimension. Garden City, New York, Doubleday

Pylyshyn, Zenon (1984): Computation and Cognition. Cambridge, MA, USA, MIT Press

Ritchie, Dennis M. (1993): The development of the C language. In: Feldman, Stuart I. (ed.): The second ACM SIGPLAN conference on History of programming languages. April 20 - 23, 1993, 1993, Cambridge, Massachusetts, United States. p.201-208

Robinson, Mike (1993): Design for unanticipated use. In: Michelis, G., Simone, C., Schmidt, Kjeld (ed.): Proceedings of the Third European Conference on Computer Supported Cooperative Work - ECSCW 93. 1993. p.195-210

Rosch, Eleanor (1978): Principles of categorization. In: Rosch, Eleanor, Lloyd, Barbara B. (Eds.) "Cognition and Categorization". Lawrence Erlbaum Associates

Rosch, Eleanor, Lloyd, Barbara B. (Eds.) (1978): Cognition and Categorization. Hillsdale, NJ, Lawrence Erlbaum Associates

Rosenberg, Jarrett, Hill, Ralph, Miller, Jim, Schulert, Andrew, Shewmake, David (1988): UIMSs: Threat or Menace?. In: Soloway, Elliot, Frye, Douglas, Sheppard, Sylvia B. (ed.): Proceedings of the ACM CHI 88 Human Factors in Computing Systems Conference. June 15-19, 1988, Washington, DC, USA. p.197-200

Rossum, Guido van, Drake, Fred L. (2003). Python Reference Manual Release 2.3. Retrieved August 6, 2003 from PythonLabs: http://www.python.org/doc/current/ref/ref.html 

Royce, Winston W. (1970): Managing the Development of Large Software Systems: Concepts and Techniques. In: Technical Papers of Western Electronic Show and Convention (WesCon). August 25-28, 1970, Los Angeles, USA. p.

Rumbaugh, James, Jacobson, Ivar, Booch, Grady (1998): The Unified Modeling Language Reference Manual. Addison-Wesley Publishing

Rutkowski, Chris (1982): An introduction to the Human Applications Standard Computer Interface, Part I: Theory and principles. In Byte, 7 (10) p. 291-310

Sanders, Mark S., McCormick, Ernest J. (1993): Human Factors In Engineering and Design. McGraw-Hill

Schefe, Peter (2000): On some epistemological problems of software engineering. In: Hansen, H. R., Bichler, M., Mahrer, H. (ed.): Proceedings of the 8th European Conference on Information Systems 2000 Conference - ECIS 2000. July 3-5, 2000, Wienna, Austria. p.1-6

Schütz, Alfred (1967): Phenomenology of the Social World. Northwestern University Press

Searle, John R. (1980): Minds, Brains and Programs. In Behavioral and Brain Sciences, 3 p. 417-424

Sebesta, Robert W. (2002): Concepts of Programming Languages (Fifth Edition). Addison-Wesley Publishing

Shannon, Claude E., Weaver, Warren (1963): The Mathematical Theory of Communication. Illinois, USA, University of Illinois Press

Shneiderman, Ben (1982): The Future of Interactive Systems and the Emergence of Direct Manipulation. In Behaviour and Information Technology - BIT, 1 (3) p. 237-256

Shneiderman, Ben (1983): Direct manipulation: A step beyond programming languages. In IEEE Computer, 16 (8) p. 57-69

Shneiderman, Ben (1997): Designing the User Interface: Strategies for Effective Human-Computer Interaction. (3rd ed.). Boston, MA, Addison-Wesley Publishing

Simon, Herbert A. (1981): The Sciences of the Artificial (2nd. ed.). Cambridge, Massachusetts, MIT Press

Simon, Herbert A. (1996): The Sciences of the Artificial, (third ed.). Cambridge, MA, MIT Press

Skou, Kasper (2001): Begreber i menneske og maskine. Master's Thesis. Department of Information and Media Studies, Aarhus University.

Sternberg, Robert J. (1996): Cognitive Psychology. 2nd. Ed.. Harcourt Brace College Publishers

Stigen, Anfinn (1992): Tenkningens historie. Bind I. Oslo, Ad Notam Gyldendal AS

Stroustrup, Bjarne (1986): The C++ Programming Language. Addison-Wesley Publishing

Suchman, Lucy A. (1987): Plans and situated actions: The problem of human-machine communication. New York, Cambridge University Press

Suchman, Lucy A. (2001). Human/Machine Reconsidered. Retrieved September 5, 2003 from Department of Sociology, Lancaster University: http://www.comp.lancs.ac.uk/sociology/soc040ls.html 

Sun Microsystems (2004). JavaTM 2 Platform Standard Edition 5.0 API Specification. Retrieved October 4, 2004 from Sun Microsystems: http://java.sun.com/j2se/1.5.0/docs/api/index.html 

Sun Microsystems, Inc. (1999). Java look and feel design guidelines version 1.0.2. Retrieved August 6, 2003 from Sun Microsystems, Inc.: http://java.sun.com/products/jlf/ed1/dg/higtitle.alt.htm 

Sutherland, Ivan E. (1963): Sketchpad: A man-machine graphical communication system. In: Proceedings of the AFIPS Spring Joint Computer Conference. 1963. p.329-346

Svanæs, Dag (2000): Understanding Interactivity: Steps to a Phenomenology of Human-Computer Interaction. Trondheim, Norway, Norges Teknisk-Naturvitenskapelige Universitet (NTNU)

Tesler, Larry (1983): Object Oriented User Interfaces and Object Oriented Languages. In: ACM Conference on Personal and Small Computers. December 7-9, 1983, Westgate Hotel, San Diego, California, USA. p.3-5

Thomas, William Isaac, Thomas, Dorothy Swaine (1928): The Child in America: Behavior Problems and Programs. New York, Alfred A. Knopf

Turkle, Sherry (1984): The Second Self: Computers and the Human Being. New York, Simon and Schuster

Weber, Maximilian (1930): The Protestant Ethic and the Spirit of Capitalism (translated by Talcott Parson). New York, Charles Scribner's Sons

Webster's Revised Unabridged Dictionary (1913, Version 0.46 in electronic version). Available at Dict.org

Whiteside, John, Wixon, Dennis (1987): Discussion: Improving Human-Computer Interaction - a Quest for Cognitive Science. In: Carroll, John M. (Ed.) "Interfacing Thought". MIT Press

Winograd, Terry, Flores, Fernando (1987): Understanding Computers and Cognition: A New Foundation for Design. Reading MA, Addison-Wesley Publishing

Wittgenstein, Ludwig (1922): Tractatus Logico Philosophicus. London, Routledge and Kegan Paul

Wittgenstein, Ludwig (1953): Philosophische Untersuchungen. Frankfurt am Main, Suhrkamp Verlag

Wordnet (version 1.7.1): A lexical database for the English language. Princeton University. [Accessible online]

Zhang, Jiajie (1991): The interaction of internal and external representations in a problem solving task. In: Proceedings of the Thirteenth Annual Conference of Cognitive Science Society. 1991, Hillsdale, NJ. p.

Zhang, Jiajie (1997): The Nature of External Representations in Problem Solving. In Cognitive Science, 21 (2) p. 179-217

Zhang, Jiajie, Norman, Donald A. (1994): Representations in Distributed Cognitive Tasks. In Cognitive Science, 18 p. 87-122

 

Join our community and advance:

Your
Skills

Your
Network

Your
Career

 
Join our community!