- Personal Homepage
Roger F. Malina is an art-science researcher, astronomer and editor. He is a Distinguished Professor of Arts and Technology and Professor of Physics at the University of Texas, Dallas where he is developing an Art-Science R and D and Experimental Publishing program. He is a Directeur de Recherche of the CNRS and former Director of the Observatoire Astronomique de Marseille Provence at Aix-Marseille University. His scientific specialty is in space instrumentation and big data problems; he was the Principal Investigator for the NASA Extreme Ultraviolet Explorer Satellite at the University of California, Berkeley. He also has been involved for 25 years with the Leonardo organization whose mission is to promote and make visible work that explores the interaction of the arts and sciences and the arts and new technologies. Since 1982 he has been the Executive Editor of the Leonardo Publications at MIT Press. More recently he has helped set up the Mediterranean Institute for Advanced Studies (IMERA) and is co chair of the ASIL (Arts, Sciences, Instrumentation and Language) Initiative of IMERA which hosts artists in residence in scientific research laboratories of the Marseille region.
- Publication period start
- Publication period end
- Number of co-authors
Number of publications with favourite co-authors
Most productive colleagues in number of publications
26.17 Commentary by Roger Malina
Paul Fishwick has been formulating for a number of years his concept of "Aesthetic Computing", broadly defined as the application of the theory and practice of aesthetics to computing; in the process an "embodied" formal language is advocated. In my view this approach becomes particularly pertinent if, in the process, the methods and content of computing as science and engineering is changed and enhanced. I have called this kind of goal the ‘strong case’ for art-science interaction  where the interaction goes beyond the demonstrative or pedagogical. It is perhaps no accident that at the Dagstuhl workshop a ‘manifesto’ was issued  as there was a general feeling by the participants that the possible contributions of the arts, design and humanities to computer science were not generally accepted outside a group of enthusiasts.
A number of developments, some of which are referred to by Fishwick are mentioned, but here I would like to address a number of issues which are included in the concept of aesthetic computing but go beyond it. Fishwick talks of aesthetic computing addressing the different elements of formal languages which are number, data, model and software.
Here it is perhaps useful to add Denning’s  seven principles of computing; these have the advantage of being process oriented and helps focus areas of possible art and design intervention:
- Computation: What can and cannot be computed
- Communication: Reliably moving information between places
- Coordination: Effectively using many computers
- Recollection: Representing, storing and retrieving information from media
- Automation: Discovering algorithms for information processes
- Evaluation: Predicting performance of complex systems
- Design: Structuring software systems for reliability
This significantly broadens the landscape of possible intervention of aesthetic computing approaches, and indeed many of these areas have been barely engaged by the arts and humanities to date.
26.17.1 Big Data Transition and the Crisis of Representation.
A major development in the last decade is sometimes referred to as the "big data" transition . As data volumes and rates continue to grow at accelerating rates scientific disciplines go through transformational changes. Astronomy was perhaps the first discipline to make this transition with the emergence of virtual observatory strategies on both data archives and software. Fields such as genomics soon followed and now all areas of business and culture are impacted (see for instance ). This has led to what might be called a "crisis of representation" and the emergence of new disciplines such as infoviz and dataviz. It has rapidly become apparent that the problem is no longer one of ‘communication’ of the content of data via illustration techniques (e.g. ) but rather the problem becomes one of immersion in data which can no longer be thought of as ‘objects’ but rather as a "fluid"; hence strategies of reification referred to by Fishwick are proving inadequate as we enter media territory. A number of researchers have been seeking to expand the reach of ‘image science’ into this new territory that requires semiotic approaches not yet developed. Most data is never analysed or viewed, and new kinds of "technologies of attention" are required to help navigate and isolate data that has particular content or meaning. This crisis of representation is a good area for the arts and humanities to be involved and will be a long term agenda for aesthetic computing; and as emphasised by Fishwick the key issue is embodiment or how data is put in forms that are apprehensible via the human senses. A number of artists have been prominent in exploratory projects for navigating through data such as Donna Cox  and Ruth West . There have been a number of exhibitions that have sought to display the variety of approaches (e.g. siggraph information aesthetics ). It seems to me that Fishwick pays insufficient attention to these developments which are rapidly leading to new research areas.
26.17.2 Artificial Life Art, Visual Mathematics and Embodied Code
Paul Fishwick points out briefly the area of serious gaming as one area where code can be seen as being embodied. It seems to me there is a far larger area of algorithmic art, visual mathematics, artificial life art which have much stronger examples of embodiment of code. The development of algorithmic art by computer art pioneers such as Michael Noll, Roman Verostko, Harold Cohen to name just a few has already established a 50 year history of making algorithms apprehensible by human vision and hearing using aesthetic means and objectives. The area of visual mathematics (see Michele Emmer and his two Leonardo books on the visual mind ) has already provided success stories of how aesthetic methods have led to scientific, or mathematical, discoveries. Following the Santa Fe Institute workshops that established visibly the bases of artificial life , and its more recent applications to synthetic biology, the arts community rapidly picked up the challenge with the development of artificial-life art projects that has led to a proliferation of projects in robotics, virtual worlds, interactive installations and other ways of bringing code into physical contact with the human senses. Recently Leonardo Journal published  a selection of works as part of the 10th anniversary of the VIDA artificial life competition, the pre-eminent forum for artificial life art practitioners.
26.17.3 Translation as a possible method for Aesthetic Computing
In section 26.10 Fishwick articulates the elements of a method for aesthetic computing. In recent discussions with colleagues Rainer Schulte and Frank Dufour at the University of Texas, Dallas , I have been impressed by their work that seeks to apply the humanities based methods of translation studies to the problems posed by computer generated or mediated forms of creative "writing" (whether text, image, sound or indeed multi-medial and multi sensorial). There are issues of media "essentialism’ as pointed out by Fishwick in his section 26.2.4 on Media, where each medium has specificities that allows or prevents certain concepts to be translated from one medium to another. But in addition - as pointed out by Fishwick - the limitations of technology may constrain certain form of embodiment. He gives the example of the experience of early word processing software where the writer had to ‘stop and wait’ for the microprocessor to catch up; similarly now certain microprocessor steps occur at faster rates than the cycle times of the human cortex, so that the act of translation from code to embodied perception requires the slowing down and time stretching of phenomena. The act of translation from the culture of the "formal languages of computing" to the "formal languages of the arts and literature", or vice versa, requires methodologies from the humanities in order to create meaningful embodiment strategies for aesthetic computing.
26.17.4 Intimate Science
Elsewhere I have written  of the general problem of how to enable cultural appropriation of scientific phenomena inaccessible to the human senses. I have called this the agenda of ‘intimate science’ that many artists are now involved in. As pointed out by philosophers many of the ways that we conceive of the world are built from our experiences from birth onwards. Our ideas of causality, or more generally of explanatory systems, are fed by our interactions with the world via our bodies and senses. But science now deals with many phenomena that are not only beyond the "amplification" or "augmentation" of our senses but are inherently ‘non-commensurate’ with the way our senses operate. This problem perhaps emerged most clearly in the case of quantum mechanics where our basic ontologies are no longer applicable (objects can be both wave and particle) and concepts of causality (in the case of entanglement) totally foreign to our experience in the macro world. I would argue that we run into similar issues in the concept of emergence in complexity science. Certainly we run into similar issues in making sense of general relativity and the presence of distortions in space-time and the structure of space itself; there is no way to experience gravity waves as a human being. It seems to me that Fishwick’s agenda for aesthetic computing in a way transposes the problem of making science intimate into the field of computer science. Computers have different internal logics than human bodies, different teleologies, and only through the mechanisms of embodiment that he discusses can we begin to ‘imagine’ the way that a computer "imagines", The exercise of transposing ‘models’ into ‘maquettes’ is one such process of cultural appropriation.
26.17.5 STEM to STEAM
During the last two years the U.S National Science Foundation, in partnership with the U.S National Endowment for the Arts, have been organizing a number of workshops that bring together the research communities in Science and Engineering and the creative communities in Arts and Design . These initiatives were responding to the sociological fact that there is a growing body of research practice that bridges Science and Engineering to Art and Design; in some cases School of Arts and Design find themselves engaged very similar research agendas to Science or Engineering departments with of course different, or overlapping, outcomes in mind. Sometimes unusual trans-discliplinary collaborations are involved, in other cases artists and designers find themselves in the role of inventors and technological innovators. This development was first recognized in information technology. The "Mitchell" report "Beyond Productivity"  in 2003 laid out the problems and opportunities. Similarly developments are now occurring in other areas of science and engineering. As a result of these workshops the NSF has put in place two research contracts to stimulate network development via the SEAD initiative , and a trans disciplinary documentation platform via the XSEAD contract . An early outcome of these workshops was the acceleration of the concept of ‘turning STEM into STEAM’. After thirty more years of national efforts in the U.S to develop the STEM workforce pipeline, the U.S is faced with shortage of trained scientists and engineers. As articulated by the STEM to STEAM movement (see for instance the congressional testimony organized by John Maeda, President of the Rhode island School of Design ) we need to integrate the Arts and Design, or more generally the Arts and Humanities, into Science Technology Engineering and Mathematics education and research strategies. Some of the fastest growing computer science related programs are in the computer arts, gaming, social media; and as pointed out by theorists in innovation studies, the process of social and cultural innovation is playing a stronger and stronger role in successful adoption of disruptive technologies. Expertise in visualization and image science areas that are grounded in the arts and humanities are promising areas that emerge from the aesthetic computing agenda that is outlined by Fishwick. As pointed out by Fishwick our institutions of higher education are badly organized to address these research agendas; the program that Fishwick has been developing at the University of Florida is one exemplar of possible approaches.
Fishwick addresses strategies of aesthetic computing for the formal language construct which are number, data, model and software. If we add Denning’s seven principles of computing namely Computation, Communication, Coordination, Recollection, Automation, Evaluation, and Design it is clear that aesthetic computing is part of a larger ensemble of arts and humanities research strategies that offer the opportunity of making major contributions to computer science in the coming decades. As I write these comments, there is a large online discussion on "The New Aesthetics", a discussion that credits its source as James Bridle’s blog "The New Aesthetics"  in May 2011. With a starting point that computing is now culturally integrated into our way of being in the world, the discussion (see for instance Ian Bogost , for a rebuttal) has been lively - indicating that we are only at the beginning of aesthetic computing.
- Malina, Roger (2011), The Strong Case for Art-Science Interaction, Retrieved from http://vectors.usc.edu/thoughtmesh/publish/120.php
- Paul Fishwick et al., 2003, 'Aesthetic Computing Manifesto'. Leonardo, 36, Issue No 4,
- Peter Denning, http://cs.gmu.edu/cne/pjd/GP/GP-site/welcome.html
- The fourth paradigm, Data Intensive Scientific Discovery, EDITED BY Tony Hey, Stewart Tansley,and Kristin Tolle, 2009 Microsoft Corporation, ISBN 978-0-9825442-0-4.
- Edward Tufte, http://www.edwardtufte.com/tufte/
- Donna Cox, http://www.ncsa.illinois.edu/~cox/
- Ruth West, http://www.atlasinsilico.net/gallery.html
- 2003 Siggraph Information Aesthetics Show case, http://www.siggraph.org/s2009/galleries_experiences/information_aesthetics/
- Michele Emmer, The Visual Mind, Leonardo Books, 1993, http://leonardo.info/isast/leobooks/books/emmer.html
- Christopher G Langton (1998). Artificial life: an overview. MIT Press.ISBN 0262621126 ,
- Leonardo Journal Vida Gallery, http://www.leonardo.info/isast/journal/toc411.html
- Beyond Productivity : http://www.nap.edu/openbook.php?record_id=10671&page=235
- James Bridle, http://new-aesthetic.tumblr.com/
- Ian Bogost, http://www.wired.com/beyond_the_beyond/2012/04/ian-bogost-the-new-aesthetic-needs-to-get-weirder/
Malina, Roger (2011). The Strong Case for Art-Science Interaction. Retrieved 2012-03-19 00:00:00 from http://vectors.usc.edu/thoughtmesh/publish/120.php