Trying To Be Calm: Ubiquity, Cognitivism, and Embodiment Simon Penny
Trying To Be Calm: Ubiquity, Cognitivism, and Embodiment Simon Penny
Simon Penny
Forthcoming in Throughout.
MIT Press 2010/2011.
Ekman and Hansen, eds.
Introduction
Mark Weiser and John Seely Brown proposed a ʻcalmʼ backgrounded
technology as a reaction to the laborious and foregrounded nature of ʻ90s
computer systems and the techno-fetishism exemplified by mid 90ʼs Virtual
Reality (Weiser and Seely Brown, 1995). This paper traces discursive and
technological transitions between the decade of ʻvirtualityʼ (1990s) and the
decade of ubiquity (2000s). The paper proposes that the notion of virtuality
was in part a product of an incomplete technology. The paper outlines the role
of the cognitivist paradigm in shaping notions of computation and virtuality
through the 90s and draws attention to the increasing importance of
discourses of embodiment in both HCI and media arts since the early 90s. The
key role of media artists in proposing and developing new modalities of
embodied interaction is observed. Two quite different classes of technology
which are often grouped under the rubric of ʻubiquitous computingʼ are
distinguished. It is argued that the ongoing paradigm shift toward embodied
and performative cognitive perspectives is critical to resolving theoretical and
(interaction) design challenges inherent in the development of ubiquitous
technology.
After Virtuality
I propose that discourses of technological ʻvirtualityʼ during the 1990s are
attributable in large part to the vestigal condition of interface technologies
during that decade, a condition which was theoretically supported by the
prevailing cogntivism. In their new roles as interactive multimedia systems,
computers were inadequately supplied with interfaces to the physical world –
the previously ʻnormalʼ roles of computers did not call for such interfaces. This
disjunction between comparative sophistication of computational capabilities
and the relative paucity of interface capabilities led, I would argue, to the
notion of the (computational) virtual, and the confused rhetorics of virtuality. In
hindsight we might say that the 90ʼs furor around the ʻvirtualʼ was symptomatic
of this technological imbalance, that much of the research work and grass-
roots development of the 90ʼs was directed at correcting that imbalance, and
that the current era of ubicomp evidences the effectiveness of that correction.
El bal de Fanalet/ Lightpools. Naric Pares, Roc Pares and Perry Hoberman, 1998. El bal de
Fanalet/ Lightpools combined sonar-based tracking and interactive artificial life-based
graphics with artifacts derived from Catalan popular culture.
The transition from the period of virtuality to the period of ubiquity was a result
of the maturation of interface technologies absent from the technological
palette of the 90s. Since then a variety of technologies linking the dataworld
with the lived physical world: sensing and tracking technologies (such as
MEMS accelerometers, machine vision, laserscanners, GPS, RFID) and
mobile communications technologies have been developed and deployed.
This has had the effect of nesting the ʻvirtualʼ back into the lived physical world,
revealing it to be a panic around an explosive and messy technological
transition period. This belated integration of data with the world caused ʻthe
Virtualʼ to evaporate. The transition from VR to more nuanced augmented and
mixed-reality modes deploying VRʼs stock-in-trade tracking and simulation
techniques indicates that ubiquitous computing is less the kind of antithesis of
VR which Weiser envisaged, and more of a continuity.
Petit Mal - Autonomous Robotic Artwork. Simon Penny, 1993-95. Shown here in the Smile
Machines Exhibition (curator Anne-Marie Duguet), Transmediale 2006, Berlin.
At the same time that human computer interaction moved out beyond the
research lab, human interaction with the world and with technology was
addressed more intensively – as is evidenced by the rapid expansion of HCI,
CSCW and related areas of research. HCI became increasingly
interdisciplinary as psychologists, anthropologists and sociologists became
involved. As recognition of the shortcomings of the cognitivist paradigm
became more widespread, new modes of cognitive science grappled with the
embodied, enactive, situated and social dimensions of cognition (Varela,
Thompson and Rosch, Suchman, Hutchins). Neuroscience research revealed
new dimensions of the mind-body relation (Edelman, Ramachanran, Sacks,
etc). Conventional philosophy of mind has been challenged on these bases by
Lakoff and Johnson, Clark, Thompson and others. This movement met media
artists coming the other way, as it were – exploring the application of
computational technologies to embodied, material and situated cultural
practices. The crafting of embodied, sensorial experience is a fundamental
expertise of the arts, an expertise which is as old as human culture itself.
Various topics of critical discourse which had been lumped-in with discussion
of the virtual have persisted, and in particular, it has become clear that many of
the aesthetic projects of ʻmedia-artistsʼ are inherently concerned with the
central issues of ubiquity.
Ubiquity : figure and ground
Mark Weiser, John Seely Brown and others made clear their motivations for a
ʻcalm technologyʼ that recedes from attention, but the term ʻUbiquitous
computingʼ is applied to two quite different types of technology. One is
industrial and embedded, effectively invisible and accessed by experts. The
other is consumer commodity, very visible and demanding of attention, while
nonetheless affording sophisticated data-gathering available to paying
customers. Although the two categories have much in common
technologically, they are very different in their relation to the social.
In consumer goods, the obsession with the interface does not seem to have
abated, the ecstasy of computation - if not the ecstasy of communication -
seems to have become a fixture of popular culture. While miniaturisation and
wireless networking have indeed moved ʻout into the physical worldʼ, it has not
resulted in ʻrepositioning it in the environmental backgroundʼ (Ekman, this
volume). Rather, the miniaturised but intensified interface, attention
demanding and insistent, is foregrounded. While the technological
infrastructure (cell phone reception, etc) has indeed become ubiquitous, on
the level of human experience, many technologies reinforce a discontinuity
between the dataworld and the physical world.
Mobile wireless technology has certainly become ubiquitous, but perhaps not
in the way Weiser hoped. These words ʻubiquitousʼ, ʻpervasiveʼ, ʻembeddedʼ
all have an ominous ring, they carry negative connotations of an oppressive
informational monoculture or monopolistic order, perhaps because of their
deployment in military jargon. While the technical modalities of the technology
are novel, the purposes to which they are put retain functions of surveillance
and control. It is not just a question of to what ends the technology is deployed
and whom it is working for or against, but of to whom the systems are visible
and to whom they are invisible.
Why did the computer, which once was a basement sized machine staffed by
attendants, morph into a desktop machine? The historical answer is that it was
applied the kinds of tasks which people who sit at desks do when sitting at
desks. Functionally, the desktop computer was an enhanced typewriter and
calculator with added filing-cabinet functionality. It follows then that it is
particularly useful and relevant for activities which resemble office desk
activities, such as record management, accountancy and letter writing, and is
decreasingly appropriate for activities whose social and architectural
placement diverges from that scenario. Many human activities, including
cultural and artmaking activities, do not resemble office work in their physical
contexts, methodologies or goals.
For the last generation, we have managed with computer technology which,
for all its touted user-friendliness, has continued to demand that we
preprocess our thoughts and experiences into a kind of keystroke mush which
is easily amenable to the limited a-d capability of these machines. If we are to
pursue the fundamental goals of Weiserʼs ubiquity, it means developing
computational technology past the stage that we and it appear to have got co-
dependently stuck in - tolerating a technology which must be spoon-fed with
little alphanumeric streams. Mercifully, after thirty years of personal computing
I no longer have to always position myself in work-position at my work-station,
from which I cannot move even a few feet without breaking my connection with
the machine by losing contact with screen and keyboard. But why, having
finally freed ourselves from the bondage of the desktop, do we tolerate having
to poke unidigitally at a miniature QWERTY on our mobile devices? What a
profound failure of imagination!
Trying to Be Calm
There is a significant difference between enhancing the control systems of
existing machine complexes and the enmeshing of computational processes
with human cultural and biological processes. Iʼve distinguished between, on
the one hand, clandestine, faceless technologies which involve distributed
units in a larger control array which itself is embedded in a larger machine
complex; and on the other hand, garrulous, clingy technologies close to the
body. Neither of these seems particularly calm. Beyond embedded
miniaturization (microcontrollers), location (tracking) and transmission (internet
and wireless communication), how far have we come along the trajectory to
calmness? Is automated processing of logical operations is necessarily
applicable and an asset in every aspect of life? Are there aspects of our lives
where digital intrusion might be utterly undesirable? (Do I need ʻblueteethʼ that
notify my dentist directly when they sense decay? Probably not. I certainly
donʼt feel the need for pop-up ads on the periphery of my vision when Iʼm
wearing my sunglasses.) To ask this question is to challenge the marketing
rhetoric of the computer industry, to challenge the assumption of the
desirability of the intrusion of computation everywhere: that automated
processing of logical operations is necessarily applicable and an asset in
every aspect of life. Computation is not value-free cognitive bedrock. There is
nothing ʻneutralʼ about the culture of computation, even if we are naturalized to
it.
In such contexts the application of digital technologies almost always has the
effect of ʻthinning outʼ the experience in question, and this is due in part to a
preoccupation with problem-solving on the symbolic plane and the ensuing
elision of the situated, embodied action. This syndrome maps onto imperatives
of computer engineering – modularity/reductivism, standardization/generality,
optimality/efficiency – instrumentality generally. These criteria are valid in their
ʻhome territoryʼ – I want my laptop battery to have maximum life, I want my file
to be compatible, I do not want anyone taking aesthetic liberties with the shape
of an airplane wing. But the validity of these criteria wanes as they are applied
In what way and for whom did Cognition in the Wild ʻexplainʼ the procedures of
coastal navigation, or to put it another way: what is the power of the
computational explanation? An unreconstructed computational explanation
would necessarily explain observed phemonema in functionalist terms
(Putnam 1967- since recanted). Functionalism asserts that a mental state is
constituted by the causal relations that it bears to sensory inputs, behavioral
outputs and other mental states. Cognitivism is just one (computational)
version of functionalism. Functionalism has a rather industrial if not von
Neumannesque cast in its reliance on the idea of serial processing, inputs and
outputs. The cognitivism of Cognition in the Wild is more nuanced. Cognition,
for Hutchins, is embedded in artifacts and practices and shared among actors
– but it is still understood as computation. As cognitive science reaches out
further and further into cultural realms where computation is an increasingly
alien concept, distinctions between technical and popular usages become
increasingly hazy, the imperializing project of computer culture insidiously
persists.
Hutchinsʼ recognition that “Interactions between the body and cultural artifacts
constitute an important form of thinking. These interactions are not taken as
ʻindicationsʼ of invisible mental processes, rather they are taken as the thinking
processes themselves” (Hutchins, 2006) are reminiscent of remarks made by
Hubert Dreyfus many years earlier in his phenomenological critique of AI: “My
personal plans and my memories are inscribed in the things around me just as
are the public goals of men in general.” (Dreyfus, 1992, 266) More recently
John Sutton has similarly noted that “…thought is not an inner realm behind
practical skill, but itself an intrinsic and worldly aspect of real-time engagement
with the tricky material and social world.” (Sutton 2008, 50) To permit that
bodily motion may constitute the medium of thinking itself is a radical assertion
for a rehabilitated cognitivist, but will come as no surprise to the dancer or
practitioner of martial arts, nor to any thoughtful person while rock climbing or
hanging out the laundry. But we must not underestimate the profundity of this
sea-change in cognitive science, it indicates a hard-won emancipation from
naturalization to the tenets of AI. Philip Agre lucidly documents his won such
emanciption. He credits his reading of Foucaultʼs The Archeology of
Knowledge specifically and poststructural writing generally as an epiphany:
“…they were utterly practical instruments by which I first became able to think
clearly and to comprehend ideas that had not been hollowed out through the
false precision of formalism.” (Agre 1997, “Towards a Critical Technical
Practice.”).
Hutchins comes close to the work of Mark Johnson (1987) and also Lakoff and
Johnson (1999) regarding the origins of abstract concepts in embodied
experience when he notes: “Motion in space acquires conceptual meaning
and reasoning can be performed by moving the body.” (Hutchins, 2006) Here
is revealed a fundamental cognitive cauterisation amongst all but the most
sensitively designed interfaces and interactive systems – a situation which has
beleaguered digital arts practices: they ignore and erase bodily engagement
of the sort that complement material artefacts and tools developed over years
or generations and which, taken together, facilitate bodily reasoning. The
navigators hoey, the engineers slide rule, the machinists caliper, the
carpenters square, are amenable to computational explanation, because
(loosely) what is involved is a relatively simple translation of geometry to
algebra. The painters brush, the violinists bow, the harvesters scythe, and so
many other artefacts, are complex and sophisticated devices for thinking with
because they have evolved in a deep structural coupling with the basic
rhythms and modalities of neural circuits and sensori-motor loops. They are
prosthetics which integrate with the user at a deep and more organic level
precisely because they do not involve a translation into and out of
mathematico-logical computation. On the subject of artifacts, Hutchins notes:
“By interacting with particular kinds of cultural things, we can produce complex
cognitive accomplishments while employing simple cognitive
processes.” (Hutchins 2006) Aspects of the environment are deployed as off-
board memory, and consistent with Hutchinsʼ notion of distributed cognition,
computation is offloaded too.
But are we not, in framing the situation in this way, reinstating precisely the
computationalist bifurcations we sought to avoid? Not simply of storage and
processing, but of the world and representation? Lambros Malafouris asserts
that it makes little sense to speak of one system representing the other:
“Although we may be well able to construct a mental representation of
anything in the world, the efficacy of material culture in the cognitive system
lies primarily in the fact that it makes it possible for the mind to operate without
having to do so, ie, to think through things, in action, without the need of
mental representation.” (Malafouris, 2004, 58) Micronesian canoeists gather
knowledge about undersea geography, colloquially ʻthrough the seat of their
pantsʼ (if theyʼre wearing any), but more accurately through a subtle integration
of proprioceptive and vestibular cues related to the movement of their craft
(canoe, catamaran) as a prosthetic extension of their embodiment. Hutchins
goes on rightly to observe: “From the perspective of formal representation of
the task, the means by which the tools are manipulated by the body appear as
mere implementation details.” (Hutchins, 2006)
The phrase “implementation details” tells the score before the game begins. It
belies a commitment to dualism that will automatically render invisible or
irrelevant aspects of embodiment. Explanation of a group human activity in
terms of computation will inevitably render invisible the significance of
embodied practice because the irrelevance of embodiment is axiomatic to the
rationale of the discipline. “Implementation details” is a phrase which stands in
for an entire corpus of disciplinary rationalizations to justify the disembodiment
of AI, as first articulated by Herbert Simon: “Instead of trying to consider the
ʻwhole manʼ, fully equipped with glands and viscera, I should like to limit my
discussion to Homo Sapiens, ʻthinking man.ʼ” (Simon 1969, 65) This arbitrary
and convenient ʻlimitʼ in the ʻroot documentʼ of cogntivism is a veritable
Pandoras box, which permitted the excision of embodied situated materiality
from AI and cognitive science for a generation. The devil is not so much in the
(implementation) details as in the desire to ignore them. “Implementation
details” cannot be swept under the rug. Like ʻhuman factorsʼ, the term has
allowed technical community to sidestep the overarching importance of human
culture – engagement of which would of course demand a challenging
interdisciplinarity which always has the awkward potential of destabilizing
axiomatic assumptions.
Conclusion.
Two decades ago, at the emergence of the ʻreactive roboticsʼ movement,
Rodney Brooks critiqued the reigning representationalism in his pithy
assertion that: “the world is its own best model,” (Brooks 1991, 15) a sentiment
which was sympathetic to emerging paradigms of embodied, situated and
distributed cognition, and also with Hubert Dreyfusʼ phenomenological critique
of AI. By virtue of evolutionary selection, there is direct cognitive correlation
between the world and the bodily experience of it. This results in a kind of
(performative) knowledge and (non-)cogitation irreconcilable with the
cogntivist ʻphysical symbol system hypothesis.ʼ But it is this embodied, situated
knowledge which provides the basis for precisely such cogitation, and for
introspection. This is the lived solution to the symbol grounding problem.
(Harnad) This double - that the world is its own best model, and that there is
direct (non)cognitive correlation between the world and the bodily experience
of it - is the core of the post-cognitivist position. It is a true paradigm shift, which
must be thoroughly internalized if real progress is to be made in the
development of ʻcalmʼ technology.
Works Cited
Agre, Philip. “Towards a Critical Technical Practice: Lessons Learned in Trying
to Reform AI.” Bowker, Geoffrey C. Social Science, Technical Systems,
and Cooperative Work : Beyond the Great Divide. Mahwah, N.J.:
Lawrence Erlbaum, 1997. 131-157.
Agre, Philip. “Writing and Representation.” Mateas, Michael, and Phoebe
Sengers, eds. Narrative Intelligence. Amsterdam: J. Benjamins, 2003.
281-303.
Bechtel, William. Philosophy and the Neurosciences : A Reader. Malden,
Mass.: Blackwell, 2001.
Beuys, Joseph. How to explain pictures to a dead hare. Performance, Galerie
Alfred Schmela, in Düsseldorf. 1965.
Brooks, Rodney. "Elephants Don't Play Chess." Robotics and Autonomous
Systems 6 (1990): 3–15.
-- “Intelligence Without Reason”. Massachusetts Institute of Technology.
Artificial Intelligence Laboratory. A.I. Memo No. 1293 April, 1991.
-- “A Robust Layered Control System for a Mobile Robot” Massachusetts
Institute of Technology. Artificial Intelligence Laboratory. A.I. Memo No.
864. September 1985
Clark, Andy and David Chalmers. “Extended Mind Hypothesis.” Analysis 58
(1998): 10-23.
Donald, Merlin. Origins of the Modern Mind : Three Stages in the Evolution of
Culture and Cognition. Cambridge, Mass.: Harvard University Press,
1991.
Dreyfus, Hubert L. What Computers Still Can't Do : A Critique of Artificial
Reason. Cambridge, Mass. MIT Press, 1992.
Ekman, Ulrik. “Introduction.” This volume.
Foerster, Heinz von. Understanding understanding, Springer-Verlag, 2002.
Harnad, Stevan. “The Symbol Grounding Problem.” Physica D 42 (1990):
335-346.
Hollan, James, Edwin Hutchins and David Kirsh. Distributed Cogntion:
towards a new foundation for Human-Computer Interaction Research.
ACM Transactions on Computer-Human Interaction, Vol.7, No.2,
June2000, 174-196
Hutchins, Edwin. Cognition in the Wild. Cambridge, Mass.: MIT Press, 1995.
---. Imagining the Cognitive Life of Things. 2006. Lambros Malafouris, and
Colin Renfrew (eds)
Cognitive Life of Things: Recasting the Boundaries of the Mind
McDonald Institute for Archaeological Research (Forthcoming 2010)
Available: http://www.ida.liu.se/~729G12/mtrl/
ImaginingCogLifeThings.pdf. 14.08.2009.
Huxley, Aldous. The Doors of Perception. New York: Harper, 1954.
Johnson, Mark. The Body in the Mind. Chicago: University of Chicago Press,
1990.
Johnson, Mark, and George Lakoff. Philosophy in the Flesh. New York: Basic
Books, 1999.
Kirsh, D. and P. Maglio. On Distinguishing Epistemic from Pragmatic Actions.
Cognitive Science. 18, (1995). 513-549
Malafouris, Lambros. “The Cognitive Basis of Material Engagement: Where
Brain, Body and Culture Conflate.” DeMarrais, Elizabeth, et al., eds.
Rethinking Materiality : The Engagement of Mind with the Material
World. Cambridge: McDonald Institute for Archaeological Research,
2004. 53-62.
Maturana, Humberto. "Autopoiesis: the organization of the living," in Maturana,
H. R., and Varela, F. G., Autopoiesis and Cognition. Reidel.
Netherlands, 1980.
Mindell, David A. Between Human and Machine : Feedback, Control, and
Computing before Cybernetics. Baltimore: The Johns Hopkins
University Press, 2002.
Newell, Allen; and Herbert A. Simon. "Computer Science as Empirical Inquiry:
Symbols and Search." Communications of the ACM 19.3 (1976):
113-126.
Penny, Simon. "Experience and Abstraction - the Arts and the Logic of the
Machine." FibreCulture 2008. Available: http://journal.fibreculture.org/
issue11/issue11_penny.html 14.08.2009. 11.
-- “Bridging Two Cultures – Towards a History of the Artist-Inventor.” Dieter
Daniels and Barbara U. Schmidt, eds. Artists as Inventors, Inventors as
Artists. Ostfildern: Hatje Cantz, 2008. 142-157.
-- “Rigorous Interdisciplinary Pedagogy.” Convergence 15.1 (2009): 31-54.
-- “Desire for Virtual Space: the Technological Imaginary in 90s Media Art”.
Forthcoming in Space and Desire, ed, Thea Brezek, ZHDK, Zurich
2010.
Pickering, Andrew. The Mangle of Practice : Time, Agency, and Science.
Chicago: University of Chicago Press, 1995.
-- “Against Human Exceptionalism,” unpublished paper, University of Exeter,
2008.