Considering I gave my presentation on Mark B.N. Hansen’s Bodies in Code on February 2 (almost two weeks ago), this post has been a long-time coming. What follows is a collection of notes that I compiled from chapter 2 of Bodies in Code—a chapter that covered the topic of virtual reality.

Direct citations from Hansen are in block-quotes and italicized.

“VR is a literal enactment of Cartesian ontology, cocooning a person as an isolated subject within a field of sensations and claiming that everything is there, presented to the subject.” – Richard Coyne (107)

“The possession of a body in space, itself part of the space to be apprehended, and that body capable of self-motion in counterplay with other bodies, is the precondition for a vision of the world.” – Hans Jonas (107)

The previous lines by Coyne and Jonas are the two epigrams that open chapter 2 of Bodies in Code. Each offer a different perspective on virtual reality and what it means to be “in” a virtual environment (VE).

Virtual Reality Bubble

An elaborate virtual reality system

The first quotation by Coyne speaks of Cartesian ontology, which hearkens back to Descartes’ famous line “I think, therefore I am.” Obviously, while a user may “leave” the real world when entering a VE, he takes his mind with him, and continues to generate thoughts while in the VE. And, because his thoughts are directed towards what he sees or experiences in the VE, perhaps forgetting the “real world,” he then becomes an embodied within the VE. At least, the user believes or imagines that he is inside the virtual environment. He will not truly become a part of the virtual environment until he can experience all  the senses he gets in reality, and with today’s technology, that isn’t possible.

Hans Jonas, on the other hand, doesn’t think that thinking and imagining oneself in a virtual environment is enough to really “be” a part of it. After all, Jonas’ essay “The Nobility of Vision” provides the inspiration for much of Hansen’s theories on virtual reality, and the title for this section of notes (but I will get to that later on). Jonas argues that one needs to be able to interact with her environment and other users within it to truly become embodied within that environment. But if one is able to simply move through a perceived, virtual space with somewhat natural kinetic motion and interact with others, would that be enough to generate a genuine feeling of reality (which I consider to be true virtual reality)? Let me put it this way: if you could play Second Life while moving around in a contraption like the one shown above-right and have other moves picked up by something like Xbox’s Kinect system, would that be enough to generate a genuine feeling of reality virtually? I doubt it, but perhaps we’re getting close.

J.J. “Gibson held that all the information needed for perception… is present in the structure of the light as it is reflected from objects and events in space. The objects and events give the light its specific organization as it reached the eye.” – Jeremy Campbell (114)

“The rhetoric associated with virtual reality clearly conveys its privileging of the visual register of perception.” – Hansen (113)

“The challenge is to make that world look real… the long-term developmental goal of the technology is nothing short of an attempt… to fool eye and mind into seeing worlds that are not and never can be.” – Frank Biocca (113)

I don’t think that the average person gives the theories of J.J. Gibson much credit today, despite the fact that Hansen describes him to be a (if not the) father of VR theory. In today’s age of 3D movies, big screen TV’s, amazing CGI animation and powers of PhotoShop, we’ve seen some pretty high-quality representations of life in a virtual environment. Yet I don’t think we’re any closer to having visual displays (or “screens,” as Sutherland calls them) that truly represent reality, or what it would be like to physically move oneself through or into different spaces, or what it would be like to be something else, like a bird or a fish.

However, when I pull out Ace Combat 6 (a highly realistic flight simulator) to play with some friends on the Xbox, occasionally someone complains of dizziness that they experience physically, outside of the virtual realm. Is this a noteworthy connection made between the virtual and the real?

The Ames Room

A image showing the optical illusion of the Ames Room

Finally, Frank Biocca’s comment reminds me of the key deficiency in Gibson’s theories. While Gibson argues that the visual can represent reality through accurate and realistic visual cues, the visual can also throw users off from what is real—it can “fool eye and mind into seeing worlds that are not and never can be.” Take, for instance, the famous Ames Room (right). It is a simple optical illusion that has bee replicated in various forms. Is this virtual reality? Or is it just a trick? Are we spatially confused or mildly amused? I can’t really speak to the full effect of the Ames Room, since I’ve never seen (or, rather, experienced) it in person. But I think that such visual tricks show how warped realities
can be when we are limited to sensing the world around us with our vision alone.

Gibson’s theory is “an unthematized, and I think, wholly implausible, hope that vision can by itself reconstitute the richness of human perceptual function.” – Hansen (116)

“Sight, in addition to furnishing the analogues for the intellectual upperstructure, has tended to serve as the model of perception in general and thus as the measure of the other senses. But it is in fact a very special case. It is incomplete by itself; it requires the complement of other senses and functions for its cognitive office; its highest virtues are also its essential insufficiencies.” – Hans Jonas (119)

“For [Wann and Rushton], what is crucial is not simulating a visually ‘realistic’ environment in purely visual terms, but rather designing an environment capable of inducing a compelling sensorimotor correlation in the participant.”
– Hansen (117)

“… virtual experience is gained not by simulating visual images but by stimulating tactile, proprioceptive[1], and kinesthetic[2] sensations.” – Hansen (120)

“… if we include in our understanding of touch the ‘fact of its being an activity involving motion,’ we add the complement of action to its ‘receptivity,’ thus elevating it, as it were, into a ‘spatial organizer’ of the different sense species, ‘the synthesizer of the several senses toward one common objectivity.”
– Hansen, with help from Jonas (121)

It is clear that, as I suggested earlier, it is not quite good enough for people to think of virtual reality as a visual simulation (no matter how convincing that simulation is). Jonas suggests that our overall sense of reality is not complete with vision alone, which is something I agree with entirely. In order to realize how important our other senses are, imagine life without vision. Everyone has heard that the blind are supposed to have their other senses “heightened,” but is that a physical change experienced by the hard-of-sight, or is it just that they are more focused to their other senses? There is a reason people tell you to “close your eyes” when they want you to picture or imagine something, and why O Noir, a restaurant in Montreal, offers the unique experience of dining in pitch black, so you are able to better take in the tastes and smells of what it is you’re eating.

Wann and Rushton (and Hansen, it would seem) think that if we are to knock vision out of its position of sensory “nobility,” we should focus on “proprioceptive, and kinesthetic sensations.” That is, the “feeling” of moving through the virtual environment from within our bodies. I tend to agree with this statement. Some of the most “real” thrills we can experience are those where we feel, through proprioception or simple touch, sensations that instill specific thoughts or emotions like fear, surprise, contentment, etc. For instance, we are able to simulate the sensation of falling with things like roller coasters and bungie cords. We can warm ourselves with Rub-A535, and go numb by sitting on our hand. If virtual reality could make this happen instantaneously and at will, it would be an extremely powerful experience where the divide between reality and virtual would be very hard to distinguish while immersed inside a virtual environment with sights and sounds as well. Without these sights and sounds, however, the feelings generated by the VR machine(s) would feel too machinic, and the connection between body and space would break.

One final thought I have on this matter is in regards to sound, which I think Hansen has for the most part overlooked. One of the most effective advancements in virtual reality technology (in my opinion) recently has been the development of surround-sound, which envelops the user in an environment wherever she may be. The horror movie genre seems to have made the most of such technology, by having subtle creeks and scuffles made behind you as you watch the screen in front, anticipating something terrible. While you can’t take your eyes off of the screen (because it’s a television and the virtual environment is framed within it), the sounds behind you give you a paranoid feeling that something from within the virtual environment has made its way out and into reality.

“… the ultimate interface would be the human body and human senses”
– Myron Krueger (117)

“The simulation of the hand’s presence in the VE via the data glove ‘allows users to manipulate virtual objects’ in a way that draws upon ‘Gibson’s belief that we grab on to our world and make it part of our ‘direct’ experience.”
– Hansen (117-8)

“The most crucial factor in creating high telepresence is, perhaps, high correlation between (1) the movements of the operator sensed directly via the internal proprioceptive/kinesthetic senses of the operator and (2) the actions and the displays in the teleoperator station.” – Hansen (117)

So far I’ve only been talking about the “feeling” one could potentially get from virtual reality. While proprioception and kinesthetics involve movement of the user, they refer to the senses of that movement, and not the physical locomotion of the user. For while the feeling of movement through a virtual environment is realistic and somewhat eerie, Krueger and Hansen imply that it is interaction with the virtual environment that would really make for a virtual reality, and not just a virtual simulation (or even stimulation, as Hansen suggested earlier).

Take, for example, the Xbox Kinect robot (below). On one screen you can see that the user is using his body as the interface to move the robot, which, to the user looking at his own visual representation of the robot, is virtual. Although we could walk up to the robot and touch it, and realize that it takes up space in reality, its virtuality lies in the fact that it is being controlled remotely, and that the user’s movements are being electronically and mechanically reproduced on a smaller-scale model. Essentially what is happening here is an example of what Hansen mentions above as “telepresence,” the ability to “be” somewhere other than your physical position remotely and virtually. By controlling an avatar remotely using only his own body the user experiences virtuality just as Krueger thought would be the most ideal way to do so. Although in saying that I have forgotten Krueger’s second condition for “the ultimate interface”—“human senses.” While the user could easily see and hear what is happening with his robot avatar, what would really take the cake for me would be if the user’s suit reacted to the robot’s interaction with it’s environment. For instance, if the user tried to life a log with the robot, I would like to see the suit give the user some resistance, so that he can feel the strain (proprioceptively) that the robot feels.

I have also included a citation from Hansen because his mention of “the data glove” reminded me of a very cool scene from Minority Report (below), where the user uses a glove to manipulate images on a screen as though he was physically touching them.

Hansen explains that this is precisely what Gibson was hoping to someday see with VR technology—the images are rendered so realistically they look good enough to touch, and with the glove, the barrier between virtual and real is bridged electronically and the human hand may extend into cyberspace in order to rearrange data so that data can be displayed pleasingly to the human eye, back in reality.

“Davies describes her aim to be that of catalyzing ‘the interior kinesthetic and proprioceptive experience of being a lived body in space, of self-movement through space, of being enveloped by space.’ She further emphasizes her deliberate choice ‘not to involve haptic perception,’ meaning ‘hands-on or literal touching, skin to skin, surface to surface,’ so that she could catalyze an experience akin to the interlacing described by Merleau-Ponty—the interlacing ‘of one’s own bodily surface with the visible surfaces of the other(s)… even though in a virtual environment these are immaterial.’ As a laboratory for testing the capacity of embodiment to generate reality, Davies’ work thus restores virtuality as a dimension of embodied life, as a technicity within the living rather than a (mere) technical artifact that affects life from the outside.” – Hansen (111)

“By purposely deploying low resolution in the HMD, Davies’ work actively counters the conventional virtual reality emphasis on vision.” – Hansen (110)

“When art evokes, it’s drawing on the experiences of the user. It becomes interactive on a much more subtle level. To me, Osmose looks at immersive spaces as a place where we can explore what it means to be embodied conscious beings.” – Davies (110)

“I imagine virtual space as a philosophical yet participatory medium, a visual/aural spatiotemporal arena wherein mental models or abstract constructs can be given virtual embodiment in three dimensions and then be kinesthetically explored by others through full body immersion and interaction, even while such constructs retain their immateriality.” – Davies (124-5)

“What you encounter in Osmose is yourself.” – Mark Pesce (125)

Char Davies’ work for the most part brings all of the previous theories together in an example of philosophical virtual reality. While the average video gamer may not enjoy Osmose (below) or Ephemere as much as their typical video games, Davies’ virtual environments allow the user to experience him/herself first and foremost. As explained, Davies intentionally leaves the virtual terrain in a dreamlike, low resolution transparency in order to place the senses of sight, sound and touch on (somewhat) equal footing. Sublte, mystical sounds play as the user leans forward, backward, to the sides in order to move through the terrain, thus combining the theories of the two epigrams we began with—the Cartesian idea of being able to cognitively create one’s world, and the ability to move through the virtual space by proprioceptively “dancing” with the environment. While Davies’ experimental designs lack the beauty and excitement of other VEs, I could see how one could lose their sense of self inside Osmose or Ephemere.

However, from a pragmatic point of view like that of Langdon Winner, is virtual reality really necessary in order for us to experience complete, uninterrupted embodiment? Can we experience ourselves in no other way than to design ingenious contraptions that push us through imaginary lands and leave us in a hyperconnected trance? Doesn’t Buddhist mediation offer something similar? Is Char Davies another example of a scientist looking for enlightenment in non-traditional ways because she’s lost it in others? Such things are beyond my knowledge, but I think they are worth considering.

Hopefully these notes have opened your mind and changed your previously held notions of virtual reality technologies and how “realistic” they really are.

– Dyldebeest

Source: Hansen, Mark B.N. Bodies in Code. New York: Routledge, 2006.

[1] “Proprioception: The ability to sense stimuli arising within the body. Even if you are blindfolded, you know through proprioception if your arm is above your head or hanging by your side. The word “proprioception” was coined in 1906 by the English neurophysiologist Charles Sherrington who received the Nobel Prize for Physiology or Medicine in 1932 for research on the function of the neuron and study of reflex action” (MedTerms).

[2] “Kinesthesia: a sense of awareness of the position and movement of the voluntary muscles of the body” (Student’s Oxford Canadian Dictionary).