Understanding Realism in Computer Games through Phenomenology

Gek Siong Low
geksiong@cs.stanford.edu

Spring 2001

The Trend Towards Realism

The past decade has seen an enormous progress in computing power and graphics hardware, and along with that a trend towards more and more “realistic” games. Each year, the computer gamer sees an improvement in graphics quality and better simulation. Racing cars look more like real cars, sound more like real cars, and handle more like real cars. Trees are now rendered leaf by leaf. In first-person shooters today, you can not only blast the enemy, but also doors, windows, walls, and whatever objects happen to be in the room, and they will burst into shrapnel in different ways depend on what material they are made of. Even the hair of characters in some games is specially rendered to drift naturally in the wind.

There is no doubt that realism is a very big thing in computer games today. Game developers tout it as their main selling point, and video game console manufacturers battle it out over whomever has the best graphics hardware to render the most number of polygons in the shortest time. Gamers and game magazines alike rave about it, or complain about the lack of it. However, “realism” is not an easily definable concept. Just what is considered “realistic” in a computer game can appear to be contradictory. Gamers consistently ignore many obviously unrealistic aspects of a computer game. For example, you can crash a car in a racing game and still continue the race, when in real life the impact would have totally wrecked the car, let alone seriously injuring the driver.

How can we understand what makes a game “realistic” and why some games are realistic while other are not? This paper attempts to understand what it means to say that a game is “realistic” and looks at how we can understand virtual reality through phenomenological concepts.

What is Realism?

Chris Crawford, author of The Art of Computer Game Design, a staple for any budding computer game designer, describes a computer game as “a closed formal system that represents a subset of reality¹.” According to him, computer games are objectively unreal in that they do not physically recreate the situations they represent. It is human fantasy that transforms an objectively unreal situation into a subjectively real one. Fantasy thus plays a vital role in any game situation. Objective accuracy is only necessary to the extent required to support the player’s fantasy.

This is a fairly rationalistic viewpoint. Virtual reality is seen as something that is a product of human fantasy. In other words, it is all in the mind of the game player. I am not saying that this is the wrong way to look at computer game design. Crawford did say that this “subset of reality” must be supported by a certain amount of objective accuracy, and he also warns against falling into the trap of striving for realism and affect game playability, which is something that game designers seem to be ignoring these days in the race for more “virtual reality”. However, virtual reality might be better understood by using a phenomenological approach that takes into account the game player in the real world.

Realism and Perception

Realism in computer games is achieved in many different ways. Perhaps the most direct and cognitively closest is in the graphical quality of games. The first thing people notice about a computer game is how “real” the graphics look. The importance of perception to us is captured in the old adage “seeing is believing”.

However, perception alone is not sufficient to lead us to feel that the game world is “real”. The bottom line is that we interact with the world. We cannot perceive a virtual game world as being “real” unless it reacts to us in a “realistic” way. Merleau-Ponty rejects the idea of perception as simply a passive reception of visual stimuli⁵. Action is a necessary component of perception. This has been shown by an experiment conducted in 1963 by Held and Hein⁵. Two groups of kittens were raised in the dark and exposed the same visual sense data, but one group was allowed to move around, while the other was kept passive. After a couple of weeks, the kittens were released. The active kittens were normal, but the ones that were kept passive kept bumping into things as if they were blind. Referring to this particular study, Varela et al (1991) said, “objects are not seen by the visual extraction of features, but rather by the visual guidance of action⁵.”

The idea of perception as requiring action is most apparent in three-dimensional computer games. The perception of three-dimensional space requires the player to move around in that space in order to perceive it as such. We are able to perceive our three-dimensional reality because our eyes shift constantly and our brains are able to process these slightly different images into the perception of three-dimensional space. A static painting on the screen would not do very much to make the player feel that this is a “real” game world that can be explored, since all the player’s eyes can tell him is that this is just a picture on a plane. Therefore the only recourse is to change the view, as if the player is moving his head to look in a different direction. Some games are designed such that finding certain important objects require the changing of the direction of view and noticing subtle visual cues, although I will not comment here on the appropriateness of employing such devices to make a game more challenging.

Objects are recognized through action. Plates on a rack in the kitchen in a first-person shooter might as well be plastered images on a box unless they respond to the player’s actions. Windows and doors are simply wallpapers if they can’t be opened or blown up or broken down. This is why much effort is also made into the simulation of real-world systems in computer games, to make everything “fraggable”, to put it in Quake terminology. Even in non-first-person-view games, we see the need for action to perceive objects. When the first-time player plays Space Invaders, how does he know that the white blocks are shelters from the alien’s bombs? Easy, he moves underneath them and realizes that the bombs will not reach him. In Pac-Man, how do you know that those big dots are power-ups and not poison? The only way to find out is to eat them.

Action is also required for the simple task of knowing which one of the things or characters on the screen is “you”. Many computer players have no doubt experienced the frustration of trying to find out just exactly where they are in a complicated game scene. This is one situation when less realism might be desired so that the character and important objects stands out. In Pac-Man, the player knows which one of the moving images is Pac-Man because it is the one that responds correctly to his actions on the controller. When friends get together to play fighting games such as Street Fighter, it is very common for them to ask the other person, “which one am I?” especially when both are playing the same characters. It takes a short while for them to figure out that Tom is the one on the left and Joe is on the right, then the fight starts out proper.

The need for action in perception is why computer game players are just as quick to bash games for their “eye candy”, as they would be to rave about their “realistic” graphics. Games that rely on amazing graphics alone are not “realistic” if they do not have a similarly realistic interactive world to match. Dag Svanaes says that it is only through interaction that objects appear to us as immediately existing in the external world⁵. The same is true for the virtual world.

Different Perspectives

The first-person point of view is the “in” thing these days. At first glance, the first-person view appears to be more cognitively realistic and therefore logical. It might come as a surprise then to some people that the first-person view is in fact not realistic at all. Painters have known for centuries that “one should not draw or paint exactly as the eye sees².” Perspective in computer games in based on plane projections. It is like tracing the outlines of objects on a windowpane. A sphere placed to one side of our vision would actually have an elliptical rather than a circular outline, if we followed the rules of true perspective. But then it would be “wrong”. This effect is called “marginal distortion”, and the reason it occurs in real-life is once again a consequence of the human body. As discussed above, we do not maintain a fixed viewpoint for long periods of time, but rather keep shifting our eyes over the scene. To reduce the discomforting effects of marginal distortions, computer games keep the angle of vision artificially narrow, resulting in what we call “tunnel vision”. Peripheral vision is a big problem in first-person perspective. The player cannot detect something that is just off his field of vision. Still, people are quite comfortable with this way of seeing because people are familiar with paintings and films, which operate the same way. Merleau-Ponty would say that we have acquired the skill of how to interpret the world through a window or plane of projection. Our phenomenal field is shaped by our experiences with similar projection-based media. We have no problem interpreting the scene on the screen as something that we would see through our own eyes.

The third-person perspective appears to be less realistic because it is a disembodied point of view². However, it is more desirable in terms of playability in many situations because it allows the player to see everything and judge distances more accurately than using the first-person view. Tunnel vision is not a problem in third-person view. Similarly as for the first-person view, we are familiar with looking at the scene through the eyes of a camera floating in the air. The problem here is how it is that game players can be “inside” the game when they see “themselves” from a detached point of view.

Realism and Disembodiment

The third-person point of view in computer games raises the question of disembodiment. In fact, it appears that playing computer games is a very detached and disembodied activity. The player is sitting away from the computer or TV set, all the action is happening on the screen, and the only link between the player and the game is the keyboard, mouse, or other controller hardware. The rationalistic view of game playing would split the mind away from the body, and say that the player can project his mind into the game world, without making reference to any need for the player’s own physical body in the whole equation. Dreyfus argued against the rationalistic viewpoint of disembodied tele-presence⁴. He says that it is because we are not aware of the way our body works silently in the background that it is so easy to think that we can do without it. The same argument can be applied to computer games.

When you ask a computer game player what he is doing, he will probably say, “I am jumping”, or “I am picking up that ammo clip on the floor” without making any reference to the controller in his hands. The controls are so second nature to him that he is no longer making a conscious effort to link up his physical actions and the virtual actions of his counterpart on the screen. In Heidegger’s terms, the controller is “ready-to-hand⁶”, perhaps more than just ready-to-hand. Merleau-Ponty would say that the player has learnt how to perceive the virtual world through the game controller, that the game controller has become part of the player’s own experienced body, and therefore his own bodily space, just like the organ player that Merleau-Ponty described⁵. Steven Poole, in his book Trigger Happy, also compares this to the way musicians remember how to play a musical instrument without calling it from memory². He calls this “muscle memory”. The player, therefore no longer presses a button to jump, he simply jumps. It does not seem to matter that the act of pressing a button has no relation to the act of jumping.

Merleau-Ponty’s theory appears to fail when applied to the disembodied property of the third-person view. Computer games are a unique phenomenon in that nowhere else can you find yourself projecting your intentions, actions and identity onto something else that is obviously not you, and yet is “you”. It is not simply “some character” that you are controlling. Not even a puppeteer tries to believe that he is actually the puppet. Merleau-Ponty’s concept of the dual nature of bodily space⁵ may help us in understanding why we do not have much of a problem seeing ourselves as an object in this virtual world, but it is inadequate. In the case of virtual worlds there are now two bodies – our physical body and a virtual body. Many computer games today try to link the two bodies together with haptic feedback devices, with varying results. Sometimes they make the game seem more realistic and exciting, but at other times they jolt the player out of the illusion of the game world, a reminder that he is just a physical body controlling a virtual one.

Breakdowns in Realism and Heidegger’s Tool Use Theory

We have seen how action is important for the perception of reality. Interactivity is therefore an important component of constructing a believable virtual world. There are many ways in which the virtual reality can fail, but the illusion does not fail because it looks “unrealistic”. Rather, if you carefully read how many computer game magazines and players discuss the issue of realism, you will find that their major complaints are more about how thing “don’t make sense” in the game. The virtual reality collapses when the game world is inconsistent with the players’ expectations. Poole describes three kinds of such “incoherence”: that of causality, function and space². He claims that it is this lack of incoherence rather than simply looking “unrealistic” that ruins the gaming experience. From a phenomenological perspective, “incoherence” is basically an extension of Heidegger’s tool use theory to the interaction in computer games.

Incoherence of causality occurs when the same action produces conflicting results under different situations. Poole uses an example from Tomb Raider III to illustrate his point. In Tomb Raider III, a rocket-launcher blows your enemies into smithereens, but it does no damage to a wooden door. Instead, to open the door, you have to find a rusty old key. It is as if Heidegger’s hammer could drive a nail into thick steel but not into a piece of wood. The incoherence becomes glaringly “present-at-hand⁶” when that happens.

Of greater severity is the incoherence of function. This is a common error made by many game designers. Many games contain objects that are used only once, and in a particular location or a particular time in the game. Poole considers this a lazy approach to game design. The player is prevented from making a sensible action. At least in the previous example the player got to try out the rocket-launcher on the door. Imagine if you are prevented from using Heidegger’s hammer at all, it stops and a sign pops up in the middle of the air, saying “Sorry, you can’t use the hammer here.” This never happens in real-life of course, therefore there is no reason for it to occur in a virtual world, even if it is “just” a game. If the game designer chooses to provide the player with a particular object, it should work consistently under all appropriate circumstances in the game.

Spatial incoherence is common mistake made by game designers. To recount another example from Tomb Raider III, at a certain stage in the game, the heroine Lara Croft finds herself at the end of a tunnel but could not exit into the corridor because the tunnel exit was at the wrong height. According to the game designers, crawling out of a tunnel involves lowering Lara down onto the floor. It just happened that that was not one of those tunnels, even though it would be perfectly logical thing to do.

When the game world is realistic and consistent, the issue of incoherence becomes invisible. Nobody ever talks about how “right” the game world is, they expect it to be so. Instead, they will rave about how realistic the game looks, and how accurately the physics are simulated, and so on. Consistency is not something players consciously look for in a game. It is thus very much like Heidegger’s notion of tool use. We don’t notice any problem until we run smack into a glaring inconsistency.

Breakdowns also occur in the physical interface itself. Buttons get jammed, and fingers slip due to sweaty hands. When this happens, the game controller no longer is an extension of the player’s body, and he becomes very much aware of the controller in his hands, and of the line between the real and virtual world. The illusion thus breaks down.

Another form of breakdown is in the mapping between the player’s physical actions and what is appearing on the screen. This is usually the result of the player having been trained to use another kind of control interface in other similar games. For example, there are two main methods of controlling a character in many third-person adventure games today. One method moves the character relative to the screen perspective, while the other moves the character relative to the character’s perspective, better known as the “Resident Evil”-style control. A player not used to the latter style of control will at first find it extremely hard and clunky. For a while, the virtual reality breaks down. However, once he gets used to it, he can do amazing things with it and find the style of control natural and intuitive, and he can now be totally be immersed in the game. Here, we find that Heidegger’s tool use theory is not adequate enough to deal with training to use an interface. As Svanaes says, “it is hard to see how it can present itself as anything but a malfunctioning tool⁵.” To understand the learning of a control interface, we have to use Merleau-Ponty’s theories of the phenomenal field as discussed by Svanaes.

Immersion and Virtual Reality

Virtual Reality (the kind with goggles and all manners of sensors, which we will henceforth call VR, to avoid confusion with the usual meaning of virtual reality) is often touted as the Holy Grail of computer gaming. It is often described as the ultimate immersive experience. One definition of immersion is “the sensation of being surrounded by a completely other reality, that takes over all of our attention, our whole perceptual apparatus³.” But is such a feat possible?

VR systems seek to flood the various senses of the player by enveloping him in goggles and a body suit complete with sensors and haptic feedback devices. However, this is a very rationalistic view of virtual reality. We know from the above discussion that we can neither deny nor escape the fact that our bodies exist in the physical world, and our interaction with the virtual world is mediated by physical hardware. The most immediate problem is that the VR player is limited to a constrained physical area. Walking, let alone, running will be a problem. There is also the inconsistency between what you see and what your body feels. Suppose in the game world you are able to leap many feet into air, but of course you cannot do that in the physical world. Your eyes tell you that you have jumped up a great distance, but your body feels the much smaller distance physically traveled. The result is the uncomfortable feeling that what you are seeing is essentially fake, plus maybe the nauseous feeling of motion sickness. When the player’s physical action matches the action in virtual space, the expectations are very much higher and the collapse of the illusion is thus even greater. The act of pressing a button to jump is so far removed from the act of jumping itself that there the player’s suspension of disbelief works in his favor. It is much harder to suspend disbelief if jumping in the virtual world requires jumping in the physical world. The “unrealness” of the act becomes even more apparent.

There is also the question of whether players want to use the same actions to control their virtual counterparts. Will players punch and kick their way through a VR fighting game? It does not make sense to make them punch and kick and getting all tired out when it would be so much easier to just press buttons. After all, Merleau-Ponty tells us that the controller becomes an extension of our body.

Murray describes computer games as a participatory medium³. Immersion in computer games requires the user to actively participate in the story. The limitations of our physical bodies in the real world limit the types of interaction we can have in the virtual world. I am not putting down VR as a foolish and impossible dream. VR systems cannot recreate reality, but they are appropriate for certain types of games and applications. Understanding how humans perceive the world through their physical bodies is essential to creating a believable and meaningful virtual environment.

Breaking Down the Walls of Reality

There is new kind of computer game that seeks to make games realistic in another way. Majestic⁸, an upcoming game to be published by Electronic Arts and inspired by the movie “The Game”, weaves a complicated story of conspiracy and intrigue in the physical world, using the player’s phone, fax, email and even instant messaging software as integral components of the game play. Dummy web sites were erected, dummy corporations were founded, and phone numbers were acquired just to create an elaborate living set for the game. It is a virtual game that takes place in the real world. Whether this radical idea will succeed is anyone’s guess right now, since there are many more factors that determine the success of a computer game other than its ability to create “reality”.

Also being contemplated are games that can be played on mobile, context-aware devices, such as the cellular phone, that takes place in the physical world. It is not hard to imagine scavenger hunts or some form of role-playing game being played on mobile phones.

Yet another form of reality being researched is Augmented Reality (AR). Milgram et al describes a taxonomy of that relates current AR and VR work⁷. On the Reality-Virtuality continuum, AR lies near the real world end of the spectrum while VR lies at the other end. The basic premise of AR is to project virtual images on real-world objects in order to assist real-world activities, with applications ranging from surgical operations to military aircraft to entertainment. In fact, we are already seeing a simple form of AR on TV right now with the chroma-keying of sports events.and weather-forecasting. The concept of AR games that take place in the real world is not so far-fetched, although there is a certain Don Quixote quality to the idea. AR might turn out to be a better solution to trying to recreate a “real” world with VR systems which, as we have seen earlier, face a lot of problems due our existence in the real world. However, it is still too early to tell whether AR games will succeed. When they finally appear on the market, we will then be able to test and perhaps redefine our phenomenological theories on people’s perceptions of reality.

* * *

References

1. Crawford, Chris (1982). The Art of Computer Game Design.

2. Poole, Steven (2000). Trigger Happy: Videogames and the Entertainment Revolution.

3. Murray, Janet (1997). Hamlet on the Holodeck: The Future of Narrative in Cyberspace.

4. Dreyfus, Hubert (2001). Disembodied Telepresence and the Remoteness of the Real.

5. Svanaes, Dag (unpublished). Understanding Interactivity, Chapter 3: Non-Cartesian Alternatives.

6. Winograd, Terry and Fernando Flores (1986). Understanding Computers and Cognition.

7. Augmented Reality Home Page. http://www.cs.rit.edu/~jrv/research/ar/

8. GameSpot Preview of the PC Game Majestic. http://gamespot.com/gamespot/stories/previews/0,10869,2655645,00.html