Game Audio Innovations - Papa Sangre

Smart phones and other portable devices have become an essential source of entertainment for people. As time passes by, more and more game applications are made for mobile devices.  However, due to the small dimensions of the screens, immersion is difficult to achieve using iPhones or iPods.

The company “Somethin’ Else” decided to innovate creating  “Papa Sangre”: a horror audio based game for the iOS. The player is supposed to be lost in the land of the dead, where everything is dark and nothing can be seen. Someone’s life is in danger and the player must rescue him and escape, or he will stay trapped in the darkness forever. The world is recreated fully with binaural soundscapes, that can only be appreciated through headphones. Just a few images can bee seen in the game for navigation purposes:

Michel Chion mentions in his book “Audiovision, Sound on Screen” (1994), that hearing sounds without seeing their cause, can create huge emotional impact.  He quotes Pierre Schaeffer who explains how acousmatic listening (the excersice of hearing sounds without seeing their source) allows the sound reveal itself in all of its dimensions, making us draw our attention in traits normally hidden. A crashing melon, for instance, can be used to represent a tank crashing a little kid, or it can also represent a funny situation at the beach (Chion 1994, p.32-46).

In a certain way, images limit the imagination, and thus, listening to isolated sounds makes our minds go far beyond those limits. Something similar happens when reading a book with illustrations or reading a book without them. This is why in horror films, sometimes isolated sounds suggest scarier things than if there were any images present.

The use of binaural technology in Papa Sangre fits perfectly into the “iDevice” culture that we live today, as everyone that has an iphone or an ipod uses headphones. Somethin’ Else created unique soundscapes and absorbing immersion using this technology, which basically consists on having a mannequin head to place two microphones, one in each ear.  The recordings can simulate sounds coming from thousands of sources, creating  a 3D impression for the listener, and thus, making the immersing experience more effective.

Even though this game was made in 2010, I personally believe it is still an innovative approach that helps giving more relevance to sound in the visual media scene. With these kind of games, people become more aware of the vast potential of audio to influence the perception of image in favor of immersion. This is crucial, as handheld devices become more popular, and the game industry has to find effective ways to create immersion.




BIBLIOGRAPHY


Chion, Michel (1994) The Audiovision: Sound on Screen. New York ; Chichester : Columbia University Press.

Doctorrow, Cory (2011). Papa Sangre: binaural video game with no video. (Internet) Available from:<http://boingboing.net/2010/12/19/papa-sangre-binaural.html>(Accessed 15 November, 2011)


Edge Magazine (2011) Somethin' Else on Papa Sangre's value. (Internet) Available from: Edge Magazine: <http://www.next-gen.biz/features/somethin-else-papa-sangres-value> (Accessed 15 November, 2011)

Papa Sangre (Internet), Available from the Papa Sangre Website: <http://www.papasangre.com/> (Accessed 15 November, 2011)

LITTLE BIG PLANET: AN EXAMPLE OF INTERACTIVE MUSIC

Little Big Planet is a game that uses various interactive music techniques throughout its levels. In this post I will analyse some of these techniques, using The Game Audio Tutorial book by Richard Stevens and Dave Raybould as a reference.

The Gardens Level (Video 1):

Music Switches And Smooth Transitions

All levels in Little Big Planet are marked up by gongs whose main function is to save the game at particular locations. Most of the times, the gongs also work as switches or triggers that activate changes in the music. As mentioned in the GAT book, switching from one song to another might sound “clunky” if this occurs at an aleatory time of the song (p.184). Therefore, in order to soften or mask these changes, transitions or certain techniques like cross fades or smoke effects are required (p.187). In Little Big Planet, both techniques are used.  The gongs have a special smoke-like sound effect that helps masking the change from one state of music to another. At the same time, there is always a subtle cross fade in the music when one song switches to a different one, helping the music vanish smoothly.

Look at minute 3:40 in the above video. The volume of the music lowers down as soon as the player reaches the gong and the wheel. When he reaches the next gong, the same music keeps playing, but now a flute melody and a "banjo" counterpoint appear along with it. Throughout the game, each time the player reaches a character to speak with it, the music volume turns down.

Layers: Subtle Changes in Music

The GAT book explains how in most games, music is only present in special occasions in order to mark emotion and have more impact (p.164). Little Big Planet is not the case, as it has music all the time.  However, it manages to achieve variety and express emotion without sounding repetitive or boring. This is mainly accomplished through the use of layers. Most of the times, when a player reaches a gong, a layer of music is added or removed. For example, the appearance of a flute solo, a saxophone improvisation, or a drum beat.  In the GAT book, this technique of overlapping stems is called Vertical Layering (p.213). This way, during a whole level, there might be just one song made up by different layers, that make the tune more interesting and varied.  The use of layers also helps solving the problem of disk space vs. music variation; with just one song made up of different layers, you can make many different combinations.

In the following video we can see an example of layering. 

The Savannah Level (Video 2):

Watch minute 2:37. As the player advances, the music gradually varies some melodies keeping the same rhythmical base. Instrumentation varies as well, providing different changes in the music texture. Some times there are less instruments than other times. Now listen to minute 6:26. The music starts playing a saxophone melody that goes in counterpoint with the flute.

Creating Impact Through Radical Changes in Music

When an extreme change occurs in the visual sequence of the game, the song changes to a different one, reinforcing the mood and events going on. This creates impact as it breaks the long period of a layering song. In minute 5:20 of the above video, the layering song stops as the player reaches a gong and gets into a club. The gong’s smoking effect camouflages the abrupt stopping, and a new song starts sounding as if it were the one heard at the bar (functioning as a diegetic kind of music). The fact of changing the song using diegetic music, is one way to contribute to the immersion and variety in the game.


Exceptions: Linear Music

As we have seen, Little Big Planet explores many different types of music techniques. It even uses ones that are not so common in video games, like linear music. As stated in the GAT book, linear music is the one that lasts a specific duration and goes along with a visual sequence. It is mainly used in film, trailers, credits, or adverts (pg.172). It’s not interactive, unless you have the option to turn it on or off.

The following example shows a level in which the music is a licensed song composed by the Mexican band Kinky. 

The Canyons Level (Video 3):

This song works fine in the game as it is long enough to cover the level. If for any reason you spend too much time in the level, it will loop without sounding boring.


Loops

In Little Big Planet we can find loops among the layers that build songs. Some layers loop, and others don’t, making the whole music sound organic and fluid.  The saxophone example mentioned above in the layering example, serves to illustrate this.

There are other types of loops that occur during the menu screens, when the player is choosing where to go next in the map. This music, referred to as “lobby music” in the GAT book (p.173), is a short tune that suddenly stops when the player presses start. In Little Big Planet, a sound effect helps attenuating the sudden ending of the song at any point. See minute 0:26 in video 2.

Final thoughts

Interactive music has to be flexible. It must be made adaptable, so that it can be easily combined and mixed throughout the game, as this is crucial in order to achieve variety.

The music in Little Big Planet includes many different styles even when disk space is limited in video games. The game’s developers and sound designers had to combine all the techniques mentioned above, in order to make the tunes varied and rich in harmonies and textures. As a result, the whole soundtrack stands out from other interactive games, and it proves that non-repetitive design in music is possible even when there is music present throughout the whole game play.

Half Life 2, Episode 2: An example of Non-repetitive Audio

Half-Life 2, Episode 2 is a great example of non-repetitive audio in video games. The ambiance, voice acting, and particular highlighted sounds, have special treatments that help achieve variety in the game.

Each location has a well defined atmosphere. Sometimes the ambiance is made up of different layers that create textures matching the mood of the scenes. There are also smooth transitions from one place to another, allowing you to hear faraway noises from the place you were just before. Often, the ambiance melds gradually with music pads in a way that makes it impossible to spot loops.

The following is an example of a well defined ambiance, with mysterious pads and subtle birds outside. See min. 6:40 - 6:60. (In this link you can see it with more quality in min 6.25). An example of a smooth transition can also bee seen in the video, when the character moves from an exterior place into a cave. See min. 4:28:

Video 1.

Cave ambiances have mysterious music pads layering with other sounds, as seen in the following video. See min. 8:59

Video 2.

There are many cases in which it seems as if there were no sound ambiances at all. We can see this mostly in exterior places. Often, specific elements appear just to give a notion of the place, but they don't remain for a long period. One particular case is the birds. In many exterior scenes, there are birds singing for a few seconds. This serves as a transition from one interior place to an exterior location. However, the birds don't loop throughout the scene; they just mark the beginning or certain spots. Silence never lasts too much, as different noises or music pads begin to appear and mix with each other eventually. The next video clip is a good example:

Video 3

The music appears when danger is near. It starts in a subtle way, with a pad that mixes with the room tone, and gradually it all becomes a beat action music to emphasize  the danger. All the atmosphere (built up by ambiance and music) add tension to the action parts. Watch min 0:48 in the next video.

Video 4

Voice acting in this game is also a key element that enhances the whole gaming experience; it’s realistic and very emotive. The computer/robotic voices have different pitches, filters and special sound effects that add variety to the game making it hard to be bored. The characters' voices achieve a high level of expressiveness not only by the great voice acting itself, but by the use of non repetitive dialogs.

Video 5

Accurate detail in some particular sounds is the third audio aspect that stands out in this game, making it non-repetitive. The “reloading weapon sounds” are very specific. Footsteps are always different. The shooting sounds vary sometimes, depending on who or what is being shot. When shooting cars, you hear the the gun shot plus the metallic sound of the car hit by the bullet. Footsteps also vary depending on the material of the floor. There are footsteps over water, over wood, over metallic surfaces, etc. So, in general, there’s good coherence between the sound and the material. These small details help to make the game more realistic and non-repetitive.

Video 6

Overall, the game has great sound design largely because of non repetitive-audio. The specific highlighted sounds, subtly add detail and realism to the game, drawing the player’s attention to these sounds instead of the ambiances. Simultaneously, the atmosphere fusions organically with the music, emphasizing the events going on screen when there's danger and action. Ambiances are either made up of layers of sounds and music pads, or they are silent with key elements which are enough to define the ambiance (like the birds). Voices are always different in terms of expressiveness, pitch, and effects. All these aspects make it hard to find loops or repetition of sounds, making the game more interesting and providing an immersing feeling to the whole game experience.

Bad Audio in Heavy Rain?

Award winning video game. Impressive graphics. Touching music. Stunning story. These are just a few words that first come up whenever someone says Heavy Rain. Could it be possible to find bad audio in such a master piece? In this post I'll discuss three main issues. First, I will briefly discuss about the voices in the game. Second, I will show my reasons to disagree with Kenny Young's claim that missing sounds in the game are audio crimes. (These two issues have been widely discussed online here and here). Finally, I will talk about a problem that has been overlooked in the game, related to the use and abuse of loops in some segments.

The first thing I noticed when I played the game was that the main character’s voice didn’t sound realistic. It sounded plain and out of context. Actually, many online articles and blogs discuss about the bad voicing in Heavy Rain. The main concern about this subject is that immersion is always interrupted when characters speak because their voices don’t match with the high level of graphics and visuals in the game, resulting unconvincing. This happens due to poor voice expression; especially in the main character Ethan. His voice always sounds as if someone was reading a script.  

Another problem regarding voices, is the repeated use of two audio samples during a scene when calling name. This could probably be common in a 1993 game like Cybermorth:

(Video by AVGN)

Even though it is not so critical in Heavy Rain as in Cybermorth, it can really be annoying to listen at sometimes:

 (Watch min 4:06.)

There’s a very interesting article that I strongly recommend, about voicing issues in Heavy Rain, posted in The Guardian’s Games Blog called Voicing concerns: the problem with video game acting, which can be found in this link. The article deepens on the importance of expressiveness in voice acting in complex games that involve dense psychological stories; the way such voices should transmit emotions to the game players, and how this can be achieved through writing stronger scripts and not leaving voice acting to the end of the production chain. In this post I won’t analyse this subject, as it has been meticulously examined in the above-mentioned blog.

Missing sounds is the second audio issue I found in Heavy Rain. The car windshields don’t have sound, even when they appear in foreground. The first scene, where Ethan’s wife is washing something in the kitchen, has no sound at all; the footsteps in the balcony make no sound, as well as some of the kids’ footsteps on the grass. In the blog Sound Spam, its author Kenny Young comments about more missing sounds, and classifies this issue as an audio crime. He mentions that the lack of some foley makes the characters seem a little floaty and fake. I agree with him in the fact that some sounds are missing, and that's why I mentioned this as an audio issue. However, I wouldn’t say this is audio crime because of the cinematic style of the game.  Due to the strong and intense story, this particular video game is suitable for different sound and music cinematic treatments. As stated in the book Sound and Music for Film and Television by Tomlinson Holman, music heard alone serves to recreate or make emphasis in an abstraction. I would say an abstraction could be an emotion, a feeling, or maybe a dream. The use of foley sound effects help the scenes look more realistic. This is why many films start a scene with just music, and gradually add sounds to draw the audience into the story.  It is a subjective matter the way people interpret these sound design decisions. For me, Heavy Rain has missing effects in parts that are charged with drama and emotion and where music has the main role; the extreme detail in sound design wouldn't contribute to this kind of enhancement intentions.  For others, it might simply be unacceptable to have missing audio.

Finally, the loop issue. Heavy Rain is a slow paced game in which the player spends a lot of time in each scene discovering all that can be done. This happens because there are numerous possibilities to explore, the actions and movements of the characters are slow, and the story is dense, so you must think carefully what choices are better for you. The problem with this is that suddenly you start identifying loops associated to specific elements that you would expect to appear randomly. An example is the TV show that Shaun watches at the beginning of the game. Sometimes it’s a dialogue accompanied by a carnivalesque music and a funny laugh, that sounds again and again, and every time a character turns on the TV. It is then followed by a drum loop, and then a soprano singer.

Take a look at this video to see the TV examples in minutes 4:44 and 8:55. In 5:16 there's the soprano singer, and then again in 9:20. The music is in 3:22 and again in 6:30. This sounds also appear in the prologue, which is supposed to have happened 2 years before.

It’s also inevitable to start listening to ambiance looping. It happens in the park scene, when Ethan and Shaun are playing with the boomerang. The wind sounds similar to the ocean waves, producing a short pattern very easy to identify. Usually you wouldn’t pay much attention to the wind, but in Heavy Rain I noticed that there are really long periods or “spaces” that lack of music, making these ambiance loops more exposed and evident to the player. So, even if you changed the short wind loops for longer and more subtle ones, you would probably still notice them and the scenes would seem too empty. Thus, I think the real solution to this issue would be planning better the relation between difficulty and music. The more difficult a scene or place is, the more you have to wonder around trying to find out what to do. Therefore, the absence of music becomes more evident and repetition of ambience loops results boring.

Those are all the audio issues I have found so far. So, even though Heavy Rain is not quite the best example of a game with bad audio (because, overall, the audio in this game is actually good), it still has some audio issues, and some of them could have been avoided. Nowadays it is very difficult to find a recent game with bad audio, specially if it's a console game. In my opinion, the most common modern issues have to do with voice dubbing. In old games there were more cases of bad audio because audio technology was not so advanced and sound for audiovisuals wasn't even an academic field and therefore I suppose sound design wasn't an important aspect for games as it is now. Thus, finding audio issues in a recent game helps us see what problems are faced today.