An Investigation into the changing world of sound design for game development

Now, before we begin, I feel it’s important to stress why I think that this is an important topic to discuss.

Generally I have two passions in life, which i would call hobbies, yet they extend further than just simply spending spare time dwelling on any one topic, and form more of an obsession.

Those are Storytelling, and Sound Design. Now the first part of that you may of seen me write a little about in my previous post here on Stratics, but the latter, is important as both a tool, and a seperate narrative of telling the stories within the lore of any franchise, and especially World Of Warcraft.

Whilst at university in the previous year (2013), i took it upon myself to explore the world of video game sound design and i managed to collect enough data to perhaps make a congent argument worthy of being published somewhere someday. I’m happy to say that with recent events allowing me to be a part of this new, growing community, i have found that place. I feel even thought this article observes more than just the game this blog is under, it affects WoW much more than anything else i could of imagined, and indeed feels at home on these pages. So i must warn you, prepare for a particularly heavy discussion here, and hopefully this inspires some of you to reply in either your own post below or even share a page with your insights on the matters at hand. Either way, let’s get this started.

The world of digitised gaming media, known as an affectionate hobby by many in the present day, and widely accepted as a growing area of career opportunities for professional voice talent, music artists, and studio musicians, to name but a few roles of an ever growing list. Is an outlet for both mainstream and less pronounced talent with a growing acceptance by people worldwide as a recognised media outlet, on par with that of film and television, with both Commercial and financial appeal to consumers and various businesses alike.

Even writing this now, it is apparent that even researchers such as Catherine Collins, whom works as a researcher for the Canadian Centre For Arts & Technology have had significant difficulties with such research being taken seriously until now with only the arrival of such acceptance within the last 8 years.

“When I first began writing about video game audio in 2002, it seemed somehow necessary to preface each article with a series of facts and figures about the importance of the game industry in terms of economic value, demographics, and cultural impact. It is a testament to the ubiquity of video games today that in such a short time it has become unnecessary to quote such statistics to legitimize or validate a study such as this” Collins, C (2008).

To get to this stage of potential mainstream acceptance, this progression is largely down to the technological, social and cross-medium changes which have bred from other industries and inventions over the last few decades. Audience interpretation and understanding of advertisement opportunities for advanced careers in a new industry were also huge in the involvement of individuals from other industries and some who would rather take a gamble in this industry than continue their hardships in the competitive world of television and film sound design.

The start of the story of technological advancements in regards to sound development with the gaming world starts within the confined home of the first home computer, the ECHO IV, an highly experimental computer designed to be the first attempt at bringing computers into the everyday home in 1965, but gaming only took off with the release of a very clever piece of software known as ‘BASIC’. The name is an acronym ‘Beginner’s All-purpose Symbolic Instruction Code.’ The first way of actively programming outside of scientific research for everyday students to learn how to write their own software. This lead to the creation of a very popular game in 1972, known as ‘Pong’, not the first game, but iconic for both its simplicity and working fluidity technically. Pong had extremely basic coding and allowed for a single sound to be emitted, a simple bleep. This sound was only generated when the on screen bar hit with the ball, and was one of the first games to have integrated sound. This simplicity shaped the future of video game sound to come. That single noise signifying a positive result in the game, lead the way for importance for sound in a video game. Without that sound, would it not be appropriate to imagine that the person playing the game would not of recognised they had performed the correct action. It is this realisation of the power of sound that intrigues me, the connections between actions and sounds, reflecting on different decisions.

Cause and effect, very simple principles within the film industry, and it is at this very first stage that comparisons could be found between the two industries, despite their difference in size and current legitimate acceptance as an actual industry not completely devoted to just a hobbyist pursuit.

It could be completely understood, though. Many early games demonstrated very little in the way of anything more than 5 minutes of entertainment followed by the realisation of costs to play them in arcades in a pretty bleak economic climate in some parts of the world. Gaming had very scientific backgrounds, but virtually no context to how enjoyable it actually was to play them. Many forms of entertainment were on offer, which offered more social experiences than this hobby allowed, and it was this, amongst many other reasons that it needed to advance before it got to the point of not continuing to be worth the time invested in making such things.

Many believed it didn’t have the potential to be anything more than just a fad that lasted the month and then got replaced by the next innovation to bide the time and offer entertainment between the daily lives of people in actual jobs. How very wrong they were.

When you look at the early days of film, it’s important to note that, that two had a significantly similar beginning with silent movies. These films used simple piano melodies and musical accompaniment to help drive narrative and tell a story without the ability to convey written messages other than on screen slates with text. The music would be catered to the original article and designed specifically for that film, with various changes in melody and tone to show changes in mood or feeling within a scene. When film eventually incorporated technology that allowed the use of actual narration and voices to be heard along side the pictures with the Phonograph, people’s reaction was unwary, and it was actually rumoured to be a failure at the time until October 6, 1927.

This day remains one of the most decisive days in the history of pop culture. It changed the course of an industry, the expectations of the public, and forever altered the form of an art and how it is perceived by all.

The event was the first public presentation of Warner Bros.’ The Jazz Singer starring Al Jolson, which was the first film to feature talking sequences.

The death of silent movies was fairly sudden with this breakthrough, and aside from some belief it was the death of an industry, it actually resembled more of a evolution in terms of an increase in quality of filmmaking and quality of sound design.

“Did the “talkies” kill the “silents”? In some ways, yes, the rivalry is as simple as that. Hollywood has always been an industry, and it always goes in the direction of profit. When “talkies” promised increased revenue, that’s where producers invested their time and money. But in other ways, the “talkies” never killed the “silents” because silent cinema never truly died.” Gallagher, C. (2009).

The film industry could see the increased quality offered by this medium and decided that this was indeed the best form of progression for the medium, and allowed for great forms of revenue from allowing for actors and actresses to earn fame and respect for their talent now they could be both seen and heard. Eventually people got used to the change and actually realised the benefits of immersion that this new form of filmmaking allowed without breaks and constant musical drive, but rather the talent and emotions of the actors seen on screen.

Video games were no different at this point and with the progression of the medium, sharing technology from various other media allowed for progression to a more advanced form of storytelling and immersive form of game creation, past that of simply a hobbyist activity, and more of an experience growing to a level that film did, albeit just as gradually.

These confined limitations of technology lasted shortly, with these arcade machines featuring games such as pong incorporating Phonographic records and compact cassettes within the machines, with the sound recorded and played as analog waveforms, which was taken directly from the music industry. This extension allowed for more elaborate sounds and basic themes to be composed and many of the games made full use of these sounds. The biggest constraint here though was the longevity of the format the recordings were recorded onto, as they tended to be very fragile and not last very long, Ultimately resulting in much doubt as to the success of this hobby.

Longevity of the platform was a short lived problem however, with the 1980’s bringing the introduction of home consoles and with them the creation of video game cartridges, a much more robust way of playing video games in the household, and with a larger set of microchips allowing for at first 8 bit, and then 16 bit memory allocation. 8-bit and 16-bit, for video games, specifically refers to the processors used in the console. The number references the size of the words of data used by each processor. The 8-bit generation of consoles (starting with the Japan’s Famicom, the equivalent of the US Nintendo Entertainment System) used 8-bit processors; the 16-bit generation (starting with Japan’s Turbografix-16) used 16-bit processors. This affects the quality and variety in the graphics and the music by affecting how much data can be stored, and thus within the same decade the limitations of those designing sound for games was indeed increased, however the same limited capacity of the cartridge was impacting on the freedom to create elaborate themes and sounds for the games, with the majority of that allocated space being used for the game itself.

Many of the original themes and melodies were composed with the ‘noise channels’ onboard the cartridges for percussion. It was the use of everything onboard the chipset of these cartridges that allowed for innovation, and at the time of the creation of these games, competition to create a unique sounding game in a very limited space became somewhat paramount to that effect.

“By altering the volume and adjusting the timing of the two pulse channels, phasing, echo effects, and vibrato could be simulated.” Collins, C (2008).

Noise is a huge component of sound, and noise in digitized music, especially in easly synthesised pieces found within video games are purely noise channels, bended and manipulated to form synthesised versions of effects found on other instruments, such as tremello, phasing, and pitch correction and alteration.

“As with rhythm, melody can stretch the experience, challenging the brain to find the relationships. We tend to be more attracted to music that has very little melodic information (lullabies), than too much” Sonnenschein, D (2001).

Simple melodies, created with very little expansion to be anything more were bound to be rather successful in terms of remembrance, as they were 2-3 minute loops and very little more than 8 tracks of noises resembling as close to real instrumentation as possible. Forged from simple noises, nothing dissimilar to Pong’s simple bleeps that started it all.

None can be said to be more explorative of such sound generation from the world of noise but R.Murray Schafer, a pioneer of research into the sonic environment through the development of civilisation, from simple noise to music.

“Unwanted Sound. The oxford English Dictionary contains references to noise as unwanted sound dating back as far as 1225.” “This makes noise a subjective term. One man’s music may be another man’s noise.” Schafer, R.M (1977).

The idea here is that interpretation of the limitations available to the programmers of the video games, and those in charge of programming these early midi based chiptunes and themes was the reason such innovation was found, and ultimately the boundaries of the technology were being pushed beyond their capabilities which resulted in the progression and innovation of new technology to fulfil the purpose of both better games and larger special boundaries to fill.

These limitations during the cartridge era are interesting, as the limitations that were in place prevented much other than simple looped melodies and themes, which i would argue cannot be replicated past this age in video game evolution as the constraints are significantly elevated and more intricate design can be used with very little re-percussions, especially as technology advances.

With the invention of the cd-rom driven consoles such as the original ‘Playstation’, which in many cases used un-original music created by musicians, these boundaries were stretched further. The fact that the early ‘chiptune’ themes, as they have come to be termed were created by the programmers of the games themselves, instead of an assigned musician, or composer to create an original score for the game, does see a transition into a larger media representation, and thus allowed gaming to somewhat make the first steps to become an equally represented and accepted medium as film and television.

Confined conditions are what formed these sometimes iconic sounds of the period of the late 1970’s and early 1980’s and dance music was largely influenced by the sounds heard in the chip-tune’s from video games of the time.

Repetitive themes run through video game soundtracks as loops, as they do within popular dance music of the time, using virtually the same synthesized technology to create tracks.

But more interestingly is the appreciation and allocation of actual music outside of the gaming sphere of influence. Many musicians are more inclined to include their tracks into a videogame simply for advertisement and a form of viral marketing and semiotic considerations that come into play when popular music is used in video games.

“Music in games is heard in highly repetitive atmospheres: to point to another statistic noted by artists, 92% of players remember the music from a game even after they have stopped playing it.” Collins, C (2008).

It’s for this exact reason that the video game industry became extremely popular with many musical tie-ins in both advertisements on television and in-game music, purely on a financial level to benefit both the artists and the success of the video game license. Much like the easly film days realised the potential of voices in their films and the use for advertistment, gaming had at this point become self-viable in it’s guaranteed place amongst the other media industries.

Influences from the world of film are clear in present day videogames, with many original scores for game soundtracks being based on film sequences from iconic films.

Hideo Kojima, the creator of the ‘Metal Gear Solid’ series mentions in many in interview that the biggest influence on both game design and sound design comes from some of his favourite films.

‘The biggest influence on Metal Gear is the 007 series — especially Sean Connery’s Bond.’ Kotaku. (2009).

The themes within the game series are heavily weighted on the theme of espionage, much like the 007 films, and as such the incorporation of Harry Gregson Williams, a composer who has worked on films such as Total Recall, Prometheus, and The Chronicles of Narnia to name but a few, was a perfect match. Sourcing top film composers and musicians to develop the feeling within the game is now commonplace, but was very recently unheard of.

‘Three of four years ago I was approached by Hideo Kojima who asked me to consider doing MGS2. He said he had a vision for his game to be scored as if it were a Hollywood action movie. He seemed like a nice bloke, and I never even considered writing music for a game before, so I thought I would give it a go. He was great fun to work with.’ Williams, H.G. (2004).

The first game was developed on a much smaller budget, and only the sequel, once the original had developed both the finances and publicity it needed was able to move to a more elaborate, movie-based soundtrack. The original game in the series was actually developed in-house at Konami with much of the music resembling midi compositions very much akin to the simple loops and themes found within the previous generation of limited sound design.

‘With a movie I’m working constantly in synch with the picture and I score every moment very deliberately. In the MGS games that I’ve done I have had to write all the music without any visual aid but from detailed, written descriptions given to me by Hideo outlining the mood, tempo, and atmosphere of various situations he was trying to create.’ Williams, H.G. (2004).

With many of the sound design elements being developed around the action on screen in film design, games like this series develop an entirely different approach to how sound design is seen within the medium and conversely, despite similarities to feeling, develop and entirely different creation process.

The evidence suggests that sound design within current day video games are impacted by the success of publicity and finance, much more than it is the availability to generate original themes and elaborate scoring due to technological changes in console evolution. A factor not considered previously in the 1970’s and 1980’s when sound was a much more significant after-thought from developing the main game.

There is however a common theme which shows a more symbiotic relationship between the music industry and that of the video games industry, games are being used to promote and sell music, and recording artists are being used by to market and sell games.

“Games with a musical subject or narrative, or rhythm-action games, have generally enjoyed great success, from the electronic ‘Simon’ game of 1978 (Milton Bradley), to ‘PaRappa the Rapper’ (SCEI, 1996) on the PlayStation, and more recently, music based games like ‘Guitar-Hero’ (Red Octane, 2005) which won five academy of interactive arts and sciences awards in 2005” Collins, C (2008).

Outside of musical themed games the same applies, whether it be a sports game, such as ‘Tony Hawks Pro Skater’, or the latest ‘Formula One’ game, the music used within is memorable for the action on screen and the branding behind that scene. It actually does provide the music industry a viable generation of custom through nostalgia.

The theme of Nostalgia is a common theme within video game hobbyists, based on memories of particular boss battles, encounters against the odds in variable situations that can be instantly recalled upon hearing the theme or sounds used in that original scene. This theme shares heavily with the film industry, in films such as ‘2001: A Space Oddity’ by Stanley Kubrick, where the memories of any scene in that film can be recalled with just hearing a few notes of ‘The Blue Danube’ a waltz by the Austrian composer Johann Strauss II.

“The self-reflexive and stylistic constructions throughout the film encouraged a constant questioning of image-sound relations, clearly moving the sound track away from the notion of realism and charging it with speculative possibilities” Whittington (2007).

The idea of a film, or a game being iconic is not born from simply the soundtrack it is given, but obviously the film or mediums context and ability to convey it’s story, however the sound design is instrumental in shaping that journey and has the ability to add feeling to the pictures on screen, and if the music used, either composed originally or sampled from another medium, as films and game do, fits well, it makes that film part of the nostalgic feeling associated with the memories of a particular scene, or the entire production. Games such as the ‘Metal Gear Solid’ series mentioned earlier share this feeling, and so does the ‘Grand Theft Auto’ series, which used music released at the same time in the real world as the time period the game is set into.

The first experience the player encounters within ‘Vice City’ for example, is that Of ‘Billy Jean’ By Michael Jackson when you step into your first car the game gives you on a moonlit night. I could go as far as describe exactly what happened during the in-game drive to the next objective for the duration of that track. The importance of the neon-lit 80’s themed Miami and songs like this coalesced into a perfect experience.

“The narcissism of the era is apparent from the outset, as you hop into your 4-door sedan to the tune of Billie Jean and drive through the bustling city, its bright neon lights lining the concrete. In terms of video game openings, you will find none greater than your first introduction to Vice City, and its soundtrack plays a huge part in convincing you that you’re a part of this hedonistic era.” Tamburro, P. (2012).

If the game had used music randomly from a different era or possibly a less popular track of the era for that scene, it would likely not have been as memorable as it was, which emphasises the importance in driving a story and adding a form of realism, despite that realism being false.

The idea of realism in sound design is an important factor for any sound design in media. The idea that sound doesn’t travel realistically in our world does not affect the chances of us hearing sound in a space-related scene in either a film or a video game.

The idea that what is realistic and what is acceptable without our ears questioning a sound is very different, as we have grown into a world where countless science fiction television series, film and games show us a rocket flying through the air, we expect to hear that rockets projection of sound, regardless of it’s altitude or atmospheric position.

In the film ‘Apocalypse Now’ 1979, a film about the war in Vietnam features a scene involving a helicopter attack where countless helicopters fly in formation in an attack on a Vietnamese village. Whilst the attack is performed the theme ‘The Ride Of The Valkyries’ by Wagner blasts out amongst the sound of rotor blades and engine noises swooping across the air.

“The intent was to construct a visceral spectacle with image and sound that mimicked combat anxieties and confusion, as it offered shifting subjective points of view”  Whittington (2007).

Neither is the use of music realistic or capable of adding realism as we know it in our world, yet it does allow for the addition of realism to the genre in which it is placed. This is more important to uphold, for this is what ensures the scene lives up to both expectations from the viewing audience, but also the normality they are used to.

A good example of what happens if sound, un-realistic to the genre would be to include noises for realistic outcomes from activities if they were performed in real life, and then dubbed over that of a game such as ‘Super Mario World’ from the 1980’s. The result would be highly confusing and detract from the main focus of the game.

Video games are not always based on our own world, and in many ways they are an escape to reality on par with fictional films and television.

There are no written laws to what can or cannot be done, but it comes natural for example to expect alien sound effects and themes from outside our existence when playing a video game that isn’t trying to be realistic.

However some video games do try to be as realistic with sound design and effects as possible. Most modern day first-person-shooters, or ‘FPS’ games such as ‘Call Of Duty’ emphasise a somewhat realistic war environment. They source gun sounds individually, from reload sounds to how much sound changes depending on rate of fire, wind speed and more. It’s to this degree that video game sound attempts to replicate the exact conditions in each scene on screen.

From the visuals side of video game creation, realism is added by motion-capture technology to replicate the exact movements of a human performing an action in a game, with animations that are realistic enough on top of that to ensure it doesn’t break immersion. In the world of sound design, the capabilities of the immersion of a soundtrack and the sampled realistic audio of those guns for example, can only be as realistic as the audiences technology allows them to hear it.

“People’s media experiences are influenced by degrees of realism and they can be immersed in these experiences as if they felt were real. Because of this immersive experience, it affects our behaviours, thoughts, feelings, and arousal.” Barlett & Rodeheffer. (2009).

Innovations in technology such as advance surround sound allow for a more immersive experience, yet inflict on the likelihood anybody will get to hear the game anywhere near that quality based on affordability of the required hardware or software.

Technological assertions aside, realism is only the focus of some games, with many, still aiming for a more immersive break from our reality, and back into the entertainment industry away from simulation.

Immersion, “characterized by diminishing critical distance to what is shown and increasing emotional involvement in what is happening.”

Grau, Oliver (2003, p.13). This argument that realism is defined by what is shown is something that stirs debate in academia, with much interest in what exactly is defined as reality within this industry.

Salen and Zimmerman (2003, p.450) argue that the immersive quality of a video game stems from not the actual game, but rather the actual playing of the experience. Referring to this as the “immersive fallacy,”, which they say that “the idea that the pleasure of a media experience lies in it’s ability to sensually transport the participant into an illusionary, simulated reality.”

Immersion is largely relative to the concept of the story I find, getting into a characters perspective is easily doable with the right storytelling, and thus the more realistic to the genre as possible.

An example would be the most recent “Tomb Raider” game. A game where you take the perspective of a young Lara Croft, shipwrecked and completely defenceless on a stormy, tropical island filled with natural disasters waiting to happen from both the fauna and the flora.

The sound design in many sequences is that of realistic bones crunching noises when she misses a jump, howls of pain as she tries to run with an open wound in her leg, with over the top, dramatic orchestra hits when danger rears it’s head.

If this game had Super-Mario-esque jumping bleeps and a musical, happy overture, the game would not be anywhere near as immersive as it is.

“Large screens close to the players face and powerful sounds easily overpower the sensory information coming from the real world, and the player becomes entirely focused on the game world and it’s stimuli.” Ermi and Mayra (2005, pp. 7-8).

This amount of sensory information processed by our brain is vast, and we react depending on what we see on screen. We may have facial expressions when a character dies, or have emotional attachments to the on screen pixels, but that is all dependant on the immersion you perceive from the situation, and it’s basis on either the real world, or convincing enough for us to believe it could be that way in another.

In many ways the realism aspired to by video games is to try not to re-create present day experiences on planet Earth, but rather the non-naturalistic manipulation of that immersion into a cinematic realism.

Walter Murch argues: “This metaphoric use of sound is one of the most flexible and productive means of opening up a conceptual gap into which the fertile imagination of the audience with reflexively rush, eager to complete circles tat are only suggested, to answer questions that are only half-posed. Murch.(2000.)

 

In video games, much like in films, the sound design is never an ‘original’ recorded production-based environment. Rarely is there a raw recording of an actual, real object. Usually it will be a collection of sounds pieced together to create  a synthesised representation, catered and tweaked to the means of the scene. But even ‘realistic’ sounds such as gunshots are more often than not treated to make them feel more ‘realistic’ than they are. Of course this is not true, and in actual fact this process only warps them into what we as listeners would expect from the gunshot, rather than realism outside the genre.

Sound effects are the foundation for inserting this immersion and their construction can be widely varied in process.

In all the techniques and special processes to create the ideal sound effect, the approach is very much the same as in the visual world, more appropriately it can be compared much closer to photography than anything else.

“Working with sound presents challenges not dissimilar to photography. In the process of capturing sound, we have many similar concerns and production techniques available. Microphone selection is similar to the selection of lenses and filters a photographer might use, and your recording device and media might also impart sonic qualities to your source material. Other tools in the process will allow great latitude in manipulating the sound in directions impossible in nature.” Viers, R. (p.1 2008).

The choice of an effect used in a scene is paramount to the overall immersion rating of the experience, and with many techniques staying the same throughout most, if not all films with the same sound.

“In the science fiction films produced during the 1950’s and early 1960’s, the body of sound effects and music tended to emphasise elements that were electronic, mechanical, and ethereal. These effects came to sonically represent the future technologies and environments and the unknown” Whittington (P100, 2007).

The ‘Wilhelm scream’, as it became known. A sound sample used throughout many an action sequence in a film featuring somebody falling from a height, or facing certain death with an iconic scream.

It comes from the 1951 movie, Distant Drums. A small band of soldiers were crossing a swamp in pursuit of Sominole Indians. Whilst wading through the water, one soldier was attacked by an alligator and dragged underwater, his last sound as he died was this scream.

The scream was placed in the Warner Brothers sound effects library, and was used regularly in their studios films, such as ‘Them!’ in 1954, The Swiss Family Robinson in 1960, and so on. With it’s most noticeable use in the first Star Wars movie. This sound effect was used in hundreds of films, and as a result, despite it’s more comedic representation of a situation in a film, still doesn’t manage to detract from the immersion of the film. This could be simply due to the quality of the original sample, and the non-defined purpose of the effect, more simply to describe somebody either being attacked, or falling from a great height in fear.

Sounds like this are commonplace in films, with many overused sounds, such as songbirds, simply used in some circumstances because they are high on the list of sound effects in a library and the sound designer needs an effect in speed, rather than taking time to ensure its exclusivity.

This is why in many films it can be noticed that the same bird calls can be heard, crickets chirping with a summer tone, despite it being winter, etc. This forced reality is expected of the viewer, or the gamer alike, and because it doesn’t detract from the experience unless you are actively aware and attempting to listen out for these re-used sounds, the immersion remains the same.

Another way of maintaining exclusivity and more reliable, ‘realistic’ sound effects, can be to use ‘Foley artists’.

Foley is the “practice of performing sound to picture.” Viers, R. (p.252, 2008.) In very literal terms relates to the use of everyday objects to create noises which can be processed later on to generate an acquired effect.

The idea is that this forced found-sound can be more authentic for the scene within a movie or game, allowing for catered levels of different tones, surfaces and textures to be replicated to particular time scales.

“No matter who’s in the pulpit in the movie, the microphone will always produce feedback as soon as they start to talk. I guess Hollywood has had some bad experiences with live sound engineers and has decided to make them the subtle butt of the joke for these scenes. The reality is that the PA system probably isn’t even turned on during filming- the feedback is added in post.” Viers, R (p.286, 2008)

Sound is generally recorded separated within films. You have a basic guide track, but usually it is covered with noise from outside the scene, cars passing by, distant aeroplanes, other people. This is usually ironed out and Narration is recorded separately on more defined microphones on collar mics, or in some cases recorded after filming in a studio.

The job of the Foley artist is to replicate that scene, and very literally build the landscape around the scene present in the pictures on screen to develop a sound realistic to that environment and genre.

Purity of sound is something that comes into question in video games. Games such as ‘Call Of Duty’ which focus on ‘realistic’ war experiences use a lot of weaponry from the real world, and each gun has recorded sound for each element of that weapon. The issue is that experts in the field of weaponry can tell the difference between a sub-machine gun of American make and another type of Russian manufacture by just hearing the sound of the weapon. Much like a motor enthusiast would recognise the sound of a particular engine over another and a farmer or agricultural enthusiast would recognise the chirp of birdsong to identify the bird that produced it.

This is usually where these ‘realistic’ games come into contention.

Usually the sound sourcing behind a weapon would be sourced from a library, choosing the required sound that fits the scene, rather than making lateral sense for the type of weaponry shown in the pictures. Most often than not this is down to lack of both time and availability to source professionals in that field to acquire those realistic sounds, or in some cases deliberate, as to further distance that particular game from the real world.

“Every type of weapon has unique movement sounds.” Gun movement sounds don’t really exist in the real world. In film though, any time the hero raises his weapon to take aim there’s an associated sound.” Viers, R. (p.301, 2008.)

Again realism to genre comes into effect, even with Foley sound, and many sourced sounds for actions in a film or game will be highly unrealistic to what we would find outside of fiction.

However the recording methods are real, and Foley artists go to great lengths to replicate a sound from genre-specific and previous films and games and make them their own, again catered to the scenes in which they feature.

Military sounds can range from wooden blocks being slammed together for a largely reverb filled gunshot, to that of actual recordings of weaponry in various modes. Dropping the weapons, reloading them, changing firing modes.

Then the environment in which the recorded sound can either be digitally altered in a studio either created using effects software, or merging recorded sounds together.

Positioning of microphones is particularly important in the Foley process, for in almost all cases, perspective can be altered depending on the sounds position in stereo channels, effecting the belief of gunfire from a certain direction or explosions in the distance.

“The sonic characteristics of gunshots are directly related to the environment in which they’re recorded. A gunshot is merely a quick pop. If recorded near a mountain range, the pop transforms into a thick report as it rips across the flat plains, bounces off the mountains, and returns with an echo, giving the sound a nice trail-off feature.”

Viers, R (p.294, 2008).

Usually weaponry will be recorded at different angles at the same time, so the sound can be cut between different angles to help show the progression of a bullet in bullet-time visual effects found in such films as ‘The Matrix’.

5 microphones are usually used for each weapon fire. One faces the way the gun is shooting, one facing the actual gun, one from the chosen side of the person shooting and two on the target being shot, one facing the target to record the hit from the front and one microphone behind the target to get the more bass-driven impact of the shot.

Could Foley sounds themselves be seen as naturalistic sounds themselves? Considering as they originated from natural sounds, and are not simply synthesised impersonations of the real sound, as gaming was forced to use within the 1980’s, in many ways is the use of the natural to make the un-natural sound realistic, but not real.

“From the strange to the supernatural to the gory and evil, horror effects call fro creative performances, unusual props, and a mop to clean up the mess.””vegetables and fruits are often the source for the sounds of mangled broken bones and sliced-up bodies.” Viers, R (p.260, 2008)

Obviously you would not go and literally record the sound of an actual murder for a murder scene, and the Foley artist needs to have an ear for sounds that do not sound out of place in these scenes. Celery is mostly used for broken bones snapping, as the sound is short and cracks like a bone, depending on the speed of the break. The likelihood is a bone snapping will not be as loud in the real world, but again this common belief for what you expect comes into play, and most humans would react with a grimace in that scene as they would be horrified by what would appear to be the actual breaking of a leg in a scene.

The power of recorded sound and foley is much more important for sound design in a video game, as many of the visuals will not be close to representing reality, and the more ‘realism’ you can add to a scene the better the overall immersion will be.

 

Aside from Foley work, much of the remainder of track layout in sound design for a video game shares it’s layout with film and television. There will always be a foley track, background ambience track, narrative and speech, music and effect tracks.

Each element is as important as the one above or below it, as they cumulatively pain the picture for the visuals, and help drive the narrative of a story. Sound is instrumental in both the belief and immersion within a scene, the closer the final mix gets to without discrepancies the less our brain can notice this reality is broken, and thus like on stage, the idea is to never break the fourth wall, unless that is the intentions of the piece.

“No matter what snake is in the movie, you will always hear it rattle. It’s a Hollywood sound staple.” Viers, R (p.216, 2008).

Various effects are used to change Foley, synthesised, and found sounds to give the impression of effects. For example using a low pass filter over narration gives the effect found when you hear somebody talking on the telephone, because the line has limited capabilities and as such runs on a low band frequency, which we accept as the norm. If you were to pick up the phone and hear the other persons voice perfectly it would likely be fine in real life. But if heard within a film would sound off, and completely unbelievable.

Schafer Argues that : “a sound event is symbolic when it stirs in us emotions or thoughts beyond it’s mechanical sensations or signifying function, when it has numinosity or reverberation that rings though the deeper recesses of the psyche.” , Shafer, R.M (p.169, 1977).

It’d true that a sound is a signifier of a certain event or scene, especially so when you hear the branding of a logo, such as ‘Metro Goldwyn Meyer’s iconic lion roar at the start of their films, or the Title credits to ‘Super Mario World’.

Music can be used to convey feeling and emotion to virtually any situation :

“Muzak is probably called elevator music because soothing melodies were used in early skyscrapers to make people feel less nervous about stepping into a contrivance that looked like a death trap.” Owen, D. (2006).

Owen talks how music in early elevators was used to soothe those who used these un-reliable, highly unseen until now devices which could result in your death. Simple melodies that would calm and make the process and enjoyable and everyday as possible. This also drowned out any sounds from the high-tension cables above the elevator and the squeaking of the devices movement up the elevator shaft. This iconic genre of music caught on and even to this day there are companies specialising in themed elevator music, catered to businesses, which in many ways become branding itself for the company.

Iconic sounds to incite nostalgia or signify an action with a sound are extremely prevalent in the real world too. The sound of a phone ringing will always have the sound of a phone ringing in a certain style, unless the setting is of an age where that sound is different, as to ensure that you instantly know that is the sound of a phone, and not a toaster for example.

“if we must be distracted ten or twenty times each day, why not by pleasant sounds? Why could everyone not choose his or her own television signal?” Schafer. R. M. (p.241-242, 1977).

Shafer speaks from a time when telephones shared similar tones and you could not simply change a ‘ringtone’ like we can on present day phones. The point that he is implying that changing the tone lessens the irritation of hearing it interrupting him during his day is unlikely, but it does show that he is psychologically effected and notes the urgency of a telephone ring. The likely hood is that if it had played a more pleasing sound to his tastes, would he not of wanted to instead listen to it rather than answer the phone, and as a result remove the urgency behind the telephones ring in the first place? Some noises are implanted into our subconscious, and any real change to that in what we see and hear in films, television and games is likely to seem out of place.

The effect of distance is also a key factor as to the location of the phone, once we hear the phones ringing and identify it, we then try to determine it’s location, as we would subconsciously in the real world in the aid of answering it.

Various effects with low band filtering can give the impression of a noise from a different room or the other side of a solid wall, different depending on the thickness of the wall and it’s density.

Phasing could be used as a technique to show dissipation of the telephone in a storm, hearing it occasionally dipping in and out of wind or other weather conditions where sound travels differently and is smothered by other effects tracked periodically through the recording. Boat scenes in video games in giant storms such as the ‘Tomb Raider’ game of this year feature the same effect. People try to shout out to the main character, but only some parts of the dialogue reach the player in the attempt to show the intensity of the storm around them even as they raise their voices.

These effects add tension and suspense, even when there may not of been any in the actual scene otherwise.

Voice acting is another avenue of careers within film television and indeed more so in video games. Many actors who have stared in big films lend their talent to a video game characters dialogue, as the familiarity of voice talent can be a selling point for the game, and in turn another way of the actor getting more apriciation and income outside the world of film and television.

However it can be said that the success of voice talent does not simply rely on the person acting within the game, many occasions in recent times have seen re-occuring voice talent, and some games, such as ‘World Of Warcraft’ for example sometimes make the error of featuring conversations between two different characters with the same voice actor. This is mostly accidental, however there have been occasions, such as in the popular game series ‘Mass Effect’ where voice acting talent is deliberately used to cover upwards of 300 none player characters within the game, with slight inflections or post effect processed channels to represent them as a different person.

Many reason could exist for this, mainly budget costs to actively recruit over 300 voice actors is unreasonable for a video game, it would take much more of the budget than would be feasible, so instead voice actors try their best to voice other characters. This is sometimes a factor in video game immersion breaking that we don’t find within television or film, yet in those other medium, there are very rarely more than 6 or 6 main characters with dialogue, with minor parts from extras. Games like ‘Mass Effect’ learn from the rather replicated voice act talent used in the same areas by making the voices more varied in areas, meaning your more likely to hear that guards voice again, but not until the next room, instead of hearing him talk in one part of the room and scream in the other as aliens attack.

Various journals exist as guides to potential developers on ways to cut costs and help make games more viable and successful to an audience.

“Voice actors are usually able to do a number of voices, so using the same voice actor for several different characters is another way of reducing your costs. Of course, it’s worth remembering that asking a voice actor to a large number of characters can affect the quality of the final product, so it’s not advised to go overboard on this one.” Filskov, D. (2013).

Some games are trying to both reduce the need of multiple voices in scenes by reducing the amount of characters within the story, alleviating costs and making the game more ‘realistic’, or simply flow much better with a flowing, un-interrupted narrative, instead of a jolted experience where the audience questions the situations and believability of the scenes in question, breaking the fourth wall.

In an interview with a games journalist website, the developer of the ‘Thief’ games talks about re-use of his main characters voice onto other characters.

“We could have pasted Stephen’s voice on top of the actions and stunts of someone else, but this wouldn’t appear natural. It really wouldn’t make any sense to capture the full performance for our other characters, but not for our star.” Verbert, J, C.. (2013).

Genuine quality of successful voice acting is much more often than not down to the quality of the takes and voice talent being provided with enough information to truly generate that character as their own, and truly represent their traits clearly and as naturally as possible. A monotone recording is unlikely to have the same effect as a passionate speech rehearsed as though it was being performed on a stage. Ways to provide this are obviously host rehearsals of a properly designed script checked and ensured to be what is needed to be rehearsed by the talent, but also in the inclusion of extra material for back up purposes.

“To bring life to your game characters, it’s often a good idea to add a selection of exclamations for each character – even though you might not think you’ll need them at first.
Recording these extra lines don’t add much overhead – but if you later decide to add them in, it may be more expensive and time consuming if voice actors will have to be brought in again to record those additional lines.” Filskov, D. (2013).

In many ways with limited technological advancements, despite animator’s best efforts, voice takes have to be much more involved and believable than they do on live action sequences in films and on television. Voice acting is therefore a much more strenuous profession, as all you hear of the actor is his or her voice, and not the actions which usually accompany them, and many voice acting situations in games, the actor has to imagine what scene will take place and react accordingly with no aids visual or otherwise, only the explanation from the director and the words on the script pages, as the games development will likely more often than not advanced to that part of development.

Usually concept art for characters is drawn up roughly to allow for actors to better get an idea of the character they will be portraying and help them get a better understanding of who or what they are lending their voice to. This helps keep actors in the mindset of what tone and stature many of the characters portray, and prevents issues where higher pitched voices are used on aliens that look as though they would talk fiercely for example in games like ‘Mass Effect’.

The availability of near professional level technology, weather it is software or hardware has allowed for many home-developed games to sound as good as their larger budget counterparts, and in many ways has only extended a new genre within the video game world which used to be riddled with potential failure.

Game development software is now easily available from the larger companies who develop computing engines for the latest games. Companies like ‘Cry-Engine’, ‘Unreal’ and ‘Adobe’ allow for their older developed engines and programs developed specifically for budding beginners in the industry to pay a small charge for a license to use their software, but some have access to powerful tools for free:

“Only Adobe Flash Player and Adobe AIR let you deliver your game to 500 million iOS, Android, BlackBerry, Kindle Fire, and NOOK mobile devices and 99% of connected computers — with no additional install. One codebase. Use amazing tools and take advantage of simple, powerful APIs and frameworks enjoyed by a large and growing game developer community.” Adobe. (2013).

This software, albeit some small costs, with the impacts on the wallet of the developer and team, virtually non existent. Which means costs can be placed into areas of actual game development, Audio deign and indeed improving the quality of the overall game, as half of the work behind how the game works, has already been achieved by one of the most powerful, and expensive systems that has been instrumental in creating the potential inspirations for the games they are making.

‘Unity’, for example is a platform service with allows for both tools specifically designed for professionals or complete beginners to game design to have access to not just the ‘Unity Engine’ which nearly all ‘indie’ games are based, but also various online tutorials, documentation and an entire community of forums and guides to help create the project you desire and have dreamt about from virtually nothing but a basic concept written on a notepad and a basic understanding of the technology and concepts behind the processes involved.

“Here at Unity our mission is to democratize game development. If you’ve arrived at this page you’ve likely already downloaded our Editor, and want to learn all the skills you need to make great games and interactive content.” Unity. (2013).

The service does require a small fee, but the offer of all of those options and tutorials as well as the development platform means that most developers will choose this format over other alternatives, which would require more expensive training and research to create an equal quality end product. And with many open source, and in many cases free platforms of audio software, editors, plug-in’s and even automated voice technology, the vastness of production costs are virtually eliminated. It’s for this reason the gaming industry is thriving, and the non-profit examples shown by companies is huge encouragement for everyone within a career related with the media, as they of course know that those individuals will likely one day work alongside them and only in turn help make there own games more successful, and grow the market as a result.

Independent companies exist and flourish, and through these programs they get noticed and they evolve to new heights and opportunities because of this freedom.

Not only has the quality of the games increased overall through technological advancements in both game engine perspectives, but also the quality of audio through cheap, affordable programs such as ‘Avid’s ‘ProTools’ software, which allows for professional level audio tracking and editing which used to be only available to extremely high budget productions at recording studios such as ‘Abbey Road’ where they recorded ‘The Beatles’ music in the 1960’s, but it is also easier to distribute an ‘indie’ project through services set up by digital distributers such as ‘Steam’s ‘Green Light’ program, which allows for new developers to post their concepts to the world and get direct feedback and support from their audiences and potential investors much more easily.

Another great inclusion which has recently become an international affair, which previously only included the United States of America, is that of ‘Kick-starter’.

This service allows for similar feedback, but also the inclusion of getting the audience of the game to invest in the development costs themselves, and in turn get the game constructed with their feedback and involvement.

Feeback amongst many other offers from these services is much more important than one may originally think, as many games of the past would of benefitted from more audience research, and would of ultimately resulted theoretically less disappointing sequels that came from various game series.

It also allows for an audience to actually create a game they want to play from the ground up a lot of the time. Many opportunities to ask their audience or potential audieces what they think of a change to a certain element of the game, or what they think of difficulty changes, themes, story plot changes, etc. It all basically results in an effective form of additional, free Q&A (Questions and Answers teams: those who test games in-house before releases of actual games.)

This in turn saves companies money with extended testing, and in many cases they offer access to a downloadable ‘Alpha’ version of the game, which they test for free and upload comments and suggestions to the games forums. Allowing for potential bugs and issues with game mechanics that would otherwise mar a successful game launch, without the need of panic at the release of the final product and the costs of patching and actively changing things with a game that has already been released so it is playable, and doesn’t get bad reviews from critics and consumers alike.

Hundreds of success stories are appearing from this new form of funding and has ultimately allowed for sound designers to work on project which have the potential from going from nothing, to becoming a gaming classic, and thus making them a significant amount of money and acclaim in the outcome.

Of course gaming doesn’t invent itself and will continue to thrive off of the success of advancements in other media, and not necessarily of technical nature.

Experimentation with what and audience will find interesting is the forefront of design, and direct influences to older works can only last for so long before innovation is the key to having your product more valuable over the next.

Techniques found in sound design for film and television has always been the progressive parallel to gaming, and it has always followed it. Now we look to the internet for inspiration, as websites such as YouTube have allowed for the everyday person to share their artistic and opinionated contributions on an international scale. Inspiration for some television programming comes from the viewing of these online shows and ventures, as it is the perfect, non-loss environment to test an idea or theory before the eyes and ears of millions of people worldwide.

The world of digitised gaming media and the progression of its sound design is advancing at the same rate as the technology can be developed for it. Much like films and television shows, it shares the talent of these other media to better create the idea of ‘realism’ within a genre and better suit the audiences needs. The simple changes in how sound was recorded from the 1980’s to present day has impacted the seriousness of the games that are developed today. From simple hobbyist ventures like ‘Super Mario World’ where noises are made from jumping around a simple palette of reds and greens in level design, to fully featured Hollywood-esque adventures on movie scale with voice actors of actors and actresses famous for roles in other media.

It’s a world where music has become an advertising tool and a method of increasing revenue and exposure to artists worldwide during an economic struggle in digital distribution in the music industry.

Allowing for changes in that industry to the point of turning a small time bad into a well known figure in society through the use of their sounds in games.

It’s this use that allows cross-bred media ventures which ultimately increase the quality of both video games in breadth and scale, but also the continued succession of the other industries involved in all aspects of the media.

Jobs are born from where there were none before and work opens for visual artists, technicians, support centres, composers, motion capture recorders, and more.

The fact is that an industry in the early 1970’s which was rumoured to be a potential failure extended so far above expectations, that it is, in actual fact one of the largest career paths of opportunity and success in all visual and sonic media, and is now widely accepted as a valuable asset for both advertisement and exposure of hundreds of different talented individuals and technology to support it.

The overall quality of productions is increasing, how much exactly is clear by how many successful games make it from concept to selling millions of copies every month on digital distribution outlets across a multitude of platforms.

That isn’t to say that the world isn’t still competitive. Many small developers and sound design companies related to the gaming media find it increasingly difficult to get a job lined up that will pay, considering the amount of studios and bedroom-savvy technical composers and programmers who will offer there equally perfect services for no fee in return of a chance to generate the experience they need to progress their own careers. The only saving grace here is through a lucky break, as it always has been in many situations cross-media. Weather it be a sound designer for a game or a film, a portfolio and contacts are much more important now than ever, much more so than an educational degree or certificate that certifies your expertise. As ever experience is 90% of the necessary components needed to land a job in this or any environment, and technologies availability has indeed been instrumental in both positive and negative aspects of growth within all media aspects.

This procedure will definitely increase the bar in terms of requirements from prospective employees and employers will have a much varied amount of choice for whom they choose, as the monetary concerns don’t always have to be an issue, as they did in the 1980’s and early 1990’s.

The human sphere of entertainment can only keep growing as technology does, and at the current rate, who knows what technology we will have in 10 or 20 years time.

Perhaps we will develop further into research into ‘3D Sound’, a new theorised format of digitised music. Perhaps we will instead look to ‘retro’ styles of development and make a full circle to create more ‘authentic’ representations of video games from a specific era as we do with Music and recording.

Personally I think that we will see a decline in ‘big-budget’ companies having an monopoly of the greatest titles, and instead with the power they have donated to new independent developers they could be sitting on an equal playing field, where only the contents of the game will be the consumers active choice to make, rather than concerns of quality or experience the developer has.

It’s highly logical to conclude from this that no matter what direction the digital gaming media does take; in essence it will only be towards growth of more cross-media platforms and will only further entertainment levels as a direct result.

The only definite answer to any predictions is that new careers and new media are generated every time it does advance, and in this climate of the media, that is ultimately welcomed beyond recognition by all.

Samuel Thomas Limb (Meoni) meoni@stratics.com.

Hopefully you have found this of interest, i feel as though i may of gone further than most would, but at least you know i didn’t attempt this by any half measures.

I heartidly welcome any comments or feelings you have for either the post or your own experiences and feelings towards the games industry and the way sound design and development is important in your own opinions.

A full reference list of ALL those i quoted or mentioned can be handily found here : Reference List