Get the latest Education e-news
 
  • Student Thesis: Adaptive Music for Video Games

    [07.28.09]
    - David Vink
  •  Foreword

    I have chosen to write about adaptive music because it is something I have often thought about myself while playing games or making music (as it happens, two of my favorite activities). I wondered why, even though it is technically possible (and has been for many years) and can really add a lot to a game, adaptive music is so rarely used in video games. I already read some articles about adaptive music so I could form my own educated guess about why it is so rarely found in games. Still, I was interested in reading more about adaptive music and decided that it would be the subject of this paper.

    I was expecting to find a little more written material on adaptive audio, but it turned out it is quite hard to find good sources on the subject. In the end I had to make do with a number of articles and papers on adaptive music.

    I also wanted to include some simple case studies, but unfortunately I didn't have the means to acquire and thoroughly play any games that have an adaptive soundtrack. 

    I'd like to thank Gamasutra.com for having a lot of informative articles on their website.

    I also want to thank Evert Hoogendoorn for assisting me in writing this paper.

    Preface

    The game industry is usually referred to as a "young industry", and it is true that we probably have some exciting years of growth and development in front of us. Some fields, however, have been developing faster than others.

    Computer graphics have been advancing in huge leaps over the past years, with each new generation of video cards adding new techniques and ways to make games look better or more realistic, and every new generation of consoles increasing the graphical power considerably (although the Nintendo Wii might be said to break this "tradition").

    The field that has been left behind is audio. Although music and sound effects for games have gotten better in terms of audio quality (advancing from simple beeps to fully orchestrated scores), new techniques and ways of using music in games have not developed as quickly as they could (and perhaps should) have.

    Recently, there is a lot of talk in the game industry about adaptive music (sometimes incorrectly referred to as interactive music). Adaptive music is the (long overdue) leap forward in the game audio field that we were waiting for, and I will try to describe its possibilities in this paper.

    The questions I will try to answer are:

    • What exactly is adaptive music?
    • What does adaptive music do that normal music doesn't?
    • What is the difference (if any) with interactive music?
    • Why has adaptive music been used in so little games up to now?
    • How does a composer create adaptive music?
    • Where is adaptive music going in the future?

    This paper is not about sound effects or ambient sounds, though I might mention them from time to time. I will focus only on music.

    What is Adaptive music?

    Some people talk about interactive music and some about adaptive music. Most of the time, they are talking about the same thing, especially when these terms are used in relation to music for video games. However, I think adaptive is a more suitable term in this regard (I will explain why after we look a little closer at what adaptive music is).

    In soundtracks for movies, the music can support the action onscreen perfectly, because it is known from one second to the next what is going to happen. This allows a composer to write music that stimulates the viewers' emotions at exactly the right moments: A startling orchestra bang when a character is shocked by a sudden movement in a scary movie, violins that start playing exactly as two characters kiss, etc.

    Movies are a 'linear' medium, and so a regular, linear soundtrack works perfectly for a movie. In video games, on the other hand, most of the time it is impossible to know in advance what will be happening on screen from one second to the next. You can never tell exactly how many seconds it will take a player to pass through a room, or when exactly the player will fire his gun and kill an enemy. Using a normal soundtrack means you can only try to catch the general feeling of an area, level or gameplay experience with the music. The music will not be 'synchronized' with the events on the screen. Much of the emotional power of music is lost that way.

    In comes adaptive music.

    Adaptive music is a unique appearance in video games (and possibly some other interactive media). It suggests that the music responds appropriately to the events taking place in the game. Imagine, for example, walking through an empty building at night. The music is very calm and has some suspense. While you are walking, you unknowingly move closer to a certain room in the building, and the music gets more eerie and dark. You finally find a door that you have to pass through to advance to the rest of the level, but the music is now so spooky you know something is waiting for you on the other side. Then, as you move towards the door to open it, two monsters jump at you from the shadows on your sides. Now the music becomes upbeat and dramatic, as you defend yourself as best as you can against these attackers. After dispatching them, the music becomes calm and soothing while you have time to take a breather..

    As you can see in the above example, instead of just using one suspenseful track for the whole level, using adaptive music enhances the impact of the events in the game just as a movie soundtrack does for a movie. From a game designer's perspective, adaptive music can be a tool with which to communicate to the player of a game, just like the graphics. The player may learn, for example, that when a certain type of music starts playing it means there are enemies nearby. Even if the graphics are available and being used to communicate a situation, adaptive audio can be a means to enhance that communication with an audible dimension, and steer the players' emotions in the direction you want. The experience of walking through a dark forest at night will be completely different if there is scary music playing then if there were soothing music playing, for example.

    Composer Guy Whitmore uses the following analogy to describe adaptive music:

    "In a sense, linear music is to pre-rendered animation as adaptive music is to real-time 3D graphics." [1]

    Instead of creating a 3-dimensional movie and playing it back for someone, a 3d game creates the 3-dimensional world onscreen in 'real-time' while the player is walking, racing, etc. around in it. Adaptive music does this with the game's music. Instead of creating a linear score and playing it back to the player, the music is created or (put together) while the player is playing the game.

    Of course, having a game play random pieces of music on and off, alone or together, loosely based on changes in the game state can not be called adaptive music (instead, I would call that interactive music). The music has to fit the narrative of what is happening in the game and add to and expand the visual representations of those events.

    There are several ways to create adaptive music, but choosing an adaptive soundtrack for a game always means the music will have to be created differently from linear soundtracks. Instead of writing a regular song, short pieces and/or layers of music (percussion loops, different melodies and instruments) must be created as separate audio files, while fitting together perfectly so they can be played at the same time or seamlessly (without a pause) one after the other, depending on what is going on in the game. These pieces can vary in length from a single note to several minutes, depending on how much adaptability the soundtrack for a specific game needs. Guy Whitmore calls this the spectrum of adaptability:

    "On one end of the spectrum is linear pre-rendered music, and on the other is music that is completely game-rendered." 1

    The definition that follows out of the previous text is this one: Adaptive music is music that responds appropriately (or adapts) to the events in a game.


    [1] Guy Whitmore , Design With Music In Mind: A Guide to Adaptive Audio for Game Designers (Gamasutra.com, 2003)

Comments

comments powered by Disqus