Game Career Guide is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Get the latest Education e-news
 
  • Student Thesis: Adaptive Music for Video Games

    [07.28.09]
    - David Vink

  • How is adaptive music created?

    So how does one create adaptive music that suits the needs of the game design? There are several ways of creating adaptive music that have already been tried and tested, and there is a lot of room still for new techniques. There is one thing, however, that is always important in the creation of an adaptive soundtrack:

    Communication

    For a regular soundtrack, sometimes the composer only needs to know the general feeling and idea for the different tracks and can then go on to compose the music on his own. With adaptive music, however, the composer must keep in mind the details of the different events and states in the game and what kind of changes in the music they trigger. Depending on the game design and the game's musical needs, a different way of creating adaptive music may be needed (think about the spectrum of adaptability mentioned in chapter 1). A close collaboration between the game designer and the composer is therefore absolutely vital for the creation of a good adaptive soundtrack.

    The game designer often has general ideas about what kind of music should be used for what situations, and the composer can add more details to that vision, adding more specific ideas to the audio design (about what kind of music works well to convey certain moods to the player, for example). Together these ideas result in a detailed audio design, which can also be used to communicate to the programmer(s) what kind of events and states in the game are going to trigger changes in the audio.

    Communicating with the programmer(s) (responsible for the audio engine) is obviously very important as well; there are different ways of creating adaptive music in a game and they require different coding.

    The programmers will need to know what events and states of thing in the game are important to the music. If the music needs to change when the player's life is lowered, for example, the game code needs to send a signal to the audio engine when this happens. With complicated adaptive music, there can be hundreds of states and events in the game that influence the music, and the programmers need to know exactly what they are.

    Ideally, the programmers will create a piece of software (or sometimes an existing program can be used) that allows the composer to easily edit what happens to the music under what circumstances, without having to change anything in the actual game code. This software should give the composer access to all the different states (and state-changes) in the game, and allow him to set for each of those states whether and how they will influence the music.

    For example: In the options for the state "Player's life drops below 50%" the composer is able to set that a new piece of music starts playing, and/or that a piece that is already playing should have it's volume raised, and/or that at the end of a current piece there should be a transition to a new piece, etc.

    Finally, the composer should be able to continuously test the adaptive music in the game to see if it has the intended effect, changing the music and the states to which it responds where needed, and then immediately testing it in the game again.

    Currently, the IASIG (Interactive Audio Special Interest Group) is working on a piece of software that will allow composers to edit their adaptive music in the way just described, called IXMF (Interactive eXtendable Music Format). The interesting thing is that this software will be platform-independent, which means it can be used to work on soundtracks for PC games, Xbox games, Playstation games, etc. (more on this in chapter 5).

    The walls between composers, game designers and programmers are slowly disappearing as adaptive music is embraced more and more by game developers. While with a linear soundtrack the composer only needs to know what the general themes for the music are, to create an adaptive soundtrack the composer needs to know how the game is going to be played (what the flow of the gameplay is like). At the same time, he also needs to understand how game engines will handle the different ways of creating adaptive music. That is not to say composers should study game design or that programmers should study music composition, etc. The idea is that everybody involved with the music in the game will have to work together closely in order to make an adaptive score work well.

    Digital audio or MIDI?

    Normally digital audio is used to create game soundtracks, but MIDI used to be very common as well, and it is very suitable for adaptive music. Both have their pros and cons.

    Wave files are full-quality digital audio files, which can be compressed to formats such as MP3 or OGG to save disk space (at the cost of some quality). The good thing about wave files is that they can come from any source, meaning you can use live recordings of an orchestra, home-made electronic music, recorded voice, etc. However, the music in a wave file is fixed, and can not be altered anymore. This means that to get a high level of adaptability with wave files you need to create a lot of separate files for different parts of the music (such as several string parts, different types of percussions, etc.). This means the size of the audio included with the game will increase, and that can be a problem for some games (such as downloadable games).

    MIDI-files do not contain any actual audio. They only contain information about when to trigger notes for different instruments, and about changing the pitch, volume, velocity and a lot of other parameters for notes. MIDI files are extremely small (several hundred times smaller than a MP3 file), so handling a MIDI-signal requires hardly any disc-reading or calculations from the computer.

    The actual audio comes not from the MIDI-files themselves, but from separate audio files. Devices such as Sony's Playstation or Nintendo's Gamecube have build-in synthesizers that can be triggered by MIDI files, so instead of including a piano song with a game, it is possible to instead include a MIDI-file and use the devices' piano synthesizer to play the song. If the sounds available on the platform are not sufficient for the soundtrack the game needs, digital instruments (a collection of samples that can be played back by a MIDI file) can be included with the game.

    MIDI makes it possible to create music that is adaptive on a very detailed level (as the game can use the MIDI signal to trigger single notes and change the pitch of a note, etc.). This makes MIDI especially handy for adaptive musical events that need to respond to gameplay fast. If a dramatic string chord must be heard when the player shoots a monster, for example, the audio must be heard before the visual cue (the dying monster) is gone.

    Timing the adaptive musical events and transitions is also a lot easier with MIDI because the number of beats and tempo of the music can be included in the signal.

    If there is not much adaptability needed in a game, fading between a few digital audio files may be good enough. And if the build-in synthesizer on the platform has good enough sounds, there is no reason to create your own audio files.

    But in all other cases, combining MIDI and audio files will give the best results. One or more audio files can form the basis of the music (for example: a slow ambient track that constantly plays and some percussion that will fade in when enemies appear), and MIDI can be used to add adaptive detail to it (adding for example a piano melody that intensifies with more notes and chords as the player loses more and more life).  To save disk-space, the digital instruments available on the platform should be used as much as possible. Custom samples can be included with the game when necessary.

Comments

comments powered by Disqus