Game Career Guide is part of the Informa Tech Division of Informa PLC

This site is operated by a business or businesses owned by Informa PLC and all copyright resides with them. Informa PLC's registered office is 5 Howick Place, London SW1P 1WG. Registered in England and Wales. Number 8860726.


Get the latest Education e-news
 
  • Student Thesis: Adaptive Music for Video Games

    [07.28.09]
    - David Vink

  • Transitions

    To create an adaptive soundtrack, transitions between different pieces of music are needed. These transitions can be implemented in different ways:

    • Silence: One piece of music ends in silence and then the next piece begins. This can hardly be called a transition and although it works in some cases, it will often give the soundtrack a fragmented feel, which in turn has a negative impact on the gameplay experience.


    Silence Between Tracks

    • Cross fading: One piece of music has its volume lowered to zero (called a fade-out) while the next one has its volume raised at the same time (a fade-in). This will create a seamless transition, but unless the pieces have matching endings and beginnings, a cross fade can be heard clearly.


    Cross Fade 

    • Direct switching: At the end of the current measure or beat, one piece of music stops abruptly and the next piece starts at the same moment. This works best with simple, electronic music, and only if the music is specially made with this type of transitions in mind.


    Direct Switching 

    • Layering: The music is built in multiple layers. In an orchestral score, for example, each instrument is played in its own channel. These 'layers of sound' can be added and removed adaptively, using fade-ins and fade-outs and cross fading. The soundtrack will have a feel of continuity to it, as transitions are never obvious to the player. The downside of layering is that it is very difficult to make sudden dramatic changes in the music while the game is being played.


    Layering 

    • Transition matrix: A transition matrix allows a game to select a suitable transition between two pieces of music from a matrix that includes transitions for all possible combinations of music in the game. All these transitions have to be created by the composer. This can be very challenging, as the music must really be viewed as circular and continues, instead of linear. In the illustration below a transition between track 1 and track 2 is used, but transitions between track 1 and 3, 1 and 4, 1 and 5, etc. may also occur in the game, and therefore transition parts for these situations must also be created. And the other way around, a transition between track 2 and 1 may also be possible in the game and must also be created. Although it can be a real challenge to create a transition matrix, it does allow for sudden adaptive dramatic changes in the music (the transition between a calm and an exiting piece can be just a second long), while still making the soundtrack feel like a single musical experience.

      
    Transition Matrix

    New jobs in adaptive music

    As new production processes appear to create adaptive music for the next generation of video games, new jobs and job descriptions in the game audio production and design field are needed. The following tasks are essential for the creation of adaptive music:

    • Audio designer: The audio designer oversees the design and production of the sound and music for a game.
    • Composer: The composer writes the actual music.
    • Interactive arranger: The arranger turns the music into audio files that are suitable to be used in a game. Sometimes a composer will just deliver recordings of an orchestra, and the arranger will then have to cut and edit them into the smaller files needed for the adaptive soundtrack.
    • Audio programmer: This is the programmer that implements the audio into the game.

    These are the tasks, not necessarily the number of people. In smaller development teams one person can take on several of these tasks (and related tasks such as designing sound effects).

    Finally, some important things to consider when creating an adaptive soundtrack (or a regular one, for that matter):

    A video game soundtrack should have a common theme or idea, something that ties the whole soundtrack together.

    There are a lot of situations in games where it is better to have no music at all (only ambient sounds). This will make the parts that do have music stand out much more and really have impact on the players' emotions (which is what you want to use music for in the first place).

    If you spend a good deal of time designing the soundtrack and players turn down the volume it means you did something wrong.

    The Future of Adaptive Music

    So what does the future have in store for game designers and composers working with adaptive music and for the gamers that will buy their games?

    At the moment there are several projects in which composers and game designers are working towards developing new tools for composing adaptive music. There already is a tool by Microsoft called DirectMusic which gives composers the ability to set in detail how a set of sound files should be handled by a game, but it's a very complicated program that composers without any knowledge of coding will find difficult to use.

    Very promising is the development of IXMF (Interactive eXtensible Music Format) by the IASIG (Interactive Audio Special Interest Group). This is a format for storing all kinds of media files (wav, mp3, MIDI, etc.) along with information about how and when those files should be played back in a game. IXMF is a cross-platform format, which means it can be used on different platforms, such as the Playstation or the PC.

    A tool like this can make creating adaptive scores for games much easier, as the composer can use the programs of his choice to create the music and include information about how and when the files should be played back and mixed together, and the music can then be played correctly by all systems. This will save a lot of time in the production of cross-platform games.

    It is always difficult to get competing companies (such as Microsoft and Sony) to accept the same standards for their platforms, but the IASIG is a group of respected game audio veterans, so they may very well be able to make the IXMF-standard a reality.

    Game developers and publishers are still hesitant to allow for more budget to create adaptive music, as they are not yet convinced it will help increase the sales, and in their eyes creating a game is ultimately about making money. This is likely to change in the near future, though, as game critics (and consumers) are beginning to recognize the value of adaptive music. The quality of the level of entertainment video games can offer is rising as technological possibilities are expanded, and all available means are used to draw players into a game emotionally: professional storywriters and actors are hired, graphical standards are raised, (online) multiplayer possibilities are expanded, etc. In the same way, better ways of integrating music in games are needed, and adaptive audio is one of the techniques that can do this (and could for many years already, but simply wasn't recognized or appreciated as such).

    As for the people that buy and play the games: They can expect to play increasingly more games that offer a much more immersive aural experience, heightening the emotional immersion and making the game more enjoyable altogether.

    Sources:

    Bernstein, Daniel, 'Creating an Interactive Audio Environment', Game Developer, 1997
    URL (June 2006): http://www.gamasutra.com/features/sound_and_music/111497/audio_tips_05.htm

    Bridgett, Rob, 'Interactive Music', Computer Arts Magazine, 2002.
    URL (June 2006): http://web.archive.org/web/20021203015700/http:/www.sound-design.org.uk/interactive.htm

    Cambridge University online dictionary.
    URL (June 2006): http://dictionary.cambridge.org/

    Fay, Todd M, Selfon, Scott, Fay, Todor J, DirectX Audio Exposed: Interactive Audio Development, Wordware Publishing, 2003

    Holland, Norman N, 'The Willing Suspension of Disbelief: A Neuro-Psychological View', Psyart, 2002
    URL (June 2006): http://www.clas.ufl.edu/ipsa/journal/2003_holland06.shtml

    Interview with Guy Whitmore (By Alexander Brandon), IASIG, 2002
    URL (June 2006): http://www.iasig.org/pubs/interviews/guy_whitmore.shtml

    Jongh, Reinoud de, 'Rillingen over je rug: Hoe muziek ons brein bespeeld', Psychologie magazine, January 2006.

    Law, Linda, 'Introducing the Interactive XMF Audio File Format', Gamasutra.com, 2003
    URL (June 2006): http://www.gamasutra.com/resource_guide/20030528/law_01.shtml

    Merriam-Webster online dictionary.
    URL (June 2006): http://www.m-w.com/

    Miller, Steven, 'Producing Interactive Audio: Thoughts, Tools and Techniques', Game Developer, 1997.
    URL (June 2006): http://www.gamasutra.com/features/sound_and_music/111497/interaudio3.htm

    Ross, Rob, 'Interactive Music...er, Audio', Gamasutra.com, 2001
    URL (June 2006): http://www.gamasutra.com/resource_guide/20010515/ross_01.htm

    Schmidt, Brian, 'Interactive Audio Development: Where the Heck Do We Go From Here?', Project Bar-B-Q speaker, 1999
    URL (June 2006): http://www.projectbarbq.com/bbq99/bbq99bs.ppt

    Whitmore, Guy, 'Design With Music In Mind: A Guide to Adaptive Audio for Game Designers', Gamasutra.com, 2003
    URL (June 2006): http://www.gamasutra.com/resource_guide/20030528/whitmore_01.shtml

Comments

comments powered by Disqus