Archive for June, 2013

Anatomy of a Procedural Music Engine

June 15th, 2013 Comments off

At IRDC last week I gave a presentation on the procedural music in Mosaic, and it’s about time I put it online. I’ve also updated the game itself a bit, with WAY better sounding music and some enhancement of the player powers. You can find the latest versions here:

Here are the slides from the presentation itself (pdf, 1.68MB). But they won’t make much sense on their own, so here’s some notes to support it:

  1. Introductory slide with the @ symbol / treble clef combo that was at the heart of Mosaic.
  2. Proteus was an influence with its procedural music. In particular it made me want to make a game that was beautiful.
  3. Tonematrix was another big influence, a sequencer you can play in your browser. The grid and the on/off switches made me immediately think of a roguelike where you move around the space and control it. Hence Mosaic!
  4. So I had this idea of music based on the grid, and the player wandering about, and it fused with a bunch of others ideas about creativity and procedural art, and so Mosaic was born! I spent most of the week on the look and the mechanics, but come the last day I knew the time for music was near…
  5. …But I didn’t know anything about music. So, with about 7 hours left before the end of my 7DRL week I went to Twitter and asked the important question – how the feck do I make music?!
  6. Michael Brough, magical indie dev extraordinaire, responded in good fashion with a hint towards the “pentatonic scale”.
  7. To Wikipedia! Except none of this musical terminology makes sense…
  8. Some further reading down the page and I’m still confused, but I pick out a string of notes – ACDEG. I think right, I’ll try these notes out!
  9. But first I tried a bunch of piano notes and shoved them into the complete game engine, copying Tonematrix pretty exactly.
  10. Er, it worked, but it didn’t sound fantastic. The notes didn’t go well together, it all sounded messy. Tried a bit more with xylophone and harp, and there was some improvement. So, new idea, how about a whole orchestra?
  11. Firstly I got a load of notes from the Sonatina Symphonic Orchestra – a really great resource! Creative Commons notes for almost all instruments – here’s the torrent file.
  12. Next I looked up Pachelbel’s Canon in D.  Y’see it wasn’t an entirely new idea, I’d thought before that an ultra cool thing to do would be the Canon in D procedurally. This was a more advanced idea than just a Tonematrix clone, and now was the time to try it out. The actual music terminology was beyond me, but I looked at the note sequence and saw how they moved in wavy lines. So, right, wavy lines, lots of notes – can’t be that hard, right?!
  13. I assigned instruments to different lines of Mosaic, with the trombones as enemies. There’s a deliberate set up to this – harp and bass are at the top and bottom as these tend to get turned on a lot. The brass notes for enemies are deliberately distinct from the strings from the grid. This isn’t just random, it’s procedural!
  14. And wow, just wow, it sounds good! I was amazed, truly amazed, that with bugger all knowledge I had quickly managed to cobble together something that worked and sounded pretty decent. I got different feedback on this, mind – some people said it sounded fantastic, others that it was utterly awful. Turns out it depends very much on how you play, which is in itself a cool feature!
  15. Now the code, which is beautifully simple. This has been reorganised since 7DRL week but is basically the same in function. Every 0.25 seconds a tonal_shift is set – which way the notes should go. This is balanced towards small shifts, but can have a whole 5-note shift on the pentatonic scale (so a whole octave). The code looks along the current column, and on each tile if level = 1 (meaning a coloured tile) then it plays the corresponding note for that tile – self:playNote(i,1) (the 1 just means volume = 100%). It does the same checking for enemies and plays trombone notes if present – playNote(11,1). The pulse references are just for graphical effects to show notes being played. The playNote() function, not shown, decides whether that instrument follows the tonal_shift or goes its own direction, and does some admin-type stuff on pitch changes to meet the pentatonic scale – this is much better now than on the original much more note-limited 7DRL.
  16. Since the 7DRL week I’ve had lots of ideas for procedural music tied to gameplay. Sounds that are based on the environment and your interactions and events in the game. Sounds that are altered in a number of ways based on the numbers in the game. Roguelikes are especially suited for this, as using plain mp3s gets really repetitive. There’s a lot of potential here! I hope to explore this more in future.
  17. Lastly, some advice I’ve received from various sources, and a few things I found out myself – especially the last point. Music really is a deep rabbit-hole to dive down. If possible avoid this and fake things as best you can!

I wrapped up by then playing a bit of Mosaic live, which seemed to go down well enough.

I hope some of this is of interest and helpful to others! Was a lot of fun, and it’s still cool to muck about with. Rogue Rage in particular has a lot of potential for application of procedural music. Have a go yourself – it may seem intimidating, but it’s very cool when you get it working!