Pesky Harmonics - Spectral Visualisation of Audio

Posted by Tom on 2012-03-01 22:17

Whoops. I got a bit distracted and I forgot I was sitting on this one.

A few weeks back I was thinking of ways to visually represent music and the most obvious one was frequency over time. Pesky Harmonics is the result and it takes an MP3 file as input and spits out a PNG showing the time along the X axis, freqency along the Y and with amplitude as the pixel brightness.

Here's a few samples (click the image to listen to the tune on YouTube):

Guitar intro to Better Living Through Chemistry by Queens of the Stone Age

Only by Nine Inch Nails (about 2:15 in)

Cosmonaut by At the Drive In (around 1:33 - the two dark patches at the bottom are where the guitar and bass plays rests)

It's a bit noisy and there are quite a few restrictions on the input format (a bit-depth of 16, 2 channels - one of which is discards completely). The result is also ripe for some improvement (the brightness needs tweaking, the resolution is pretty crappy at low frequencies, I'd rather have a logarithmic frequency scale) but it's possible to make out some sonic features and I imagine I'll keep chipping away at the idea.

As usual, the smart parts are provided by better people than I. I'm using libav to load the audio file. FFT is provided by KissFFT. Image writing is handled my libpng.

And also as usual - it's available on GitHub.