Classical Convert

Score Processing, Part III

Here is the latest product of my epic battle to nicely animate a score to music:

(widescreen here)

The motivation here was to get a program to work out when notes were being hit, just by looking at the volume. The idea is that when the volume suddenly increases, a note is being played. If I can work out exactly when all the notes are being hit in a recording, then I can map the analyzed score (that is, the raw notes from the sheet music) onto the timings, and make a neat animation.

The video shows a moving plot of the volume. Well sort of. It’s the absolute value (i.e., all negative values are made positive) of the amplitude of the waveform. You know how when you look at a speaker cone real close you can see it vibrating? The amplitude tells you how how far it is moving. If the amplitude is large, it is compressing a lot of air, which sounds loud. If the amplitude is very small, the speaker is barely moving, and we can hardly hear anything. This doesn’t exactly correspond to what we hear as volume, because there are lots of psychological effects which affect our perception (for example, hearing low and high pitches differently).

As you can see, it works pretty well when not much is happening, like during the first few minutes. It is interesting to see how the sound slowly drops off after each note is struck. When things get more hectic it gets way harder to separate the notes, since the sound level is continuously fairly high. Quieter notes get lost in the sustains from previous ones.

Imagine hitting a low C on the piano and then immediately hitting a high one, much more quietly. We could probably hear both because they have such different frequencies, despite the volumes. But if instead of hitting the high C you quietly hit the low one again, it would be really hard to hear. That’s pretty much what is happening here. We are not using any of the pitch information.

There is a way to do that, but it is much trickier to program.

Tags: beethoven, classical music, score analysis, visualization

Score Processing, Part II

No-one exists here. We are still within the midst (or midsts? Which is it? Well, given that it is from the 15th century I guess I can get away with either. That’s how they used to roll.) of thanksgiving. No-one is here. Except me. I think my supervisor would have murdered me if I left this weekend, given that I was skipping around NYC for most of last week.

I did have some time to play around a bit more with the score processing. Here’s my standard guinea-pig type piece, Beethoven Op. 111:

Do you love the glorious widescreen? Oh wait… the embedded player doesn’t work with widescreen yet. Well, if you (like me) are hot for 16:9 you can watch the full thing here. HOWEVER. It still won’t have any audio. Why not? It turns out that synchronizing the audio is actually the real crux of this score analysis malarkey.

You see, the notes in the video are a literal transcription of the score (extracted from MusicXML versions of MIDI files), but no-one ever plays a literal transcription of the score. The tempos vary all the time. So making the audio match the notes is a much more difficult problem than getting the notes themselves. I have about five different ideas for getting this to work, but all of them are several day long programming sessions.

Still, the video looks kind of pretty, right?

Tags: animation, beethoven, classical music, score analysis, youtube

Awesome Beethoven Transcriptions

Guys! After all of the mentions of transcriptions in the last week, and JonJ and Yvonne commenting that there is a well established history of transcripting stuff, I got off my internet arse and discovered several very joyous things. Firstly, the clavier-wunderkind Liszt did piano transcriptions of all of Beethoven’s symphonies — this is probably exceptionally common knowledge to all the seasoned classical listeners out there, but news to me. Secondly, there is a box-set of these available on Naxos, which is itself available on eMusic (see: cheap, good quality, DRM-free MP3s). Hooray!

So far I have listened to the ones I know the best: 6, 5, 9, 7. Only once each so far, as it all just got downloaded about an hour or two ago. Aside from noticing a bunch of stuff which apparently had been completely obscured by my ears during the myriad previous symphonic listens (key changes and modulations seem way more obvious, for example), it’s striking how similar certain sections sound to the (Beethoven) piano sonatas. I don’t yet know if that’s due to Liszt deliberately orchestrating them like that, or if it’s pure Beethoven shining through… or if it’s me trying to be clever. I’ll try and keep you posted on that one. Hopefully it’s the middle one.

Tags: beethoven, liszt, piano, symphonies, transcriptions

Windows Hearts Beethoven

The latest Windows update refuses to install if it senses a lack of Beethoven’s 9th:

Thank you Microsoft for insisting that Beethoven’s MOTHER ********** NUMBER 9 NEEDS TO BE INSTALLED ON MY MOTHER ********** COMPUTER before Service MOTHER ********** Pack 3 can be successfully installed.

Your programmers are morons. Go to hell, the lot of you.

If Windows prefers Beethoven, I bet baroque old Linux likes the intricacies Bach better. What about Apple? Somebody who has form as an utmost priority I suppose… Webern? Nah, that doesn’t feel right. It has to be universally accessible as well. Any suggestions?

Tags: apple, beethoven, classical music, composers, linux, windows sp3

Picking up the pieces

In my last post I wanted to see if anybody could guess a particular classical piece from hearing a single note, the single note in question being:

Two listeners (Miss M. and ACD) impressively pinned this down to Beethoven almost immediately. Given just a slightly longer rendition of the sample resulted in a correct guess of the particular piece, the part in the presto of Beethoven’s 9th in which — as another commenter, Mitch, described — “it goes from brooding to joyous”.

I find it really interesting trying to determine at which point a series of notes changes from being, well, just a series of notes into being a universally recognizable melody. Everyone reading this probably has a certain section of their brain reserved for that particular theme, but when listening to music when does your brain kick in and scream “I know this!”?

It’s probably not quite here:

Or here:

But maybe now?

It’s particularly fascinating that after listening to the whole theme:

if you go back and listen to the short ones they seem a lot more obvious. Apparently, consciously knowing which music is about to be played makes your brain give that sharp recognition response with far fewer notes than would ordinarily be needed. Obviously it’s not terribly surprising that you can “recognize” a piece from a single note when you are told what that piece is going to be, but it isn’t just consciously recognizing a piece. It’s not just like someone telling you “you are about to hear the Ode to Joy theme” and you listen to that one note and confirm that indeed, it is the beginning of the theme. When you hear that single note again it is a primal feeling, a low level blast of recognition. Once you know what music the note is from it is almost impossible to avoid that recognition. It’s like trying not to read writing: once you see a written word you instantly identify it with a concept, it’s impossible to see it as just a bunch of lines.

Listening to a new piece of music is sort of like learning a new language. You initially get faced with all these sounds and melodies which you can sort of follow, but don’t have their own bit of brainspace. If you hear the notes which make up the melody one by one, there probably would not be a sudden moment of recognition, they are all just notes because you do not yet understand the piece. They are like a foreign word, which you understand is a word, but you do not have any mental object associated with it. If it were spelled differently you wouldn’t notice. It’s non-meaning would not be affected. In the same way you probably couldn’t tell if one of the notes in the music were altered.

Eventually you get to the stage in which those melodies provoke an instant response. The words make sense. The music has been imprinted into your head in such a way that it has become a piece of you. You can hum the melodies.

I can sort of feel this process happening sometimes. When a piece has first managed to burrow into my head and stake out a bit of territory for itself, I will sometimes have stress dreams in which it is endlessly looping in the background. It’s an almost unpleasant experience, but it’s also almost a given that this music will then become one of my favorite pieces, so all the not-quite-sleep trauma is totally worth it.

Tags: beethoven, classical music, understanding music