| Subscribe via RSS

Realtime Liner Notes: Visualizing Music

December 5th, 2007 | No Comments | Posted in classical music, liner notes, youtube

Or should that be Living Liner Notes? (thanks Miss M!)

When YouTubing this post about Bach on the accordion, this visualization video popped up in the “related stuff” sidebar:

Which is very nicely related to all of the understanding music via automated liner notes I have been incessantly harping on about recently. Some thoughts on this particular instance of visualization are:

  • A surprising number of patterns jump out at you when you can clearly see the basic relationships between notes being played - but in an intuitive, not mentally analytical fashion.
  • It takes a lot of work to produce something that both sounds decent and is visually clear.
  • People are fascinated by this - there are almost 600,000 views of this video.

Now what would be really great is something which can suck in an MP3 and spit out something like this. That’s a fairly lofty goal, but I’m going to play around a bit with visualization methods and see if I can anything remotely convincing to work. I’ll try and document my progress on here, starting with the most basic, trivial visualizations.

But perhaps I’ll wait until after my Friday biochemistry final to get started…

Realtime liner notes: More thoughts

December 2nd, 2007 | 3 Comments | Posted in classical music, dg, liner notes, mp3, naxos

A couple of days ago Matthew Hodge added a comment to this post about my automated liner note fantasies which I think elucidates several important points, so I’m gonna give it a post all to itself right here:

At the risk of offending ACD (and I know there are many people who hold strong opinions about all this), I think the Concert Companion still is a fantastic concept and (if done properly), I believe would totally open the world of classical music to new people.

I know this because a couple of years ago (long before I knew about the Companion or anything like that), I tried an experiment where I invited round to my home half a dozen friends who knew nothing about classical music but were willing to have a listen.

I handed out sheets that explained the music and broke it down in sonata form, themes, etc. with descriptions matched to particular times on the CDs we were going to listen to. I wasn’t sure what to expect the first time, but to my amazement, I watched as this bunch of novices not only grasped the form of the music, but were absolutely riveted listening to an entire overture, Mozart violin concerto and Beethoven symphony. Half an hour long pieces, and they followed the whole thing!

Let me tell you, it is the most exhilarating thing in the world to see someone’s eyes open to music.

I think the biggest problem that we face in reproducing this in the concert hall is not the technology but:

a) Existing audiences (who’ve been fortunate enough to learn music, etc.) get very particular about what goes on in concerts and usually aren’t keen for new things. (e.g. “What’s those flashing lights on in the concert hall? Highly distracting.”) Granted, these people would have killed off surtitles as well, and in my opinion, they’re the greatest thing to happen in opera for years.

b) You can only explain so much music while it’s on the go. Really, you need about half an hour beforehand to explain to people the big picture of musical structure (sonata form, movements, etc.). Ideally, this would be perfect for a pre-concert talk. But how many good pre-concert speakers are there nowadays? Not many.

c) Far more serious, there is a severe shortage of writers who can explain music to the layperson in an interesting way, so even if you had a running commentary, would it make the music exciting or just boring. (For examples of how to make music exciting, I suggest reading George Groves’ “Beethoven and His Nine Symphonies” or David Hurwitz’ “The Mahler Symphonies: An Owner’s Manual”.) If those guys (well, George has been dead for 100 years) wrote for the Concert Companion, it would probably still be running today.

I think he’s bang on. Discovering the previously (to me) hidden structures of classical pieces completely knocked my uninitiated little socks off when I first discovered it, and I’m pretty sure a lot of other people would feel the same way. However, it’s also clear that there are a bunch of problems doing it in the concert hall: experienced people are likely going to get pissed off by the blinking whatevers, and for inexperienced people it’s probably going to go by too fast. I think the answer is the internet. Something like keeping score, but with many more pieces and more options. I think if one of the online classical music suppliers put something like this together, executed well, it could be very profitable for them. Do you hear me DG? Naxos?

This is just the kind of thing I meant

November 14th, 2007 | No Comments | Posted in classical music, liner notes

Zoltan gave me a fantastic link to the San Francisco Symphony and Michael Tilson Thomas project called “Keeping Score”, which is almost exactly what I described in my last post on automated liner notes. Just click on the “Explore the Music” link when you get there. I strongly encourage you to take a look (and subsequently waste a whole bunch of time there). Now if only they did it for more pieces, and did it for the entire piece!

More Thoughts on Automated Liner Notes

November 12th, 2007 | 3 Comments | Posted in beginners, classical music, liner notes, mp3, theory

A few days back I wrote a post in which I longed drastically for a form of real-time liner notes. When you play an MP3 or CD on your computer you could click a button and choose from either the standard background information you get with the CD inlay, or a much more exciting commentary on the piece which updates as the music progresses. In its most simple form it would indicate when a new theme enters, or an old theme comes back, or when the exposition ends, for example. You could also choose a more detailed “expert” version which would give a more detailed analysis of the structure of piece: keys, inversions of melodies, etc.Two things became clear after some excellent responses to the post. One is that (somewhat unsurprisingly) this has (sort of) already been thought of. Multiple times. ACD brought my attention to the (fleeting) existence of the Concert Companion, a handheld device which one was given to use during a live concert: “the Concert Companion delivers explanatory text, program notes and video images in real time with the music”. Hmmm. That sounds like a bunch of people playing on the internet during a concert. Yvonne told me that in Queensland they tried something similar but it was actually projected on the walls of the concert hall.

Neither of these sound like a particularly great idea. The idea of a non-optional thing is just horrible for a start. If there is going to be an optional thing then I don’t think it should be an alternative stream of information to the concert at hand, which is kind of hard to pull off in a live setting as you tend to be watching the performance. Another problem is that in a concert-hall setting it’s probably not going to be particularly appealing to any experience level. I would imagine that unless it is extremely well designed, a listener unfamiliar with the piece will be given too much information, too rapidly, whereas a listener who knows it well will probably find it an annoying distraction.

I think it would be far better suited to a situation in which you can listen to a piece multiple times; that is, at home. The depth-revealing kind of information which is provided by liner notes is usually most interesting and valuable to me after hearing a new piece about four or five times (so that my head has a basic conception of the melodies and how they fit together). It also seems like the kind of situation in which it would be extremely valuable to be able to replay a section or movement. When you are informed that a new melody is coming in, for example, it’d be nice to be able to replay that bit until you can see it yourself, otherwise it’s just like listening to someone else talk about how well they know the music.

ACDs second link showed me that he was thinking along similar lines after learning the horror (for him) of the dreaded Concert Companion. Over two years ago he wrote:

“And then, in an epiphanic flash, it struck me. While the Concert Companion in the concert hall is a genuine horror, there really is an appropriate and proper place for the device: in the home, keyed to a specially prepared classical music or opera CD or DVD coded in manufacture to deliver the proper signals to the Concert Companion.”

I think there is an even greater advantage now with the rise of the MP3 format and truly digital music. It would be almost trivial to write a piece of software that would display notes in sync with a particular recording of a classical concert (since each performance has different speeds you would have to adjust the timings accordingly, but that would be pretty easy. If you wanted to be really crafty you could try and do very basic sound recognition and use that to automatically adjust the timings), You could even incorporate it directly into a website with a built-in music player. There would be a little applet in which you could select a piece, and real-time information would be provided alongside it. That would be great, and not terribly hard to set up, as long as you had non-copyrighted music to use, and of course the liner notes!

I think the big advantage of mp3s and the internet is that people can independently create the liner notes for a particular piece, or individual movement, completely independently from the people who originally performed the piece. It doesn’t have to be incorporated into the CD itself and the recording company doesn’t have to write any special software. The work can be done as an add-on by enthusiasts and freely distributed on the web.. I’d love to put something like this together, and I don’t think it would be too hard given the right textual and musical resources.

Anyone want to try it?

I Want Realtime Liner Notes

October 30th, 2007 | 6 Comments | Posted in classical music, liner notes, mp3, store

Yeah yeah, I know you know I love liner notes. Since I’ve recently been lamenting and revising my previous lament on the lack of them on a certain classical web-radio type dealie, I thought I’d finish off my current whining with a more structured post.

This is what I’d like to see: while playing my exciting and varied classical playlist (in ultra-extreme quality, with the option to download and to donate money to the composer/performer/conductor, but that’s another story) there’s a little expandable link to click, which on activation gives me a couple of options. I can find out general information about the piece (the standard “Smith’s Opus fifty was composed in blah blah blah and is in five movements”) or the composer (”Smith’s first serious exposure to music was when he began glockenspiel lessons at the age of three”). That kind of thing.

Pretty standard so far.

The cool bit would be when you dig down further. The more technical stuff that is sometimes chucked into liner notes (”the third movement starts in D and, unusually, finishes in G#”) would all be interactive - switching on this technical mode would enable something like a status bar, which would constantly update with information about what was going on in the piece (”Listen as he modulates to G#, led by the winds… now”).

Ideally you could have a couple different levels of in-depthness with this. The simplest one would indicate when the different themes were present in the piece, for example, and point out different instruments. The more serious ones would bring in more subtle things: key shifts, inversions of melodies, etc. You could even make it graphical, with colors indicating various qualities (themes, instruments, etc.)

I know if I had had something like that when i started listening it would have been fascinating, and I bet it would encourage a lot of people to start listening to classical, once they realized all of the little games and complexities going on - that its not just music which keeps on going and doesn’t have any noticeable structure. I don’t think non-classical people appreciate all the structure that’s in there, and by that I mean they aren’t aware it exists. I wasn’t.

If there was something like these hypothetical real-time liner notes which illustrated that underlying structure, I think there would be quite a few more classical converts.