Crowd-Sourced Forensic Audio Analysis (or, How Glee Did It)

Before you read the rest of this: go here and read about what Jonathan Coulton has done to try to make the best of an ugly situation, and buy his track!

OK, now welcome back.

So, I've long known that Jonathan Coulton's fans were... different. Enthusiastic, one might say. Insane, one might say, but in the nicest way. My experience has been that they are great people, and very sharp people. Nerds? Well, that does kind of go with the territory. And honestly, while Mr. Coulton is very talented, Coulton fandom seems, to me, to be not so much a cult of personality as it is a group of people interested in new possibilities. What if musicians didn't need labels? What if they didn't even need studios or producers? As the Glee incident shows, though, they do need lawyers. And it doesn't hurt that they have fans who will watch their backs for them.

So in my last post I isolated the duck quack on the Glee version of "Baby Got Back." My previous blog entry was linked up by Wired and even CNN, and others. Some of them quoted me. Some tried to summarize my point. Some of them even got the basic idea right, but some seemed to say that I had shown that there was no smoking gun, which is the opposite of what I set out to show. So, ummm... let's see... let me make this as clear as possible:

I made a little video to go with this essay.

What I believe we’ve shown -- I say we, because other Jonathan Coulton fans have contributed their brains and ears -- is that Glee used Coulton’s Karaoke track as a starting point, then center-cancelled it to get rid of the bass, and some of the percussion. Then they did some editing, to simplify the banjo parts a little bit, and to kill some of the more complicated bits of Coulton's backing vocals, and mixed their new bass, drums, and vocals with it.

They did not start with the full song in FLAC form or from the CD track, because if they had done that, they could have perfectly killed the duck quack. They didn't start with Jonathan Coulton's distribution of source tracks, because if they had, they wouldn't have had the "quack" in there (it isn't even present on any of those source tracks). They didn’t start with the regular song in MP3 format, because if they had done that, the center kill would have left a hint of Coulton’s lead vocal. They did not use Coulton’s bass line, as Caleb Hines has shown. They used the Karaoke track, and it left a trace of the duck quack there. 

They chopped it up and copied and pasted some segments to choose simpler parts of the banjo and get rid of some of the bits of Coulton’s backing vocals (the “little/middle” bits). They tweaked the opening hand claps (as Jon Schell has shown). Most of Coulton’s backing vocals, though, are _still there_. They added a new bass line and some additional percussion. They added some new backing vocals (with a sh*t ton of auto-tune) and added a new lead vocal, and some little fake “string squeak” noises. 

So that’s what they did, and in the text below I describe in more detail how they did it. If you don’t believe me, try the experiment yourself!

OK, still here? Let's go into it a little deeper.

The various versions of "Baby Got Back" that Coulton released -- the MP3, the Ogg, the FLAC, and the versions on the Thing a Week and JoCo Looks Back CDs -- all have the duck quack. They are all made from the same stereo mix. If you want to listen to it, it's at 2 minutes 40 seconds into the song -- you can hear it in this video. Or you can listen to this short clip taken directly from the FLAC of Coulton's "Baby Got Back."

OK, so in the last article I mentioned that Jon Schell proved it is possible to cancel out the "center" of Coulton's song. Let me clarify what that means.

In a stereophonic music track, we have two channels, left and right. But audio engineers tend to think a little bit differently. When preparing a song, they want to make sure it will sound good on as many different kinds of playback devices as possible. They test for mono compatibility -- does it still sound reasonably good on one speaker? They test to make sure it still sounds OK on a system that can't reproduce low bass notes. They test to make sure it sounds OK on a car stereo, where the driver might hear mostly the left channel. So we are accustomed to thinking of them as just those two channels, but audio engineers sometimes think of them a little bit differently -- as middle and side channels. How much sound, and what sound, is in the middle, and how much exists only on the side? To find out, you do a "center-cancelling" trick.

If you take a mono audio track, copy it, flip the copy upside down, and mix them together, then the copy will have a waveform below zero at the point where the original has a waveform above zero, and vice-versa. The net effect, in digital audio, is that the two tracks will perfectly cancel each other out -- you will hear silence.

With a stereo track, you can do a more complicated trick than that. You can take the left side as it is, invert the right side, and mix them together. The result is a mono track that contains the sounds that are identical on both left and right. These are the vocals or instruments that are mixed precisely in the middle of the track, not heard more loudly in the left speaker, or the right speaker. You can then use this "center only" track with the original stereo track to kill the things that are mixed to the center. If the main vocal is mixed dead-center (this is very often, but not always, the case), you've completely killed the main vocal. A free karaoke track!

In the case of Coulton's "Baby Got Back," this works really well, as you can hear in this sample. That's the same bit of the song as the previous clip, with the center killed. So, Fox could have used that track to turn Jonathan Coulton's track into a karaoke track, and added their own singers on top of it. But there are two complications: first, the bass and some of the drums are also centered. So these were also completely killed. The track now has no bass and has only shaker and tambourine and hand claps for percussion.

So, to use this technique, Fox would have to re-record the bass and drums. Caleb Hines has done a close investigation of the bass line, and his results suggest that they did just that.

And second, if they did this, the duck sound would have been killed, just like it is in my sample. Right?

Well. Hmmm.

If you listen closely to the Glee track, you might notice that their backing instrumentation doesn't sound very good. It's a little distorted. There's a reason for that. The reason is that they used Jonathan Coulton's Karaoke MP3 file to build it. (Here's a link to the store on his web site). So they were starting out with compressed audio, which they uncompressed, and then added tracks to, and then re-compressed.

How do I know they used it? Via a process of elimination. The "ghost quack" is present in the track -- so they didn't use the FLAC or a track copied directly from one of his CDs. But there are no "ghost vocals." What do I mean by these "ghost" sounds? To understand this, you have to understand a little bit about MP3 compression. MP3 compression basically throws away most of the information in your music. It replaces your original music with, essentially, instructions to your computer for creating something that sounds vaguely like your music. (This, by the way, is why I buy most my music in CD form instead of MP3 form).

It's sort of the like difference between the Mona Lisa and a paint-by-numbers copy of the Mona Lisa. You don't get back the original waveform, you get back a waveform that has most of the same frequency and timing information as the original. The details of the actual audio waveforms will not look the same when viewed up close. The stereo image of the reconstructed audio may be shifted slightly -- the balance of information between the left and right tracks no longer perfectly matches. And so, the center cancel doesn't work perfectly. The result sounds like this.

In that clip you can just barely hear a little sibilant trace of Coulton's vocal, and the quack. It's a little bit more audible if you apply some equalization: here's the same clip with the high frequencies isolated.

So they didn't use this full track, or at least I think they did not, because we don't hear those "ghost vocals" in the Glee mix.

But we do hear the quack. So that leaves only the karaoke version -- which has no lead vocal at all. They did the "center kill" technique to remove the quack (and with it the bass and some of the drums), leaving only the "ghost quack." Here's the same clip from the karaoke track. Here's the same clip with the center killed. Here's the center-killed clip with the same high-pass equalization.

Now, to compare. I realized that if this technique works to help isolate Coulton's accompaniment distinct from the lead vocal, why not try to isolate Glee's accompaniment too? Sauce for the 47% and all that, right?

Here's the same part of the song, Glee's version. You might be able to hear the remnant of the "quack" better if you listen to it reduced to mono -- it makes the mid/side distinction less distracting to the ear.

And now Glee's version, with the center killed. That version leaves a sort of "Glee backing track" in which you can hear Coulton's instruments more clearly. The Glee lead vocal is almost completely killed, but there are some odd sibilant leftovers due to MP3 compression. There's also a lot of processing with the auto-tune effect. Wow, do they use a lot of auto-tune, especially in the backing vocals at the end of the song. I guess it's cheaper than hiring people who can sing.

Anyway, so we have a Glee "backing track" now. What does that give us, besides a way to make fun of the backing vocals? It gives us a way to more clearly compare just the accompaniment. I'm not going to post the entire Coulton and Glee center-killed tracks here, because I think that would be pushing the fair use exception of copyright a little too far, but you can do this experiment yourself, and then compare them by any means you like. Here's a clip: in the left ear, Coulton's backing track. In the right ear, Glee's backing track. Funny, it sounds like the same track, just with some Glee vocals added on the right. Isn't that interesting? Those are Coulton's instruments: his banjo, his mandolin, his guitar, copied and pasted to repeat certain parts of the original and leave out others. You can even hear some of Coulton's original backing vocals, slightly buried under Glee singers, although some bits like, the part where he sang "little middle," have been edited out with more copying and pasting.

Anyway, that's today's crowd-sourced forensic audio analysis. The Coulton fans have discovered some other interesting facts. We've discovered that Glee added fake string squeaks to try to "sell" the video clip and make it look like the guy miming playing banjo was really playing the banjo part -- in the context of this blatant theft, that's almost sad. That was Caleb again -- he's got a great ear. We've discovered the ways in which Glee copied and pasted segments of the synthesized backing track to simplify Coulton's string parts and cut out some bits of the backing vocals on the karaoke track. At this point there's really no question as to whether Glee took Coulton's audio track; there's no question as to how. I am curious as to the exact details, but that's mainly to determine just how correct I was. Oh, and of course, there is still the small matter of whether they are going to get nailed for copyright infringement.

And finally -- you don't have to just take my word for it. Grab the tracks. Listen. If you have an audio-editing program like Pro Tools, Apple Logic, or even the free and open source Audacity, and a little patience and willingness to learn, you can hear what I'm hearing and judge for yourself!