Wednesday, January 23, 2013

About That Quack

UPDATE: the case that the producers of Glee appropriated Jonathan Coulton's recording has gained some further evidence. I have done some more recent work on this issue, and so have some other Coulton fans; you might want to take a look at my newer article here.

So, the Internetz are abuzz with the story that Glee, a tele-visual musical drama of some kind that I've never seen, um, appropriated Jonathan Coulton's arrangement and/or recording of the Sir Mix-a-lot easy-listening hit "Baby Got Back," an ode on the callipygian ideal.

We're still waiting to see if the song will actually air on the TV show tomorrow, but meanwhile there is a lot of commentary. The issues roughly break down like so.

First, why is this a big deal? Coulton recorded a cover, and paid the artist through the usual channels to release his cover, and presumably so did Glee.

Well, there are covers, and there are covers. The original song has no melody to speak of, and no chordal accompaniment to speak of; it's mainly a rap. Coulton created a sung melody for the song, and an elaborate backing for it, including banjo, mandolin, guitar, and sound effects. Coulton altered the original lyrics slightly (singing "Johnny C's in trouble" instead of "Mix-a-lot's in trouble," "take your pretty picture" instead of "take your picture," and a few other minor changes. Glee used Coulton's lyrics exactly.

OK, so maybe he has some kind of legal or moral entitlement to his altered lyrics -- they aren't big alterations, right? That isn't a big deal, is it?

Well, not only did they use his altered lyrics, but they sang his melody -- identical in every detail to the one he wrote, except they ended the song a verse earlier and didn't use his spoken bridge. Oh, and they used his arrangement. It is not entirely clear to me whether Coulton can claim legal copyright on his melody and arrangement, but morally -- it's his original work.

OK, well, so they just recorded a new "sound-alike" song and replicated his arrangement with some bored studio musicians, right?

Not so fast. Not only did Fox use his lyrics, melody, and arrangement, but they used his recording.

Coulton has released his song in several different forms. It was on the Thing-a-Week collection for download. It was on the Thing-a Week CDs. You can download it in Ogg format. You can download it in FLAC format. But they couldn't have used that as a backing track, right? It's got his lyrics on it?

Well, maybe. But it is actually often possible to do a studio trick that will cancel a lead vocal very effectively. Fox might have done that. They might have bought his full song in FLAC format at high quality, cancelled the main vocal, and layered new parts on top of that. Jon Schell demonstrated that this can be done with JoCo's full song here. He also showed that the Glee track probably involved cutting and pasting Coulton's original  handclaps in the intro. Actually that's a lot of proof, right there.

Coulton's also released his track in the form of a Karaoke track -- an MP3-only version that has all the instruments but no lead vocal. They might have used this track and also done that handclap fix to remove some extraneous backing vocals during the handclaps.

And, he's also released, at least in a limited way, his source tracks -- in the form of MP3 files of each aligned track that he mixed together to make the final song. He offered these as a special bonus for a a fundraiser for Creative Commons, if I recall correctly. Anyway, now, some musicians and home studio nerds like me have been trying to figure out just exactly what Fox appropriated.

I've been listening to the banjo, guitar, and mandolin parts against the Glee track and they align perfectly. Even some of his minor playing glitches are replicated. It's absolutely clear that they used Coulton's recording somehow. But there are some fades, and so I don't think they built it up from his source tracks. They might have done the vocal-removal trick on his finished song and mixed their vocal with that as a backing track, or they might have used the Karaoke track with all the instruments pre-mixed.

It's a little hard to prove it. What we have from Glee is a compressed audio file. Its been remastered and had new vocals mixed over it. As a musician I'm absolutely convinced they used Coulton's recorded material, because I can hear it in the mix, but as an audio engineer it's difficult to prove it really conclusively. If I had an un-mastered, uncompressed recording of their mix, it might be possible to do some things like "phase cancellation," where if Coulton's track is perfectly lined up and mixed with their track but inverted, it should cancel out his instruments. In fact I can get some cancellation, but when you remix, remaster, and compress a track, it changes it quite a bit and does a lot of damage to the stereo imaging, and so it won't cancel completely.

So here's a quacking gun.

Coulton noted that he thought he heard a "duck quack" in the Fox track. What's the duck quack? In his recording, Coulton bleeps the word "fuck" and replaces it with a sound effect of a duck quacking. It's not in his separated lead vocal source track; he just left silence there. Anyway, here it is in Coulton's Karaoke track. This is the waveform:

That quack is pretty loud. The big "pop" at the beginning there is probably a kick drum and a slightly off-the-beat bass string pluck.

Here it is in a spectrum view: you can see the "arc" of the quack sound effect in the midrange, almost like a spine that you might see in an ultrasound:

In the Fox track, they were starting with one of Coulton's tracks, and wanted to kill the duck sound, they might have used a tool like Izotope RX, or they might have applied some equalization to that spot in the mixed track. There's a problem, though -- in a mixed audio track with a lot of different instruments and vocals playing in different frequency ranges, you can't just surgically remove one thing, especially not when it exists in some of the same frequency ranges as the other sounds, without producing collateral damage. Tools like RX are very good, but if you take too much out the result will sound funny. They took out as much as they could and covered up the change with some high-pitched backing vocal "aaahs," but if you listen closely, you can still hear the "ghost" of the quack in the Glee song.

It's a little hard to hear. If you are listening to the mixed track on ordinary desktop computer speakers, it would be very hard to hear, buried in the mix. I have some good studio monitors and that helps. And this might help: here's a clip where I took Fox's version from the iTunes store and selected a few seconds around the "quack," with a narrow EQ band applied, created by selecting only a band of midrange frequencies in Izotope RX and exporting just that selection. The "ghost duck" sounds kind of like a French person saying "oui" in that flat, nasal way you sometimes hear it said, right where the f-bomb would go.

For these comparison images, I used a bit of audio captured from the Glee song from YouTube. The ghost of the quack is hard to see, but it reveals some other interesting features. Here's the YouTube capture showing roughly the same milliseconds of the Glee song, around the point where the "quack" was in Coulton's track:

It's been re-mastered and re-compressed, and that alters everything. This hides frequencies and rounds-off and approximates a lot of the details in the track. The effect is kind of like the way JPEG compression adds artifacts that appear unnaturally sharp, while losing some of the smooth gradations of the original. Despite these changes, you can see that the ghost quack is still there. Here is the spectrum view:

The horizontal line that appears mostly in the left channel is the new background vocal "ahhhhs." It's been distorted, but a narrow band of the "arc" of the quack is still there. Let's look a little closer, showing the waveform and spectrum together. Coulton, then Glee:

These are maybe a little hard to "read," in but basically, see if you can align features vertically between the two versions; left channel is on top and right channel is on the bottom in each of those screen shots. The features in the Glee capture show signs of additional lossy audio compression, which blurs and alters the details of the spectrum, but in my opinion these similar features indicate that Fox used Coulton's backing track and altered it to hide the quack. The spectrum views just confirm what the ear can hear.

Update: the next part of this argument I now consider incorrect and obsolete, since further investigation suggests that the producers of Glee did in fact replace the bass and drum parts.

Besides the quack, the very low end actually shows even more smoking-gun similarity. That's the bass, and kick drum. The energy from the kick drum and bass are represented by the very lowest edge of the spectrum display, and the pattern is energy in those frequencies is just too similar to be coincidence. This view is displaying under a second of audio -- more like a quarter second -- and so it would be extremely unlikely for re-recorded bass and percussion parts to "land" like this and leave the same "footprints" on the spectrum within that same quarter-second.

Let's take a look at that low end in more detail. Here's a color spectrum version that I set up to look more closely at frequencies below 4KHz, using Soundtrack Pro:

Click any of those to get a bigger version.

Would this hold up in court? I have no idea. Would Coulton or his representatives try to use the evidence of some random bearded geek on the internet? Probably not. But I think it's pretty convincing. Together with some investigation that other Coulton fans have done, ranging from putting the tracks side-by-side for comparison, to pointing out how the vocals could have been cancelled in the original, to noting other glitches in the editing, it's compelling. It's not a case of appropriating Coulton's lyrics, or his arrangement, or his melody -- they appropriated his recordings. And to date he's been given no compensation or credit of any kind.

If you want to try to replicate my results with your tools, you can find my short audio excerpts here. Coulton "Quack" Clip and Glee "Missing Quack" Clip.


Cecily said...

For science!

Kenneth Knight said...

I wish I could work upalevelofpassion on things, any things, like you just showed. Ivenever watched Glee but I think they have done this play first and ask for permission (or at least acknowledge the source) later type of thing before. Maybe it'll give JoCo a boost and help show what amire copyright and licensing are.

Paul Potts said...

Ken, agreed. The reason I'm interested in this is not that it is Coulton's track per se, but that I think this is an interesting test of an artist who became famous and released everything first online. A lot more artists are doing that sort of thing. If the major corporations are just going to entirely ignore copyright, Creative Commons, or other licensing when it comes to working with the material those artists produce, that's a big deal, and it needs to change, or else the incentive to do creative in music online, outside the major label and studio system, will be effectively killed.

Mr Funny said...

Your post falsely assumes this is actually Glee- we have no real proof they're behind this obvious ripoff. Seems odd a multi-million dollar production would use an indie artist's actual recordings instead of just redoing it themselves. And have a character named Adam sing "Johnny C".

Danielle Nelson said...

1) neat! (the science, that is, not the stealing.)

2) re: "We're still waiting to see if the song will actually air on the TV show tomorrow" - the track *is* listed on the official Sony/Columbia site as part of the soundtrack for this episode. So I suppose the part that remains to be seen is whether or not they use this arrangement.

Angelastic said...

Good job. This reminds me that I actually emailed JoCo ages ago asking why there was no quack track with the other source tracks on the Creative Commons USB drive; I'd been hoping to use it to replace the same word in First of May. :) He didn't respond, though.

Paul Potts said...

MrFunny, the track is for sale on both the U.S. and Swedish iTunes store credited to the Glee cast, so they are "behind" at least that much ripping-off already.

Danielle, I am curious what the visuals look like for the song and whether they might at least put Coulton in the show credits. We don't have cable TV so I'll have to hear about it online.

BoffoYuxDudes said...

I posted this link to a Copyright lawyer friend - so I'm curious what his take on it will be.


Timothy Makin said...

Here's a little red ink for you:

"and layered new parts on TALK of that."

Jules said...

Just in case anyone is still skeptical -- despite the fact they were selling it on official accounts -- it did indeed air in tonight's episode; Johnny C and all.

emmetocuana said...

To Mr Funny - well there's the Sugarhill Gang precedent, when a rap was stolen featuring the identifying "I'm the C-A-S-A-N-O-V-A and the rest is F-L-Y" line from Grandmaster Caz.

Paul Potts said...

Timothy, thanks. Fixed.

Jules, thanks for letting me know. I am hearing conflicting stories. Someone said it was a slightly different track in the show as broadcast.

Mr Funny said...

Jules is wrong, it DIDN'T have the Johnny C part of the song. It was shortened for TV, that whole section wasn't aired (I'm not buying it so I don't know if the version they're selling says Johnny C). It seemed to match the leaked version perfectly. Maybe they did re-record the arrangement but it was definitely Coulton's version. I didn't catch any hint of a quack. As a Glee fan I'm not only offended but shocked that they would be so brazen, even after the controversy went public.

When I posted my comment it was only appearing on the Swedish iTunes and nowhere else, could've been a mistake- there was a record of all the other songs they were singing online but not BGB. It's now appearing on Canadian iTunes and I assume everywhere. I believe in innocent until proven guilty, Glee proved itself guilty and I hope Coulton sues their asses off and the public shaming goes mainstream.

Paul Potts said...

I can't bring myself to watch the video clip from the show too many times, but it seems to me after a quick listen that it was the same track, just edited shorter.

Caleb said...

Here's my own scientific analysis. I've come to the unfortunate conclusion that, at the very least, the bass part is subtly different.

Dave Leigh said...

When aired and first released on Hulu, the "Johnny C" part was ABSOLUTELY in there. It's since been rather ineptly cut out, along with the whole verse about "silicone parts are made for toys". It was 3:47. Now it's 2:00. But on the Internet your mistakes never die... it's here, at 2:17:

I also think we can safely stop speculating on how they might have extracted or disguised things in the karaoke release. Based on a suggestion from Caleb Hines, I did a quick Google search. Paydirt on my first try. All of the tracks are available via torrent as part of the "JoCo Looks Back" CD.

I found 12 currently active seeds. So the source tracks WERE available to Glee, and even if one or more were replaced, the most problematic of them (such as the banjo) remain for THEM to explain.I don't think they have a studio musician who can reproduce those tracks, note for note, with no performance differences.

Dave Leigh said...

It occurs to me that if they used the source tracks available to them on "JoCo Looks Back", there would be compression artifacts in "their" source files that they couldn't hide. Of course it may take access to Glee's source files to discover that.

Really, what's the smoking gun to me is that banjo. JoCo is, to be fair about it, a thoroughly mediocre banjo player, which you can plainly hear when it's not hidden in the mix. Any studio musician they got would have done it better. It's also quite difficult for a decent musician to play "down" to replicate someone else's mistakes with fidelity.

Paul Potts said...

Dave, I agree with you the torrent was available -- but if they used the source tracks, they wouldn't have had the remnants of the duck quack. So based on more experiments I believe they center-cancelled the karaoke MP3 file. When you do that it completely kills the bass and most of the drums. There is a remnant of the quack left over because the MP3 decoding is lossy and doesn't perfectly maintain the stereo image. Then they probably would have done a little shifting of sections of the instrumentation to fix timing issues, and added a new bass line (Caleb's proved they didn't use Coulton's original), and drums.