The Colbert Report Green Screen Challenge

by Manuel Alducin

On October 11, 2006, viewers of the The Colbert Report were suprised by the Star Wars: Episode III inspired show intro and greenscreen entry by viewer George L. I decided to track down the Guerrilla VFX artists in charge of this entry.

The Colbert Report Green Screen Challenge: Attack of the Colbert Clone

Manuel: Did you consider bringing Stephen to the ILM stage or did you use everything in the video?

Tom: We had the footage synched up with what movement he was doing. We didn't use every frame of what he had done. We picked the pieces that looked the best with what the action of the robots were doing. So if we thought robots were falling in half we - okay let's find a scene where he swinging down and we tried to make it match up to where they're getting shot or cut in half or something.

Todd: I think we had to flop like one or two shots of Colbert to match. He was like oh that works perfectly but he's going in the wrong way. So we flopped it and it worked out really well. 

There was one shot from Episode III that I did not do in the original animatic. It's the shot where Obi-wan kind of jumped out of his cruiser.

Because of where he lands and there's a couple of droids in there. Do you know that shot?

Manuel: Yes.

Todd: I had done that shot for Episode III and I didn't not put that in the original animatic because I was scared to death of trying to do that. I thought this was going to be way too hard to do, so I didn't even put it in there. So Rob Coleman and George [Lucas] saw our animatic and they were like "oh, this was pretty cool, but you really should try to do that jumping out shot".

Manuel: With all the summersaults?!?

Tom: Oh shit, it's going to be a nightmare.

Todd: So I had to gather a really quick comp that I thought would work and try to find a good section of Colbert so that it would actually support a land, you know with tons of reanimation and stuff like that. 

Jar Jar talking to Stephen Colbert
Stephen Colbert jumping out of the starfighter was added to the sequence later on.

Manuel: Did you have to create a digital model of Colbert? Or were you able to also repurpose something?

Tom: No way!

Todd: No way. It had to be comp.

Tom: We didn't have time to do that. It had to be comp-centric. This whole thing had to be to 2D comp-centric because all the work with Jar Jar.

Todd: And I did a quickie comp of it and I was lucky because there's a certain section where he kind of squats and puts his hands like he's setting himself and I kind of back tied everything for him like landing right at that point.

And everybody loved it. I was like oh no, I can't do this. 

Tom: But even at that stage it looked great and you could tell the idea was sound and it was going to work.

Todd: So for that shot we didn't know how to take out the droid and we had some other angles that we really wanted to get, but we didn't have Colbert footage so we actually went out to the mocap stage here at ILM and had a little greenscreen session here. One of our matchmovers, Tyler Ham, has a very similar stature and hair -

Tom: And hair.

Todd: Hair was most important for Stephen Colbert and he had hair like Colbert, and a suit.

Tom: He had the hair look.

Todd: The hair was very important and Tyler was all for it. So we were able to get some reverse angles and some extra action shots where you wouldn't see his face for the sequence, but hopefully people don't even realize it, but we did have a supplementary greenscreen shoot

Tom: And extra body parts.

Todd: Well yeah, and Stephen jumping out of the cruiser and landing. We shot an insert of Tyler's feet like him kicking a droid and then I grabbed that and put it - I erased Stephen's other leg and added in Tyler's leg and made it look like he was kicking the droid over and it worked really well. This was once again the guerrilla tactics that we had. We shot all of those, our greenscreen, with Tom's DV camera, shot it in like less than an hour. We came back to our desk and I have a Mac at my desk and we imported it into iMovie and we had some footage ready to comp in like 30 minutes and that's just - that's the kind of efficiency that we needed to have to get this done on time. We had that stuff working really fast and it was a real thrill to get that stuff done so quickly. 

Jar Jar talking to Stephen Colbert
Matchmover Tyler Ham, disguised as Colbert, was quickly filmed to create needed shots and for extra body parts to tie in the shots..

Manuel: So from looking for stuff in archives to the finals was two and half weeks?

Todd: We had a little bit of time, had some lead time trying to get stuff restored, but in terms of artistry and -

Tom: Actually working on the shots -

Todd: Working on the shots. Tom lit all the Jar Jar shots like all by himself pretty much.

Tom: I was going to split it up between me and two other people, but it just turned to be a lot faster for me to just do them all myself instead of trying to pass off information back and forth to people about doing it and the lighting. It was more efficient to do everything myself.

Todd: Another interesting thing is we kind of settled on these angles that they could have this conversation - that Colbert and Jar Jar could have this conversation, so we had to create these very long moveable backgrounds for both Jar Jar, the one shot and shots of two them. We really wanted to have the space battle going on behind them in the hangar.

The longest shot in the sequence in Episode III was probably like two seconds or three seconds long. So to come up with a 30-40-50 second long movable panel there was no chance of reanimating. No chance of re-rendering that stuff. So I had to take the longest shot that we had and I did all sorts things to make this super long like 800-900 frame sequence that could run in background. Totally out of focus but nobody was really looking at it to work for two shots. But if you do look carefully at that space battle it looks like the Star Destroyer is kind of going forward and then it kind of goes backwards and then they kind of go forward again, but nobody really noticed that yet. Only the crew really knew about it and those are the kind of little things we had to do.

Manuel: How did you set up the animation for Jar Jar? Did you have the animator working on his own and then you just light the shots and comp him?

Tom: Rob Coleman was the supervisor of the animation for us, so our animator actually wasn't even here. She was working for us over at Ranch, which caused its own problems because we had permission problems with each other's networks and things are not connected as seamlessly as it is throughout the building so we would have conversations on the phone and Rob would go through the animation and send it over to us and then between Michael and myself we'd get it up and working. Inevitably something was broken because that's just the way it was. We were working so quickly and with the old data, but Michael [Koperwas] somehow managed to fix everything that was broken. I don't know how.

Todd: He's really amazing. 

Tom: He was able to get all that stuff to work, but yeah I would just get a new tape from her and then I would just work my lighting. I really only had two basic lighting setups for each, either the one shot or the two shot and I tried to move them around so they were all happening in the same 3D space for the lighting that was set up for one would work with the other one from that same angle. Then we had a little bit different lighting when they were talking to each other. That was all based off the lighting from the greenscreen. That's when we started the lighting from that and then we tried to match that when we had the one shot, but we had to match the background as well. So it was a little tricky to sort of combine those two. 

Todd: With those considerations that's why we worked so hard at the very beginning to decompose the shots so that we could get away with the most for so many difficult challenges, lighting, camera angles, and hope we didn't have - we had to be very careful in how we picked our compositions.

Tom: All the while watching carefully all the stuff that was coming up on YouTube.

Todd: Oh yeah. 

Tom: Just making sure that nobody was doing anything better than we were doing. That was a big concern. Because if ours wasn't so much better than everybody else's, that they didn't want to do it.

Manuel: Did you have to do any extra work, like FX, sparks, particles, laser blasts?

Todd: We tried to keep all that comp-centric. You know we had renders of laser blasts that was done for the original movie that we could then reanimate.

Tom: Yeah, cut one out and then just animate it by hand.

Todd: You know adding all those little saber hits. We didn't use all our regular tools for that stuff. So yeah trying to keep it very clean and sharp and fast. You know no time for sin, no time for your best work, and no time for any of that stuff. That's another reason why we didn't pick the Geonosis arena sequence from Episode II because that would have required a lot of dirt and dust in a very different environment whereas the Federation hangar is clean and pristine and all we had to do was really comp some shots, put Stephen's reflection in the ground and that's a relatively straightforward thing to make a couple of shadows in there in and out and you're done. So and it all was a part of the process. 

What was great was we had a small crew, but we all contributed ideas and we really free formed. Lorelei David our editor was also a huge Colbert fan. She just had some fantastic ideas tying everything together. It was a lot of fun.

Tom: Yeah. Everybody jumped in wherever help was needed. So TDs were doing their own roto and comping and just everybody just lent a hand wherever it was needed. It wasn't like "oh send that off to this department to have it done". We all just worked together really closely. 

Manuel: What about the rendering considerations and sending all those files around?

Tom: Well, aside from the creature and the shaders yeah that's about all that we needed from the old stuff. Everything else was done in our newest rendering tools. So yeah once we got the creature working we went ahead and just used all our latest set of shaders. Like I said the other ones wouldn't work. We couldn't get it to compile without a lot of work and it didn't seem like we had enough time to put all that effort into it. So I just went ahead and used our latest shaders and reassigned everything and remapped things that I had to and tried to match it off of whatever pictures I could find of Jar Jar. I never rendered him before. I went on the internet and found some pictures of Jar Jar and said "oh that can be done". Yeah that was it. 

Todd: You know, when we delivered it and the Colbert people were really thrilled. I mean those guys are such big - they're fans of Star Wars. And Stephen is a huge fan of Star Wars and they said they were really looking forward to the show; George [Lucas] being on the show was a real thrill. They were looking forward to it for weeks. 

And a few of us actually got to go to New York and be there for the taping and got to meet Stephen very briefly and he was showing off to us some of his Star Wars stories. He had one of the cool lightsabers; he was showing us that. He was showing us his Jedi cap and they were really jazzed about the whole thing. The feeling was mutual. I've been a big fan of Stephen for a long time and we all love The Colbert Report so much so there was a lot of mutual love going around and it was really cool to be a part of that. 

It was all worth it. I mean it was really hard. We would want to work it as much as we could on the Colbert thing and then Tom and I would look at each other and go "oh we got to get to Transformers quick". It was really hard on a couple of our artists who were in the last weeks of A Nightmare Before Christmas

Tom: Yeah.

Todd: We had a very difficult -

Tom: They were working seven day weeks on that show and still finding time here or there to work on our show.

Manuel: But hopefully it will lead to a few more challenges like that so.

Todd: Well, you know it's something that ILM is not necessarily known for being able to spit out a lot of high quality shots in a very short amount of time. That was something that Tom and I really wanted to show that we could do. Not only ILM but to the world at large. We have a lot of very talented people here with a lot of great ideas and it doesn't necessarily have to be a huge show like Pirates of the Caribbean or Star Wars to really knock out quality stuff in a short amount of time as well.

Manuel: Hopefully this will show everybody the quick turn around you can achieve and the great work you can turn out. Do you have any other battle stories?

Todd: I'm looking at our final show report. Oh, on the sound front. I should really mention that for a little while we didn't know if we were going to get Skywalker Sound to help us out. 

But Matthew Wood, who was one of the original sound mixers on I think Episode II and III, really did an amazing job on the sound mix. Once again in a very short amount of time to burn all that stuff together. All the lightsaber hits, the music and we did get to record on the sessions. I think three sessions with Ahmad Best recording, I think Rob Coleman directing. 

Tom: I'm not sure about that.

Todd: I'm not actually sure, but Matthew Wood who did the voices of the battle droids, he did for all of the Star Wars movies. He had all those funny lines that George wrote for the droids. He really pulled them together very fast and he recorded a lot of tight coordination between Lorelei [David] and Nick [Anastassiou], our editors, and Matt [Wood], because it all happened basically the last week. So that's definitely worth mentioning.

Manuel: I'll try to let everybody know about the great work and I guess I'll let you guys go to lunch but that was fantastic.

Todd: I'll make sure to send you the credit list. Our full credits.

Tom: Don't forget R2D2.

Todd: Oh, oh yeah. Tom and I were like, we wanted to add one more - it was part of the original animatic that I did. We wanted to do the R2D2 shot where he projects the hologram.

The one that shows Anakin and Obi-wan where the Emperor is being held. It's the hologram of the Federation cruiser. 

We really wanted to make that project The Colbert Report logo. So we resurrected that comp and basically in After Effects in about a couple of hours we worked together and pretty much nailed that shot. We were hoping that they would use that as a bumper for the show for The Colbert Report show, as an intro animation and it worked out really well and they used it. They loved having that little bone, and from beginning to end about two hours or so.

R2D2 hologram of The Colbert Report
The R2D2 hologram of The Colbert Report title card, done in about two hours.

Manuel: Sounds you were very busy!

Todd: We have Michael Bay coming today for Transformers so we have to all stand up straight, comb our hair and look like we actually know what we're doing.

Manuel: Well, the trailer looked really good, so good luck on that one. 

Tom: Thanks.

Todd: Thanks. Take it easy man.

Tom: See ya.

Manuel: Okay, thank you very much Tom and Todd.

All The Colbert Report images © Comedy Central. Courtesy of ILM.