Willi Geiger, CG Supervisor at ILM, recently took some time to discuss some of the tremendous challenges in creating complex fluid simulations, lighting and rendering setups and a completely digital cruise ship for Poseidon.
Willie Geiger Discusses the Fluid Dynamics, Lighting and Rendering in Poseidon
Willi Geiger joined ILM to work on Star Wars: Episode II Attack of the Clones. Previously he worked in the research department at The Moving Picture Company. At ILM he has worked on complex dynamics and fluid simulations in such projects as Terminator 3: Rise of the Machines, Star Wars: Episode III Revenge of the Sith and Harry Potter and the Goblet of Fire. He recently took some time to discuss with ILMfan some of the tremendous challenges in creating complex fluid simulations, lighting and rendering setups and a completely digital cruise ship for Poseidon, supervised by Kim Libreri.
Manuel Alducin: So, how are you doing?
Willi Geiger: I really enjoy being on a break. Poseidon was a fun show and I enjoyed the work, but it was hard. We worked a lot of hours on that. Straight after that I went on to Pirates of the Caribbean 2. A really big year.
Manuel: You were one of the CG Supervisors on the movie.
Willie: That's right. My main responsibility was rendering the ocean effects.
Manuel: Did you know early on that some of those shots were going to be pretty much 100% all digital?
Willie: Yeah! That was pretty much mandated by the client. The ship was so big, it's supposed to be 1,000 ft. long, that there's really no way you could shoot that with a miniature and make the scale work. Secondly the client wanted a complete 3D version of the ship.
Manuel: How much were you involved in the design of those shots? Did you do a lot of research on rogue waves and the ocean?
Willie: Absolutely. We had a pretty long preproduction period. I started working on the show pretty much as soon as I finished Star Wars Episode III. That would be around May 2005 when I got involved with Poseidon. We looked at all kinds of ocean references. Probably the two great pieces of inspiration were French photographer Jean Guichard, who took pictures of lighthouses and waves crashing, and the other great piece of reference was actually a sinking cruise liner. We had helicopter footage of a real cruise liner sinking, which was obviously great. We had about 6 months of pure R&D (Research and Development) for this show, completely updating our fluids dynamics engine, rewriting all of our rendering software, which was my main responsibility, and also building the boat.
Manuel: You were in charge of the fluid lava effects, when they were CG, for the Star Wars Episode III Mustafar sequences. How did that work compare to the one you did for Poseidon?
Willie: In terms of fluid effects we went a different route. On Episode III it was all about speed, control and the ability to artistically direct it. So for Episode III I really chose the simplest techniques that I could find. For Poseidon we started right at the other end of the spectrum: we knew it had to be completely photorealistic and have realistic behavior. We went with a full mathematically correct computer fluid dynamics solution. And we hadn't really pushed that software hard since Terminator 3: it was the software we originally used for all the liquid metal effects. It hadn't been developed much since then. We used it for a handful of shots: we used it for a shot in the first Pirates of the Caribbean movie where Barbossa drinks from a bottle wine and you see the wine going through him, and we used it again in Harry Potter 4 for the boat rising out of the lake, that one shot. Apart from those shots it really hadn't been used for a long time.
So the guy from Stanford, Ron Fedkiw, put a huge amount of work into developing a new fluid dynamics engine. A lot of new control features and the big thing he added for us was the ability to actually run the fluid simulation over multiple machines which was a huge timesaver. We needed to run fluid simulations at extremely high resolutions. Each one of our simulations would probably have taken 2 weeks to run on a single processor. So by being able to complete the fluid simulations over 16 different processors we could even get extremely high resolution fluid simulations overnight. We probably couldn't have done it without that. We put a lot of development into using the results of the original base fluid simulations to drive secondary simulations to get more detail. The base fluid simulation gives you a basic surface, particles that represent the water splashing above the surface and it gives you particles that represent bubbles trapped under the surface. We wrote tools to extend that to give us spray and foam, layers and layers of foam floating on the surface of the water. So we had 6 months of heavy preproduction developing a lot of new tools for the water and also for the boat.
For the boat, which is so big, it's going to be seen on so many levels of detail, from very long shots and in extreme closeups like seeing Josh Lucas on the deck of the boat. So in order to do that we split the boat into many small pieces, almost 200,000 individual components and each of those components could be rendered at multiple levels of detail. So for each shot we could assemble the boat using the correct set of components in the correct level of detail for each shot. Without that there would be no way we could render the boat. The last thing we had to develop for the boat were tools and controls illuminating it with global illumination. We would calculate multiple diffuse bounces, multiple specular bounces, etc. to get all the subtle lighting on the boat. In some of those shots we had a thousand lights on the boat, a very complicated lighting setup.
Manuel: That sounds pretty insane!
Willie: Yeah! The opening shot is around 4,000 frames long. It took a week to render that shot on the whole renderfarm. You don't have many chances to get it wrong! Another thing we did to help with that, because some of the renders were so heavy, is we split the renders into many individual passes as well as the usual thing where we render the specular pass, a beauty pass, etc. We could actually split it up per light so we had separate render passes for individual lights so the compositors could pre-light the shot in the composite so if there were any changes we didn't have to render the whole thing again.
Manuel: So you had hundreds of layers for the compositors.
Manuel: For the ocean, were you able to use multi-resolution techniques?
Willie: The geometry of the ocean was actually created at render time. What the fluids engine actually output was an implicit surface, it basically outputs a volumetric grid, which is a huge 3D dimensional grid full of numbers. The actual mesh is rendered once it's created at render time. So it's incredible detailed geometry. For most of the shots the ocean was represented by a subdivision mesh with up to 5 million quadrilaterals and that surface never went to disk, it was only created in the render stage. So we could create it at whatever resolution the shot required.
Manuel: So you only had to worry about loading the ship geometry.
Willie: Right and even the boat geometry we would only load what we needed. Every single component was an archived RIB. So what we would actually do was have a skeleton of the boat, a very simple representation, and then at render time we would swap the geometry at the correct level of detail for each component which was loaded from disk. It was a very efficient pipeline. Even so, some frames took hours to render.
Manuel: So the water interaction with the ship, was it done with the high resolution components or were lower levels of detail good enough?
Willie: We generally didn't use the highest resolution components for the hull collisions. Again it depended on the shot. When you see the boat from a distance we could use a quite coarse resolution of the boat and then on shots where you are at close range with close water interaction we had a higher resolution version of the boat. And all the collisions were actually volumetric so we didn't collide with actual hull meshes: we had a volumetric representation of the boat which made the fluid simulations much faster and more accurate.
Manuel: How did you start designing the opening shot, figuring out how to place Josh Lucas on the digital ship?
Willie: That was a huge amount of previsualization and matchmoving. That shot was masterminded by the Associate VFX Supervisor, Mohen Leo. It's all previsualized which is a huge aid. It's a great thing for visual effects, plan out the whole action before the shoot, etc. From the previsualization we can do two things: we work out the exact camera moves inside the computer and we can take that same camera move and use it for the cable camera. So we actually shot Josh Lucas against a giant bluescreen at Sepulveda Dam with the cable camera using the precalculated camera move. Then we have that same camera move inside the computer for the rendering. I think the compositors were incredibly impressed, they couldn't believe that Mohen had done such a good job. When they put the bluescreen cable camera shoot and the CG ship, it just matched. Very little tweaking. It all comes down to planning. If you are able to plan things down, then it just works. If you don't have the planning time you do your best, shoot with a wild cam and then try to make it work, it always makes it difficult. But that opening shot was great because we had all the planning time to do it right.
Manuel: I'm guessing you didn't have to simulate the whole ocean for that whole 4,000 frame shot.
Willie: We kinda broke it off. The first section when we are underwater was done as one simulation with the hull going through and obviously we had to make it look like we were under the water. At the point where you transition, where you break the surface to above the water, we had a nice break there where we essentially make a cut and hide the transition. The other thing we did is we only had to simulate the ocean in a patch just around the boat just big enough to get the wake. The rest of the ocean stretching out to the horizon is actually done done with a few flat pieces of geometry. The reason why it all matches together and looks so seamless is because we added another layer on top of the actual fluid simulation. We have a deep water simulation which essentially simulates the waves that kind of ride atop the ocean. Everything from huge swells down right to tiny little ripples. They were made with a patch which tiled so we could actually tile with physically correct waves and ripples right down to the horizon. So we only had to fully simulate around the boat and the rest of the ocean is a patch with very simple geometry.
Manuel: So you used those same techniques when the ship is overturned.
Willie: That's right. We just simulated around the boat where we are close to the action. Some shots we had to simulate a bit bigger like the high overhead shot with the boat sinking below the surface. We wanted a big wave coming right across the plane crashing against the boat, we simulated that whole area. But even in that shot we had bits of the ocean that were not fully simulated. That was a real timesaver, in shots where you see the horizon it could be that you're seeing 10 miles, you can't really hope to simulate that big of a mesh. So it was invaluable to be able to create the ocean with the deep water simulation.
Manuel: Would the fluid simulations act on the boat itself?
Willie: They have full interaction with the boat. Even in places where it's quite subtle and you might not immediately see it. Every moment when you see the boat in the ocean we had full 3D fluid simulations interact with the boat geometry.
Manuel: How was the huge rogue wave accomplished?
Willie: It was a real problem. At the beginning we hoped to use the fluid simulations for everything. We found that the client had such specific ideas about the shape that the wave would take, that fluid simulations for those moments wasn't the best solution. The shot where you see the wave approaching the boat, it's actually hand sculpted. Then at the moment when the wave actually hits the boat, that was a fluid simulation. So we created the shapes we wanted for the approach and then we had a tool where we could actually take the sculpted shapes and make the simulated water control that shape and then let the fluid simulations take over. We could sculpt the shape of the wave in one shot where we wanted a very specific shape and then in the next shot we could pass that shape to the fluid simulation engine as an initial condition (the sculpted shape as a starting point), and then we let the simulation take over, so the sculpted shape then crashes into the boat. So we could actually go backwards and forward between sculpted shapes and fluid simulations. It really did a good job. And of course we could render the sculpted waves and the fluid simulation waves with the same techniques so they actually look the same.
Manuel: That allowed you to completely art direct those shots.
Willie: Exactly, which was very important in those rogue wave shots. The client wanted to be able specify exactly how the wave looked. The biggest problem that we found was that it's very hard to sustain a giant wave like that. Waves always want to collapse or break so the reason we had to sculpt this is to create the effect of the wave sustaining and growing.
Manuel: So the spray coming of the wave, was that driven by the simulations or did you place particle emitters?
Willie: We did a bit of both. Wherever we could we used the fluid simulations to actually drive the spray. That way we could get a very close interaction and tight integration. That was one of the things the client was absolutely insisting upon: have really believable interaction of all those elements. However there were times when for the sculpted wave we generated the spray manually and in some other shots we also generated it by hand as well. For example, those shots where you have the ship rolling and smashing into the water with huge jets of spray and water, we used spray driven by the fluid simulation but we also hand placed particle emitters. We could still use the fluid, we could take the ship geometry and the fluid geometry and calculate the intersection and the relative velocities and use all that information to get a very good spray integration but there were times when we had to place particle emitters for the spray.
Manuel: I noticed that when the ship was turning over you could actually see the swimming pools spilling over. Were those simulated separately?
Willie: I'm so glad you noticed that! Yes we did, that's absolutely right. They were run as separate fluid simulations. We would fill the pool with water and run the thing as the boat turns over.
Manuel: I like those kinds of little details, you can even see the lights from inside the pools.
Willie: Another thing we put a lot of work into was calculating the way light scatters through water. That was a very important feature. We have this boat at night which is covered in lights like a Christmas tree. So it became incredibly important to figure the way all those lights shine through the water and glow and scatter by the water. So that was another big area of research and development. So one of the R&D guys, Dan Piponi, worked on that.
Manuel: Yeah, he worked on the Matrix sequels!
Willie: That's right! He was head of R&D at ESC.
Manuel: So how much global illumination was used and were you able to get away with some other tricks?
Willie: I don't think we cheated anywhere. That was GI in every shot. We used our pipeline for calculating all that stuff. It was fast enough that we could run all those global illumination lighting simulations for every shot.
Manuel: Was it interactive enough for the lighting artists to make changes on the spot? How do you handle that with so many lights?
Willie: I wouldn't exactly say interactive. The way we make it more tractable and faster is we don't try to calculate everything in a single render pass. So we have 3 passes. We have a pass that calculates the occlusion, how much each object is in shadow from the environment. Then we have another pass where essentially we spray photons into the environment and cache them on disk as a huge particle cloud, where every particle is essentially a photon. Then we can take that prepass, that photon pass, and with that we can actually calculate how will that light bounce around. That, together with the occlusion pass, pretty much gives us a complete model of all the secondary illumination. Finally we run a beauty pass, a final pass, where we calculate the direct illumination from all the light sources and then we mix in all of those prepasses. So by breaking it up in that way you make something that, while not interactive, is fast enough for the artist to be able to work.
Manuel: So how do you organize the work? I remember for The Perfect Storm you had a team for the fluids, another for spray, maybe another for the foam, etc.
Willie: We didn't have to break it down that way, as finely divided as in The Perfect Storm. It's six years later, the technology is more advanced, the computers are faster, so we can just do more. We would generally have on average three artists per shot: one would be running the base fluid simulation, one would be rendering the boat and one would be running the secondary simulations like spray, mist and foam and rendering the ocean. It's nice not to break it down so finely because you want to give the artist a sense of ownership over their shots. You want somebody to feel that they actually produced the image of the shot rather than working on one tiny little bit.
Manuel: Did you have to run simulations on the boat as well? You see things getting smashed and buckling when the wave hits.
Willie: Good question. We did a lot of rigid body dynamics simulations on top of the boat. For example the shot when the water crashes over the bow of the boat very early on and we see the crane get hit by the water, swinging around and crashing into the window of the cabin, that was done with rigid body simulations. So we run a simulation where the force of the water pushes the crane into the window and when the crane smashes the window we run another rigid body dynamics simulation to get the buckling of the window frame, the shattering of the glass. Some of the shots have the base fluid simulation, we have secondary fluid simulation for the foam and the like and then have multiple rigid body dynamics simulations. Then we have simulations for other things that were there, like strings of lights and they sway, and in the opening shot where there were many flags, you can see them fluttering.
One of the things that we tried to do often was to have as little of these hand animated elements as possible. As far as possible we would try to have simulations for everything whether it's for fluid effects or soft body effects, etc. It just helps in the believability of the whole thing.
Manuel: So you didn't really need much of miniature elements except for perhaps the explosions?
Willie: There were some practical elements used for the explosions although some of the explosions were CG as well. All the underwater explosion were digital. There are a few bits of elements for closeups like spray that we would occasionally layer in, but apart from that pretty much everything was digital. Right at the end when you see the characters on the liferaft floating away from the boat, that liferaft was shot on a tank. The liferaft and the wave it rides on they were shot on a tank and then blended into our digital background.
Manuel: So you only needed to blend the real water with the digital water.
Willie: Yeah, so some of those shots in the end where you see the closeups of the liferaft trying to get away from the sinking boat was a combination of real water around the liferaft and the digital water. Which is always hard. When we got those simulations right it was quite gratifying how usually close the interaction we got between real water and the digital water was. I think that part was perhaps more fulfilling than on The Perfect Storm on which they did a tremendous job.
Manuel: So how do you see the future of fluids simulations?
Willie: I think this movie was a huge advance to us in that it clued us that we can pretty much do anything with CG water. Where can it go from here? It has to get faster and we got to get more control over it. Eventually it would be lovely if it was like a video game, the artist should be able to use it pretty much in real time, move object around and have the water interactively react. That's still a long way out but it's a great ideal and I think it would change the way we work. That different way of working, interactively like a video game which has it's own way of working, I think would allow us to get a very different visual look
Manuel: Thank you very much for your time!
Willie: Cool! Thank you very much Manuel.
Many thanks to Megan Corbet for all the help conducting this interview and to Willi Geiger as always for his time and amazing insights.
All Poseidon images © Warner Bros. Courtesy of ILM.