Client: Element Animation, Mojang
Role: Lighting artist, Compositor, Rigger
Link to the video: https://www.youtube.com/watch?v=0cpQC5jVMDg
Element Animation produced a song together with an animated music video about the Warden, a monster in the popular video game Minecraft, for the Minecraft Live 2022 event. I was hired by Element Animation to work on the lighting, do all of the compositing, and due to time constraints at the end manage the last two weeks of production, making sure it got finished on time. Additionally, I created the rigs, smoke effects, and a special grain algorithm used in the video.
Since the music video was meant to be scary, getting the lighting right was very important. We decided to plan out the lighting for the entire video beforehand. I created a lighting plan for the video. It starts with references that show what the look is that we are going for. I liked the idea of going towards horror movies from the 80s, and the people at Element Animation and Mojang agreed. The references that I found, often used the same colour of blue for all of the lights in the shot, and instead used brightness to create contrast and define shape. They also often used silhouettes.
The video itself takes place in the Deep Dark, a place deep underground with no natural light. There are bioluminescent blocks, soul lanterns, and candles that provide dim lighting.
Next to the Warden, we have two characters, Alex and Steve. Alex has a lot of bright greens and oranges in her design, while Steve has a lot of browns and dark blues in his design, making Alex easily stand out in the set, but Steve will more easily blend in. Because of this, we needed to pay extra attention to make sure that Steve has enough contrast with the background and that Alex doesn’t stick out too much compared to Steve.
The Deep Dark is fully underground, which means that we’d have a ceiling of deepslate (a kind of rock). We generally want the background to be really dark, which would basically mean that the ceiling of deepslate would be entirely black. An empty background would be a bit boring, so I came up with the idea to add those glowing dots from the sculk in the ceiling as if it’s a sky filled with stars.
Based on all of this, I created eight rules for the lighting of the video:
This is combined with a few quick paintings to showcase the visual rules.
With all of this defined, we can plan out the lighting for each shot by simply doodling on top of a screengrab. Rather than creating a concept painting (called a colour key) for each shot, we create an abstract idea of the lighting, allowing us to focus on the important parts, like storytelling and making sure that we are creating enough separation between the characters and the set. Since it’s abstract, anything that isn’t important can simply be left out and whoever does the lighting for that shot, can fill in the blanks based on those eight rules and our references.
In order to be able to create enough separation between the characters and the background, we needed to be able to have the background get pitchblack. Global environment fog would not work, since it would raise the blacklevel of objects in the distance. But we’d still need smoke in order to create silhouettes, which is important in getting the separation needed. We could have used actual volumetrics, but I wanted it to feel more like Minecraft. Instead, we went with a sprite-on-particle approach. I created a particle simulation using Bifrost in Autodesk Maya, which then had the smoke particle texture found in Minecraft applied to every particle. The opacity of each particle was set to a very low amount in order to create a volumetric effect that is still “blocky”, just like the game. This was all packaged into an asset, containing eleven variants of the smoke simulation, that we could drop into the shot during lighting.
Joel Houghton, the art director at Element Animation, and I worked together to do the lighting for this video, using the lighting plans as a guide. Most of the lighting was achieved using focused area lights, creating pockets of light with harsh shadows. The sculk sensors, which light up, were mesh lights using the tendrils as the shape.
While normally, a character would be lit with something that is somewhat close to a three-point lighting setup (a key light, fill light, and edge light), Minecraft characters are blocky, making such lighting setups work very differently. For this video, however, we are mostly either lighting the character brightly and putting them in front of a very dark area, or silhouetting the character. This slightly simplifies the lighting setups normally required for Minecraft animations, but we’re still dealing with the problem that a flat face isn’t going to show lighting directions. In some shots this is fine, but in other shots we need that information, either by adding in a specular edge on one side of the face or a gradient over the face.
Steve’s hair is another tricky area as well. If we light him normally and then put him in front of a very dark area, his dark hair will blend into the background, which will look weird. We need to preserve Steve’s outline. One trick that we use is to add a bit of smoke behind just his head.
Alex’s design uses brighter colours than Steve’s design, which makes Alex naturally stand out more than Steve, while we often don’t want that. Often we’ll either brighten up Steve or darken Alex to compensate for this.
Since a lot of the inspiration for the lighting style came from 80s horror movies, I wanted it to at least somewhat feel like it was shot on photographic film. We are using an ACES colour management pipeline, but with a custom colour rendering that is less contrast-y and has display gamut compression. Minecraft often uses purely saturated colours in its textures, which can cause problems when combined with saturated lights, when using the sRGB gamut to render in. The ACEScg gamut is much wider and with that provides more ideal results. However, the default Reference Rendering Transform found in ACES doesn’t produce nice results for the art style of Element Animation, so I created a custom OpenColorIO configuration with a custom rendering transform that solves our issues with the default RRT.
Next to that, during compositing we are adding halation, a glow around bright objects that is more extreme for the reds. The halation creates a colour shift in the glow, bringing in some reddish tints in an otherwise very blue image, and adding a bit more visual interest overall.
Grain is also important in photographic film, but it also shouldn’t be a lot. None of the other animations that Mojang has, contain even the smallest amount of grain or noise. Unfortunately, video compression handles grain very badly, removing the grain from the image and replacing it with compression artefacts. Sometimes a static image of grain is used instead of animated grain, but when the camera moves the static grain becomes very noticeable and is still as bad for compression as animated grain. Instead, I created a custom node in Fusion that uses the world position pass to add grain that sticks to objects. This adds the texture that we are used to from photographic film, while also being very compression friendly.
Preview of just the grain for one of the shots in the video.
For this video, I created custom rigs for Alex, Steve, the Warden, and the bats. The characters are made entirely out of cubes that do not bend or deform, rather they move and rotate. The limbs and the torso use simple IK and FK controls. The face is achieved using animated UVs. Every controller also has a hidden surface to make it easier to select that controller. Instead of having to click on the controller itself to select, for example, the left foot, the animator can simply click on the lower half of the leg. Combining it with the tweak mode in Autodesk Maya’s Move Tool settings, allows for interactive posing by dragging on different parts of the body. The rigs are made to be as simple as possible in order to increase performance, allowing artists to easily animate larger shots in real time.
For characters like Alex and Steve, we use a generic rig where the texture applied to them contains all of the character design, like clothing, hair, eye colour, and more. Steve’s mouth is a bit different. Instead of a normal mouth that can change width and height and optionally show teeth, Steve’s mouth just moves up and down. We originally looked at animations produced by Mojang themselves and noticed that they have his mouth move up and down to create different facial expressions. Since this video is for Mojang, we decided to simply copy them.
The Warden uses the same base rig as Alex and Steve, but then with added control for the ears, rib-like things, and the Warden’s heart.