Minecraft Live 2022: A Warden’s Song

Client: Element Animation, Mojang
Role: Lighting artist, Compositor, Rigger
Link to the video: https://www.youtube.com/watch?v=0cpQC5jVMDg

Element Animation produced a song together with an animated music video about the Warden, a monster in the popular video game Minecraft, for the Minecraft Live 2022 event. I was hired by Element Animation to work on the lighting, do all of the compositing, and due to time constraints at the end manage the last two weeks of production, making sure it got finished on time. Additionally, I created the rigs, smoke effects, and a special grain algorithm used in the video.

Since the music video was meant to be scary, getting the lighting right was very important. We decided to plan out the lighting for the entire video beforehand. I created a lighting plan for the video. It starts with references that show what the look is that we are going for. I liked the idea of going towards horror movies from the 80s, and the people at Element Animation and Mojang agreed. The references that I found, often used the same colour of blue for all of the lights in the shot, and instead used brightness to create contrast and define shape. They also often used silhouettes.

The video itself takes place in the Deep Dark, a place deep underground with no natural light. There are bioluminescent blocks, soul lanterns, and candles that provide dim lighting.

Next to the Warden, we have two characters, Alex and Steve. Alex has a lot of bright greens and oranges in her design, while Steve has a lot of browns and dark blues in his design, making Alex easily stand out in the set, but Steve will more easily blend in. Because of this, we needed to pay extra attention to make sure that Steve has enough contrast with the background and that Alex doesn’t stick out too much compared to Steve.

The Deep Dark is fully underground, which means that we’d have a ceiling of deepslate (a kind of rock). We generally want the background to be really dark, which would basically mean that the ceiling of deepslate would be entirely black. An empty background would be a bit boring, so I came up with the idea to add those glowing dots from the sculk in the ceiling as if it’s a sky filled with stars.

Based on all of this, I created eight rules for the lighting of the video:

1. Hard slightly blue tinted lights, creating lots of shadows. Hard shadows create interesting shapes in the shadows, make the shadows deeper, and make for nicer looking volumetrics The blue tint helps to convey the bioluminescence of the sculk and it naturally reduces the vibrancy of the image, helping with scariness, but since the sculk is already a particular shade of blue and the stone is meant to be somewhat grey, the blue tint should remain slight as to keep the expected look of the blocks. Similarly, the characters have specific colours in their skin that we do not want to distort too heavily.
2. No fill on the set and minimal fill on characters. Shadows very quickly fall off to pure black. The atmosphere should be eerie, and in order to do that, we use a lot of shadows and pure blacks. These very dark areas create an eerie sensation since we can’t really see what is there, creating what you could call “The Unknown”, which we find scary. For characters, having the shadows that fall on them be so dark, gives the feeling of being in constant threat of getting absorbed by the darkness / “The Unknown”, furthering the eeriness.
3. Characters are by default either brightly lit on a dark background or dimly lit / fully in shadow on a bright background. The goal is to create as much separation between the characters and the background. The colour of the lights are essentially fixed to a slight blue tint and to create enough colour separation between Steve and the ancient city, is going to leave one of the two looking very strange. Therefore, we use separation of luminance, which essentially results in the rules above.
4. Characters may also be fully in shadow on a somewhat dimly lit background with their faces lit brighter than the background. It’s about luminance separation between the characters and the ancient city, and this also produces enough of that separation.
5. Characters are by default lit by only one light. It should feel like the characters are in constant threat of getting lost in the darkness. So, by lighting them with only one light and letting the side not illuminated by that light fall into the shadows, we visually imply this threat. They can be lit by a front light or a backlight, but in general not both.
6. Torches and candles are dim and only used for accents. The ancient city is meant to be eerie and dark, so we want the main and only colour to be blue, even on the characters. Therefore any torches and candles shouldn’t give off much light or very saturated oranges, as if the ancient city is taking away their power. They should only be used for accents.
7. No global environment fog, but local pockets of smoke. A global environment fog will raise the black level, which will actually reduce the separation between the characters and the ancient city. Local pockets of smoke allow us to still create atmosphere in a controlled manner without raising the black levels in areas that we do not want.
8. Every shot must have smoke in it, however minor. We use smoke to create atmosphere and to create solid areas of bright colours in the image if we need to create separation with the characters. In order to keep the look and feel of the video consistent, every shot must have some amount of smoke, even if it is very minor.

This is combined with a few quick paintings to showcase the visual rules.

With all of this defined, we can plan out the lighting for each shot by simply doodling on top of a screengrab. Rather than creating a concept painting (called a colour key) for each shot, we create an abstract idea of the lighting, allowing us to focus on the important parts, like storytelling and making sure that we are creating enough separation between the characters and the set. Since it’s abstract, anything that isn’t important can simply be left out and whoever does the lighting for that shot, can fill in the blanks based on those eight rules and our references.

Smoke

In order to be able to create enough separation between the characters and the background, we needed to be able to have the background get pitchblack. Global environment fog would not work, since it would raise the blacklevel of objects in the distance. But we’d still need smoke in order to create silhouettes, which is important in getting the separation needed. We could have used actual volumetrics, but I wanted it to feel more like Minecraft. Instead, we went with a sprite-on-particle approach. I created a particle simulation using Bifrost in Autodesk Maya, which then had the smoke particle texture found in Minecraft applied to every particle. The opacity of each particle was set to a very low amount in order to create a volumetric effect that is still “blocky”, just like the game. This was all packaged into an asset, containing eleven variants of the smoke simulation, that we could drop into the shot during lighting.

Lighting

Joel Houghton, the art director at Element Animation, and I worked together to do the lighting for this video, using the lighting plans as a guide. Most of the lighting was achieved using focused area lights, creating pockets of light with harsh shadows. The sculk sensors, which light up, were mesh lights using the tendrils as the shape.

While normally, a character would be lit with something that is somewhat close to a three-point lighting setup (a key light, fill light, and edge light), Minecraft characters are blocky, making such lighting setups work very differently. For this video, however, we are mostly either lighting the character brightly and putting them in front of a very dark area, or silhouetting the character. This slightly simplifies the lighting setups normally required for Minecraft animations, but we’re still dealing with the problem that a flat face isn’t going to show lighting directions. In some shots this is fine, but in other shots we need that information, either by adding in a specular edge on one side of the face or a gradient over the face.

Steve’s hair is another tricky area as well. If we light him normally and then put him in front of a very dark area, his dark hair will blend into the background, which will look weird. We need to preserve Steve’s outline. One trick that we use is to add a bit of smoke behind just his head.

Alex’s design uses brighter colours than Steve’s design, which makes Alex naturally stand out more than Steve, while we often don’t want that. Often we’ll either brighten up Steve or darken Alex to compensate for this.

Compositing

Since a lot of the inspiration for the lighting style came from 80s horror movies, I wanted it to at least somewhat feel like it was shot on photographic film. We are using an ACES colour management pipeline, but with a custom colour rendering that is less contrast-y and has display gamut compression. Minecraft often uses purely saturated colours in its textures, which can cause problems when combined with saturated lights, when using the sRGB gamut to render in. The ACEScg gamut is much wider and with that provides more ideal results. However, the default Reference Rendering Transform found in ACES doesn’t produce nice results for the art style of Element Animation, so I created a custom OpenColorIO configuration with a custom rendering transform that solves our issues with the default RRT.

Next to that, during compositing we are adding halation, a glow around bright objects that is more extreme for the reds. The halation creates a colour shift in the glow, bringing in some reddish tints in an otherwise very blue image, and adding a bit more visual interest overall.

Grain is also important in photographic film, but it also shouldn’t be a lot. None of the other animations that Mojang has, contain even the smallest amount of grain or noise. Unfortunately, video compression handles grain very badly, removing the grain from the image and replacing it with compression artefacts. Sometimes a static image of grain is used instead of animated grain, but when the camera moves the static grain becomes very noticeable and is still as bad for compression as animated grain. Instead, I created a custom node in Fusion that uses the world position pass to add grain that sticks to objects. This adds the texture that we are used to from photographic film, while also being very compression friendly.

Preview of just the grain for one of the shots in the video.

Rigs

For this video, I created custom rigs for Alex, Steve, the Warden, and the bats. The characters are made entirely out of cubes that do not bend or deform, rather they move and rotate. The limbs and the torso use simple IK and FK controls. The face is achieved using animated UVs. Every controller also has a hidden surface to make it easier to select that controller. Instead of having to click on the controller itself to select, for example, the left foot, the animator can simply click on the lower half of the leg. Combining it with the tweak mode in Autodesk Maya’s Move Tool settings, allows for interactive posing by dragging on different parts of the body. The rigs are made to be as simple as possible in order to increase performance, allowing artists to easily animate larger shots in real time.

For characters like Alex and Steve, we use a generic rig where the texture applied to them contains all of the character design, like clothing, hair, eye colour, and more. Steve’s mouth is a bit different. Instead of a normal mouth that can change width and height and optionally show teeth, Steve’s mouth just moves up and down. We originally looked at animations produced by Mojang themselves and noticed that they have his mouth move up and down to create different facial expressions. Since this video is for Mojang, we decided to simply copy them.

The Warden uses the same base rig as Alex and Steve, but then with added control for the ears, rib-like things, and the Warden’s heart.

Reel