Checkerboard render detail

A Quick Primer on Checkerboard Rendering

Posted on:
February 28, 2017
Posted in:
Computers, Games, Graphics

Checkerboard rendering has been a popular topic recently because of the PS4 Pro launch. The upgraded PS4 isn’t quite powerful enough to make the leap from 1080p to 4k without sacrificing some visual quality (either in terms of a lower framerate or simpler graphics).

A 1080p image (1920×1080) contains 2,073,600 pixels. 4k is exactly twice as wide and tall (3840×2160) but that means it contains exactly 4 times the number of pixels – 8,249,400. While the PS4 Pro is certainly faster than the PS4, it turns out it can pump out about 2x the number of pixels per frame, not 4x.

Conventional Upscaling

Conveniently, 1080p is exactly 1.5x 720p in both width and height. As a result it contains about 2x the number of pixels. It wasn’t uncommon in the previous console generation to have games render at 720p and then upscale to 1080p – it cuts the number of rendered pixels in half and you get a regular scaling ratio. This idea of reducing the pixel count to improve speed is nothing new.

The problem with conventional upscaling, though, is that the rendered pixels don’t map onto the display pixels as evenly as you’d hope. Every 4 pixels from 720p maps to 9 pixels in 1080p, with only the top left pixel representing a true rendered value and the other 8 being interpolated.

Splatoon 720p thumbnail
Left: 720p, Right: 720p conventionally upscaled to 1080p

The abundance of interpolated pixels vs. actual data leads to a blurry look.

Checkerboard Rendering

The appeal for Checkerboard Rendering over conventional upscaling is that by rendering the pixels in a checkerboard pattern you vastly increase the amount of actual data that makes it to the screen vs. the number of pixels that have to be interpolated.

The easiest method for Checkerboard rendering is to simply render every other pixel and then to interpolate the remaining pixels as a simple average of its four neighbors.

Extreme Closeup of interpolation
  The rendered pixels (and black un-rendered pixels) give the distinct checkerboard pattern. Here, one pixel is filled using the average of colors from above, below, left, and right.

The resulting image does have a bit of a noisy look to it, but also tends to preserve details better than a simple bilinear upscale.

Checkerboard render detail
Left: rendered pixels; Right: rendered pixels + interpolated pixels

After the checkerboard upscale there are a number of post-processing effects that PS4 Pro games use to clean up noise (including simple motion blur and including pixels from previous frames while interpolating) but they’re kind of beyond what I’m able to demonstrate in Adobe Photoshop.

Why Does This Look Better?

Looking at the image above it’s clear we’re not going to get the crisp full-resolution image using this technique – but it perhaps looks better than 720p upscale – right? The question of course is why would it look better? After all – it’s not representing any additional data.

The answer is that our brains are very good at correcting for high-frequency noise. Whether it’s dithering, half-tone screen printing, or JPEG artifacts, we’re very good at getting a feel for the image and then correcting for the tiny 1-pixel errors.

It’s one of the reasons, in fact, that JPEG sticks around as well as it does. There are better image formats out there – image formats that have a better peak signal-to-noise ratio and that, when you blow up the image, deliver obviously better results. But a funny thing happens when you display the images at 100% size on a 100dpi or more monitor – we start to see less of the encoding errors and instead perceive an overall feeling of “sharpness.”

There’s also the matter of mapping the input pixels to the output resolution. When upscaling from 720p, as I mentioned above, for every 1 pixel that maps neatly to the output display, 8 pixels have to be interpolated. Only 11% of what you see is data, 89% is interpolation. With a checkerboard method – 50% of what you see is data, 50% is interpolated.

Examples

These images were generated using 4k renders from CEMU (since those are easy to find and work with).

Bayonetta 2 at 720p
Bayonetta 2 – Checkerboard 1080p
Splatoon – 720p
Splatoon – Checkerboard 1080p
Xenoblade Chronicles X – 720p
Xenoblade Chronicles X – Checkerboard 1080p

All of these images are nearest-neighbor resized to 4k since that’s the first common multiple of 720p and 1080p.