Improving edge-rendering quality with multisample antialiasing (MSAA)

Apply MSAA to enhance the rendering of edges with custom resolve options and immediate and tile-based resolve paths.

Overview

MSAA improves the rendering of lines and edges by using several color and depth samples per pixel. An MSAA texture normally stores two or four samples per pixel. But some GPU devices support eight samples per pixel. After the app renders a scene to the MSAA texture, it resolves it to a normal texture that contains one sample per pixel. The built-in resolve uses a simple average, but sometimes you want to use a custom resolve.

A custom resolve is essential when the built-in resolve isn’t sufficient. For example, you may want to tone-map high dynamic range (HDR) samples before averaging them. HDR means that color values exceed the normal color intensity range of zero to one. Tone-mapping means that values are compressed into the zero-to-one range for output to a display device.

This sample shows you how to use a custom resolve with immediate-mode devices and tile-based deferred rendering devices. The immediate-mode custom resolve uses a compute kernel to process the MSAA texture, while a tile-based shader works before the rendering is completed, saving time and memory bandwidth. The following image shows the results of applying MSAA to a colored set of thin shards with one, two, four, and eight samples per pixel.

[Image]

The sample displays a user interface for you to control the code paths to use MSAA. The app customizes the options presented based on the current device. Some devices support eight samples or the tile-based resolve. You can choose between the built-in resolve or a custom resolve that replicates the built-in choice or applies a tone-mapping operator to the samples. You can choose between two, four, or eight samples per pixel and the immediate-mode path or the tile-based resolve.

There’s a checkbox to toggle MSAA on and off. There’s also a list of reduced resolutions—ranging from one-sixteenth resolution to full-resolution—that makes it easier to see the difference MSAA makes. Pressing the space bar on a macOS device or tapping the display on an iOS device causes the shards to rotate.

Configure the sample code project

To run this sample, you need Xcode 12 or later on a macOS, iOS, or tvOS device or simulator. To experience the tile-based custom resolve feature, you need a physical device that supports MTLGPUFamily.apple4, such as:

A Mac with Apple silicon running macOS 11 or later
An iOS device with an A11 chip or later running iOS 13 or later
A tvOS device with an A12 chip or later running tvOS 14.5 or later

On a device with an Intel or AMD GPU, or on a simulator, this sample runs with the custom resolve in immediate-mode rendering (IMR) only.

Check for MSAA support

GPUs on Apple devices can support a variety of sample counts but need to support at least a sample count of four. The app checks the device’s supported sample count with supportsTextureSampleCount(_:) and uses a default sample count of four.

Create a multisample texture

The app needs to create a multisample texture before rendering the scene. The following code shows that the texture declaration is the same as a normal texture.

The app configures the texture using a texture descriptor. It uses MTLTextureType.type2DMultisample as the texture type, and sets the sampleCount property with the current count selected in the UI.

If the app is running on a device that supports tile shaders, then the usage and storage mode enable the renderer to take advantage of memoryless storage. That means the device discards the texture when it finishes the tile-based resolve. In comparison, the immediate-mode custom resolve requires the texture to stay in memory, and the renderer uses a separate compute pass to resolve the MSAA texture. The app sets the shader-read flag to allow the compute kernel to access the texture. The following code shows how to set the usage and storageMode properties in the descriptor.

The renderer also creates a resolve texture to store the final image that’ll be copied to the drawable texture. The following code shows how it uses the same width and height for both textures. The _resolveTextureDescriptor uses a single sample for the resolve texture.

Configure the render pipeline

The renderer ensures the render pipeline state object (PSO) has the current sample count as the texture to use as a render target. It also decides the correct fragment function depending on whether HDR is enabled or not. The app uses a separate custom resolve step if HDR is enabled, so it doesn’t apply tone-mapping. But if antialiasing is disabled, then it uses a fragment function that applies tone-mapping. If the HDR resolve option is chosen in the UI, then toggling MSAA on and off shows the result of using tone-mapping. The following code shows how the renderer customizes the render pipeline descriptor before it creates the PSO.

After the the renderer sets up the PSO, it also sets up the render pass descriptor to use the multisample texture as a color attachment. If MSAA isn’t enabled, then it sets up the _resolveResultTexture as the main color attachment. But if MSAA is enabled, then the following code shows how the renderer chooses the store action. If tile shaders are available or you choose the built-in resolve, then Metal resolves the texture without needing an extra compute pass. The code also shows how to prepare the render pass for a resolve using tile shaders that the README discusses later.

Use a custom resolve in IMR

The built-in resolve averages the color samples in the texture together to create the final pixel color in the resolve texture. But developers can use a custom resolve by using a compute kernel. Devices that don’t support tile-based rendering need to use a compute kernel. The compute kernel reads the multisample texture and writes a normal texture. The following code shows a compute kernel that accumulates the samples and calculates their average.

The following code shows how to resolve an MSAA texture with the compute kernel. It sets the two input textures for the kernel and dispatches the work.

Use a custom resolve to tone-map an HDR image

Images with HDR need a tone-mapping operator before they’re output to a computer display. The sample shows how to use a custom resolve to prepare the image. The following code shows the simple tone-mapping operator that uses luminance to scale the input color.

The sample uses a different kernel function, hdrResolveKernel, that tone-maps the samples before it takes the average. Developers may choose to use a similar approach for applications where two mathematical functions don’t commute. In this case, tone-mapping the average of four samples isn’t necessarily the same numerical result as averaging four tone-mapped samples.

Use a tile-based resolve on an Apple silicon device

On Apple silicon devices, this app can resolve a multisample texture using a tile-based resolve, allowing the device to perform the resolve before the device saves the tile to memory. The app gains two advantages: The multisample texture doesn’t need to be stored, saving memory space, and it doesn’t require a second render or compute pass, saving compute time and memory bandwidth.

Not all devices support tile shaders, so the app uses the following code to check whether the device supports the feature.

If the device supports the feature, then the renderer creates a new PSO that uses the _averageResolveTileKernelFunction. The following code shows how to set the kernel function, pixel format, sample count, and whether the kernel function expects the tile size to match the thread group size.

After it creates the PSO, the renderer can dispatch the work before it finishes encoding the work. The following code shows how the app avoids this step when it uses the built-in resolve.

Finally, the following code shows a custom kernel function that resolves the MSAA texture when the input texture has HDR samples. The other custom kernel function in the app is similar, but it only averages the sample values, like the built-in resolve.