Rendering in FxPlug

Rendering in FxPlug

Rendering in FxPlug Use Metal or other frameworks to render images with your FxPlug plug-in.

Discussion

FxImageTile is a new class for FxPlug 4 that combines and improves the frame information previously found in FxRenderInfo and FxImage, including bounds, pixel transforms, field and field order, and the image’s origin. Incoming images are now presented as an //com.apple.documentation/documentation/IOSurface, which is one of the biggest changes to FxPlug 4.

Rendering always involves these four steps:

  1. Gather the state your plug-in needs to render by checking parameter values and making calculations. //com.apple.professionalvideoapplications/documentation/Professional Video Applications/communicating with the plug in state explains how to prepare required information for you plug-in to render.
  2. Tell the host application what the output area will be, given the input area. See //com.apple.professionalvideoapplications/documentation/Professional Video Applications/working with tiled images — Set the destination rectangle bounds.
  3. For each tile of the output the host requests, tell it which tile of the inputs you need to render. //com.apple.professionalvideoapplications/documentation/Professional Video Applications/working with tiled images explains how to set input and output bounds.
  4. Draw each output tile. See `doc://com.apple.professionalvideoapplications/documentation/professional_video_applications/FxTileableEffect/renderDestinationImage(_:sourceImages:pluginState:at:)`.

When rendering, your plug-in will be given an FxImageTile for output. Use this to return a texture to the host application when your plug-in has finished rendering. The outputImage contains a deviceRegistryID, which identifies what graphics card the texture is on. This is a read-only property.

For an example of how to work with deviceRegistryID, see the FxDynamicRegistration example. Specifically, look at PAEMTLBrightnessRenderer.m, in renderWithBrightness where the OutputTexture uses the OutputImage’s deviceRegistryID value to create Metal state, and Input and Output textures.

Note: Your plug-in may suffer from a performance hit if you set //com.apple.documentation/documentation/IOSurface/kIOSurfacePixelSizeCastingAllowed on your IOSurface, so its use is generally not recommended. You set this flag to ensure that pixel locations are kept in order in VRAM, and then you can use custom pixel packing in your plug-in. If you do need this flag, when converting from 4:4:4 to 4:2:2 formats for example, the host app will provide a copy of the input and output IOSurface, so you can unpack your specific pixel format in the plug-in, and likewise pack your specific pixel format into the output //com.apple.documentation/documentation/IOSurface.