Analyzing media

Use the FxPlug analysis API to analyze frames of source media before rendering them.

Overview

Frame-by-frame analysis of video clips is commonly required for rendering tasks that occur in an FxPlug plug-in. For example, tracking of objects and stabilization of video both require a pre-render analysis of video frames before rendering.

FxPlug 4.1 introduces FxAnalysisAPI, which FxPlug plug-ins can use to analyze video frames from source media in Final Cut Pro X or Motion. FxAnalysisAPI uses the host app’s built-in analysis infrastructure so that your plug-in’s analysis can be performed in the background, and you can view the progress UI in the same location as the host’s own analysis processes. This provides a consistent user experience for both native effects and FxPlug plug-ins that require analysis.

The example plug-in FxBrightnessAnalysis demonstrates how to implement the analysis APIs for a filter that analyzes all frames and then adjusts the brightness of the frames to a consistent level.

Set the desired analysis time range

Your plug-in can tell Motion and Final Cut Pro what time range to analyze. In the following example, using desiredAnalysisTimeRange(_:forInputWith:), the desiredRange is set to whatever the host provides in inputTimeRange.

Set up your plug-in for analysis

Set up data arrays or variables that will be used later for analysis in the setupAnalysis(for:frameDuration:) method for FxAnalyzer. You can also use other FxPlug APIs like FxParameterSettingAPI_v5 within this method to set parameter states during analysis, as in the example below.

Store data to persist between application launches

Your plug-in needs to store the analysis data in a parameter so that it can persist between application launches. You typically do this with a custom parameter added in addParameters(). You also supply the user with a button to start the analysis in addParameters(). When implementing custom parameters, implement the class(forCustomParameterID:) method so your custom parameters can be serialized and deserialized properly.

Start the analysis

In the FxBrightnessAnalysis example plug-in, when the user clicks the Start Analysis push button parameter, the host app calls the plug-in’s -startAnalysis: function, which then initializes the FxAnalysisAPI and requests that the host start a forward analysis.

[Image]

Analyze the frames and store the results

Use the analyzeFrame(_:at:) method allows you to analyze each frame and store results. In the following example the plug-in samples all the pixels to determine the average luminance of each frame, and then stores the averageLuminance value for each frame (frameTime) in the analyzedData array.

The example above calls the -saveAnalyzedData: function shown below every 10 frames. The function then saves the results into a custom parameter using the setCustomParameterValue(_:toParameter:at:) method of FxParameterSettingAPI_v5.

Clean up after analysis

Use the cleanupAnalysis() method to do any necessary cleanup after the analysis is complete. The following example is from the FxBrightnessAnalysis plug-in.

Determine the Current Analysis State

The -analysisStateForEffect: method returns the current analysis state as one of the following keys:

kFxAnalysisState_NotAnalyzing: Not analyzing.
kFxAnalysisState_AnalysisRequested: Analysis has been requested, but it hasn’t started yet.
kFxAnalysisState_AnalysisStarted: Analysis has started.
kFxAnalysisState_AnalysisCompleted: Analysis has completed.
kFxAnalysisState_AnalysisInterrupted: The analysis was interrupted.

Although not necessary, you can use these states to provide UI feedback, and to determine when the analysis is complete or whether it was interrupted.

Retrieve analysis data for use in rendering

During rendering, retrieve the analyzed data for each frame from the custom parameter in your pluginState(_:at:quality:) method. Use getCustomParameterValue(_:fromParameter:at:) to get the frame’s value, process a result that your renderer will need, then pack it into the plug-in state for that frame. This example shows how to retrieve the analysis data for use in rendering. The steps that follow take you through the example in more detail.

In the above example from the FxBrightnessAnalysis plug-in:

The analysisData for the current frame is retrieved from the custom parameter using getCustomParameterValue(_:fromParameter:at:).
A pointer to the data is then passed to a function called brightnessAtTime:::: to determine the analyzed brightness value of the current frame.
The desiredBrightness value that was set in the kParam_Brightness float slider parameter by the user is retrieved.
The actualBrightness and desiredBrightness values are packed into the brightnessData struct and returned as renderData.

See Communicating with the plug-in state for more information on working with pluginState.