Music Understanding
Analyze audio content and extract music information.
Overview
Use the Music Understanding framework to analyze your audio content and receive information like the rhythm, pace, loudness, key, and instrument activity.
[Image]
You create a MusicUnderstandingSession, specify which analysis to run, and receive the results. When complete, the framework presents the results in a single aggregate from MusicUnderstandingSession.SessionResult, or as a livestream of partial results. You can provide audio from an AVAsset for file-based analysis of a complete audio track, or an AsyncSequence of AVReadOnlyAudioPCMBuffer for a stream analysis of audio buffers.
Some analyses also support incremental delivery through async sequences on the session. Use loudnessResults to receive values intermittently while analysis is in-progress. Call cancel() to stop an in-progress analysis.
The framework returns each analysis type as a dedicated result structure, giving you detailed, time-stamped data for the requested analyses:
- Rhythm
Beat positions, bar boundaries, and tempo in beats per minute (BPM).
- Key
Musical key (tonic and mode) detected over time ranges.
- Loudness
Perceptual loudness measurements following the ITU-R BS.1770 standard, including integrated, short-term, momentary, and peak values.
- Pace
Perceptual energy and momentum expressed as an events-per-minute rate, independent of fixed tempo.
- Structure
Song-level organization in terms of sections, segments, and phrases.
Topics
Essentials
Audio analysis
Content results
MusicUnderstandingSession.SessionResultRhythmResultKeyResultLoudnessResultPaceResultStructureResultInstrumentActivityResult