steliyanh/kadr-audio

The thing worth knowing first

Most of a typical music library cannot be exported into a video. Apple Music tracks are DRM-protected: MPMediaItem.assetURL is nil for anything from a subscription, and no API turns one into a file. Only music the user owns — purchased, or imported themselves — exposes a URL.

A consumer that does not know this ships a picker where nearly everything the user taps fails, with nothing useful said. This package makes that case explicit in two ways:

// Offer only what will actually work
let usable = try MusicLibrary.usableSongs()

// Or resolve and handle the refusal properly
do {
    let track = try MusicLibrary.audioTrack(for: picked)
    video = video.music(track.volume(0.4).ducking(0.2))
} catch let error as MusicLibraryError {
    // "Songs from Apple Music are protected and can't be exported.
    //  Music you own or imported yourself will work."
    show(error.localizedDescription, error.recoverySuggestion)
}

The error text names the reason rather than suggesting a retry, because retrying with another Apple Music track fails identically.

Audio that is actually audible

A fresh iOS app gets the .soloAmbient session category, which obeys the ring/silent switch. So a user with a muted phone opens a video editor and the preview is silent, with nothing on screen explaining why. Nothing in the kadr family configured this before v0.2, which means every consumer had the bug.

try AudioSession.configure(.preview)   // audible with the phone muted
try AudioSession.activate()            // takes audio focus — do this when the preview appears
// ...
try AudioSession.deactivate()          // lets the user's music resume

Configuration and activation are separate calls on purpose. Activation takes audio focus from other apps, so a host that activates at launch silences the user's music for as long as the app is open.

Interruptions are a stream:

for await interruption in AudioSession.interruptions {
    switch interruption {
    case .began: player.pause()
    case .ended(let shouldResume) where shouldResume: player.play()
    case .ended: break
    }
}

shouldResume is the system's opinion, not a formality. Ignoring it is how an app ends up talking over a phone call.

Voiceover

let recorder = VoiceoverRecorder()
guard await VoiceoverRecorder.requestAuthorization() else { return }

try recorder.start()                     // session configured, latency measured
// ... the performer speaks against the preview ...
let take = try recorder.stop()

video = video.audio { take.audioTrack(startingAt: previewTime) }

The latency correction is the point. A voiceover is performed against playback: the performer reacts to audio that already left the device late, and their voice arrives at the input late again. Both delays land in the recording.

Wired headphones add a couple of milliseconds. Bluetooth adds 150–200 ms — several frames at 30 fps, and unmistakable on a lip-sync. audioTrack(startingAt:) places the take earlier by exactly the latency measured when recording began.

Measured at start() rather than at construction, because plugging in AirPods between the two changes the answer by two orders of magnitude.

Required entitlement: NSMicrophoneUsageDescription.

Loudness

Every social platform normalises on upload — Instagram, TikTok and YouTube all target roughly −14 LUFS. A composition mixed by ear is re-levelled after publishing, usually downward and unevenly across clips mixed at different times. Measuring first is the only way to control what the platform does rather than discover it.

let measured = try await Loudness.measure(url: musicURL)
let track = AudioTrack(url: musicURL).normalized(from: measured, to: .social)

measure(url:) reads every sample, so it costs about what decoding the file costs. Measure once and keep the result — that is why normalized(from:to:) takes a measurement rather than making one. Loudness.integrated(samples:sampleRate:channels:) is the pure arithmetic underneath, if you already have samples.

Implemented per ITU-R BS.1770-4: K-weighting, 400 ms blocks at 75% overlap, absolute gate at −70 LUFS and a relative gate 10 LU below.

Loudness.integrated takes samples rather than a URL on purpose — it is pure arithmetic, so it is testable anywhere, and the caller decides when to pay for reading a five-minute file.

Gain above 1.0 raises peaks as well as loudness, and nothing here limits them, so a very quiet source pushed to −14 LUFS may clip. Platforms normalise downward far more often than upward, so the common direction is the safe one.

Importing an audio file

A picker returns a URL, and a content-type filter is a guess — .audio admits files with no decodable audio track, and a user can rename anything.

let track = try await AudioFile.audioTrack(for: pickedURL)
// throws: "“holiday.mov” doesn't contain any audio."
//         "Pick a music or voice file — a video file won't work here."

One asset load turns a silent failure at export into a sentence at import.

Before recording

if AudioSession.recordingWouldCapturePlayback {
    // On the speaker the microphone hears the playback too, so the take
    // arrives with the backing track already in it.
}

Quick Start

.package(url: "https://github.com/SteliyanH/kadr-audio.git", .upToNextMinor(from: "0.7.0")),

Add KadrAudio to your target's dependencies. Kadr is pulled in transitively — 0.1.x resolves >=0.20.0, <0.21.0.

Use .upToNextMinor, not from:. from: means .upToNextMajor, and SwiftPM does not special-case 0.x — so from: "0.7.0" would accept every future 0.x release including breaking ones.

Required entitlement: NSAppleMusicUsageDescription in your app's Info.plist. Without it, requesting authorization terminates the app.

Platforms

iOS 17+ and visionOS 1+. macOS is declared in the manifest so the package resolves against kadr and can be tested in CI, but the MediaPlayer surface is unavailable there. tvOS is excluded outright — MPMediaPickerController does not exist.

Roadmap

See ROADMAP.md. v0.1 is music-library resolution. Loudness normalisation (LUFS) and AVAudioEngine effects are the next candidates — both belong here for the same reason, and neither is in core.

The kadr ecosystem

| Package | Purpose | |---|---| | kadr | The engine. Declarative video composition and export — clips, tracks, transitions, filters, overlays, keyframe animation. | | kadr-ui | SwiftUI components — preview, timeline, inspector, overlay host, keyframe editor. | | kadr-persistence | Save a composition to a file and open it again. | | kadr-audio | Music library, voiceover recording, LUFS loudness. | | kadr-captions | SRT, VTT, iTT, ASS and SSA parsing and authoring. | | kadr-photos | Photos library integration. |

And a reference application: Kadr Studio, a short-form vertical video editor built on all six.

License

Apache-2.0. See LICENSE.

Contributions are accepted under the Contributor License Agreement, which is signed once and covers all future contributions. It does not transfer ownership — you keep the copyright in your work.

Package Metadata

Repository: steliyanh/kadr-audio

Default branch: main

README: README.md