fdenis75/mosaickit

Features

• 🚀 Metal-Accelerated Processing - Hardware-accelerated mosaic generation for maximum performance
• 🎨 Multiple Layout Algorithms - Classic, custom, auto-screen, dynamic, and iPhone-optimized layouts
• ⚙️ Configurable Density Levels - From XXL (minimal) to XXS (maximal) frame extraction
• 📦 Multiple Output Formats - JPEG, PNG, and HEIF with configurable compression
• 🔄 Batch Processing - Intelligent concurrency management for processing multiple videos
• 🎯 Hardware-Accelerated Frame Extraction - Uses VideoToolbox for optimal performance
• 📊 Overlay Annotations - Per-frame labels (timestamp, index), customisable metadata headers, watermarks, and Color DNA strips
• 🎬 Video Preview Generation - Create short highlight reels from any video, either exported to file or as a live AVPlayerItem composition

New in 1.1.4

• Adaptive metadata headers now size themselves from the actual rendered content instead of relying on thumbnail-row heuristics.
• File paths get their own shrink-to-fit header row, which keeps long source URLs and paths readable.
• Per-frame label typography now scales from the thumbnail’s dominant dimension for more consistent captions across portrait and landscape layouts.
• Preview compositions and exported preview videos now show each extract's source timestamp in a bottom-left dark pill for the first second of the extract.
• On macOS 26 / iOS 26, preview video composition setup now uses AVVideoComposition.Configuration and AVVideoCompositionCoreAnimationTool.Configuration instead of the older mutable composition setup.

Requirements

• macOS 15.0+ or iOS 15.0+
• Xcode 16.0+
• Swift 6.2+
• Metal-capable device

Installation

Swift Package Manager

Add MosaicKit to your Package.swift:

dependencies: [
    .package(url: "https://github.com/fdenis75/MosaicKit.git", from: "1.1.4")
]

Then add it to your target dependencies:

targets: [
    .target(
        name: "YourTarget",
        dependencies: ["MosaicKit"]
    )
]

Xcode

1. Go to File → Add Package Dependencies...
2. Enter the repository URL
3. Select the version you want to use
4. Click Add Package

Quick Start

Simple API (Recommended)

import MosaicKit

// Simple one-step generation
let videoURL = URL(fileURLWithPath: "/path/to/video.mp4")
let outputDir = URL(fileURLWithPath: "/path/to/output")

let generator = try MosaicGenerator()
let config = MosaicConfiguration.default

let mosaicURL = try await generator.generate(
    from: videoURL,
    config: config,
    outputDirectory: outputDir
)

print("Mosaic saved to: \(mosaicURL.path)")

Advanced Usage (Direct Access)

For more control, use MetalMosaicGenerator directly:

import MosaicKit

// Create a video input from a file URL
let videoURL = URL(fileURLWithPath: "/path/to/video.mp4")
let video = try await VideoInput(from: videoURL)

// Configure mosaic settings
var config = MosaicConfiguration.default
config.width = 5000
config.density = .m
config.format = .heif
config.outputdirectory = URL(fileURLWithPath: "/path/to/output")

// Generate the mosaic
let generator = try MetalMosaicGenerator()
let mosaicURL = try await generator.generate(
    for: video,
    config: config
)

print("Mosaic saved to: \(mosaicURL.path)")

Configuration Options

MosaicConfiguration

public struct MosaicConfiguration {
    var width: Int                      // Output width (default: 5120)
    var density: DensityConfig          // Frame density (default: .m)
    var format: OutputFormat            // JPEG, PNG, or HEIF
    var layout: LayoutConfiguration     // Layout settings
    var includeMetadata: Bool           // Add metadata header
    var useAccurateTimestamps: Bool     // Precise frame extraction
    var compressionQuality: Double      // 0.0 to 1.0 (default: 0.4)
    var outputdirectory: URL?           // Output directory
    var overlay: OverlayConfiguration   // Per-frame labels, header, watermark, Color DNA
}

Density Levels

Control the number of frames extracted from your video:

// Fewer frames (faster processing, smaller file)
config.density = .xxl  // Minimal - ~25% of base calculation
config.density = .xl   // Low - ~50% of base calculation
config.density = .l    // Medium - ~75% of base calculation

// More frames (slower processing, larger file, more detail)
config.density = .m    // High (default) - 100% of base calculation
config.density = .s    // Very high - 200% of base calculation
config.density = .xs   // Super high - 300% of base calculation
config.density = .xxs  // Maximal - 400% of base calculation

Layout Options

Choose from multiple layout algorithms:

var layout = LayoutConfiguration()

// Aspect ratios
layout.aspectRatio = .widescreen  // 16:9
layout.aspectRatio = .standard    // 4:3
layout.aspectRatio = .square      // 1:1
layout.aspectRatio = .ultrawide   // 21:9
layout.aspectRatio = .vertical    // 9:16 (portrait)

// Layout modes
layout.useCustomLayout = true     // Three-zone layout with large center thumbnails
layout.useAutoLayout = true       // Adapt to screen size
// Or use classic grid layout (default)

// Visual settings
layout.visual.addBorder = true
layout.visual.borderColor = .white
layout.visual.borderWidth = 2.0
layout.visual.addShadow = true

Output Formats

// HEIF - Best compression, smaller file size (recommended)
config.format = .heif
config.compressionQuality = 0.4

// JPEG - Good compression, universal compatibility
config.format = .jpeg
config.compressionQuality = 0.8

// PNG - Lossless, larger file size
config.format = .png

Overlay & Annotations

All overlay layers are controlled through MosaicConfiguration.overlay, an OverlayConfiguration value that groups four independent subsystems. Every property defaults to the original hardcoded behaviour so existing code requires no changes.

config.overlay = OverlayConfiguration(
    frameLabel: FrameLabelConfig(...),   // label drawn on each thumbnail
    header:     HeaderConfig(...),       // top metadata band
    watermark:  WatermarkConfig(...),    // optional branding layer
    colorDNA:   ColorDNAConfig(...)      // horizontal colour strip
)

Per-Frame Labels

Each thumbnail can display a timestamp, a sequential frame number, or nothing:

config.overlay.frameLabel = FrameLabelConfig(
    show:            true,
    format:          .timestamp,     // .timestamp | .frameIndex | .none
    position:        .bottomRight,   // .topLeft | .topRight | .bottomLeft | .bottomRight | .center
    textColor:       MosaicColor(red: 1, green: 1, blue: 1),
    backgroundStyle: .pill           // .pill | .none | .fullWidth
)

Metadata Header

The top band that appears when includeMetadata is true is fully configurable:

config.overlay.header = HeaderConfig(
    fields: [
        .title, .duration, .fileSize, .resolution,
        .codec, .bitrate, .frameRate, .filePath,
        .colorPalette(swatchCount: 8),          // row of colour swatches
        .custom(label: "Director", value: "Jane Doe")
    ],
    height:          .fixed(80),     // .auto (fit content) | .fixed(Int)
    textColor:       nil,            // nil → platform default
    backgroundColor: nil             // nil → semi-transparent dark default
)

Watermark

Stamp text or an image onto the assembled mosaic:

// Text watermark
config.overlay.watermark = WatermarkConfig(
    content:  .text("© Studio 2025"),
    position: .bottomRight,          // any WatermarkPosition corner or .center
    opacity:  0.35,                  // 0.0–1.0
    scale:    0.12                   // fraction of mosaic width
)

// Image watermark
config.overlay.watermark = WatermarkConfig(
    content:  .image(URL(fileURLWithPath: "/path/to/logo.png")),
    position: .topLeft,
    opacity:  0.5,
    scale:    0.08
)

Color DNA Strip

A thin horizontal band where each column shows the dominant colour of one frame — a classic MovieBarcode-style visualisation:

config.overlay.colorDNA = ColorDNAConfig(
    show:     true,
    height:   24,                    // pixels (minimum 8)
    position: .bottom,               // .top | .bottom
    style:    .barcode               // .barcode (hard columns) | .gradient (smooth)
)

Full Annotation Example

var config = MosaicConfiguration(
    width: 5120,
    density: .m,
    format: .heif,
    includeMetadata: true
)
config.outputdirectory = outputDir

config.overlay = OverlayConfiguration(
    frameLabel: FrameLabelConfig(
        show: true,
        format: .timestamp,
        position: .bottomRight,
        textColor: MosaicColor(red: 1, green: 1, blue: 1),
        backgroundStyle: .pill
    ),
    header: HeaderConfig(
        fields: [.title, .duration, .resolution, .codec, .colorPalette(swatchCount: 8)],
        height: .fixed(80)
    ),
    watermark: WatermarkConfig(
        content: .text("© My Studio"),
        position: .bottomRight,
        opacity: 0.35,
        scale: 0.10
    ),
    colorDNA: ColorDNAConfig(
        show: true,
        height: 24,
        position: .bottom,
        style: .gradient
    )
)

let mosaicURL = try await generator.generate(for: video, config: config)

Advanced Usage

Batch Processing

Process multiple videos with intelligent concurrency:

let generator = try MetalMosaicGenerator()
let coordinator = MosaicGeneratorCoordinator(
    mosaicGenerator: generator,
    concurrencyLimit: 4
)

let videos: [VideoInput] = [video1, video2, video3]

let results = try await coordinator.generateMosaicsforbatch(
    videos: videos,
    config: config
) { progress in
    print("Video: \(progress.video.title)")
    print("Progress: \(Int(progress.progress * 100))%")
    print("Status: \(progress.status)")
}

// Check results
for result in results {
    if result.isSuccess {
        print("✅ Success: \(result.outputURL?.path ?? "unknown")")
    } else {
        print("❌ Failed: \(result.error?.localizedDescription ?? "unknown")")
    }
}

Progress Tracking

Monitor generation progress in real-time using MosaicGeneratorCoordinator:

let result = try await coordinator.generateMosaic(
    for: video,
    config: config
) { progress in
    // progress.progress is 0.0–1.0
    // progress.status indicates the current phase
    print("Progress: \(Int(progress.progress * 100))% - \(progress.status)")
}

Custom Video Input

Create VideoInput manually with specific metadata:

let video = VideoInput(
    url: videoURL,
    title: "My Video",
    duration: 120.0,
    width: 1920,
    height: 1080,
    frameRate: 30.0,
    fileSize: 50_000_000,
    metadata: VideoMetadata(
        codec: "H.264",
        bitrate: 5_000_000
    )
)

Performance Metrics

Track generator performance:

let metrics = await generator.getPerformanceMetrics()
print("Average generation time: \(metrics["averageGenerationTime"] ?? 0)")
print("Total generations: \(metrics["generationCount"] ?? 0)")

Cancellation

Cancel ongoing operations:

// Cancel specific video
await generator.cancel(for: video)

// Cancel all operations
await generator.cancelAll()

// Or cancel batch operations
await coordinator.cancelAllGenerations()

Video Preview Generation

MosaicKit can generate short highlight-reel previews from any video. A preview stitches together evenly-distributed clips from across the video into a single condensed output.

Two delivery modes are available:

| Mode | API | Use case | |------|-----|----------| | Export to file | PreviewVideoGenerator.generate(for:config:) | Share, upload, or store the preview | | Composition | PreviewVideoGenerator.generateComposition(for:config:) | Instant playback in AVPlayer — no file written |

Basic Preview Export

import MosaicKit

let video = try await VideoInput(from: URL(fileURLWithPath: "/path/to/video.mp4"))

let config = PreviewConfiguration(
    targetDuration: 60,          // ~60-second preview
    density: .m,                 // 16 clips
    format: .mp4,
    includeAudio: true,
    outputDirectory: URL(fileURLWithPath: "/path/to/output"),
    compressionQuality: 0.8
)

let generator = PreviewVideoGenerator()
let previewURL = try await generator.generate(for: video, config: config)
print("Preview saved to: \(previewURL.path)")

Instant Composition (No Export)

Generate a ready-to-play AVPlayerItem without writing any file — significantly faster than exporting:

let playerItem = try await generator.generateComposition(for: video, config: config)

let player = AVPlayer(playerItem: playerItem)
player.play()

Every preview extract also displays its source start timestamp as a bottom-left dark pill with white text for the first second of playback. The same overlay is rendered in both live AVPlayerItem compositions and exported preview files.

Preview Configuration

public struct PreviewConfiguration {
    public var targetDuration: TimeInterval          // Target preview length in seconds
    public var minimumExtractDuration: TimeInterval? // Per-clip floor, nil disables
    public var maximumPlaybackSpeed: Double?         // Speed cap, nil disables
    public var density: DensityConfig                // Number of clips (same levels as mosaic)
    public var format: VideoFormat                   // .mp4, .mov, .hevc, etc.
    public var includeAudio: Bool                    // Include audio track in preview
    public var outputDirectory: URL?                 // Output folder (nil = video's parent directory)
    public var fullPathInName: Bool                  // Embed full source path in filename
    public var compressionQuality: Double            // 0.0–1.0
    public var useNativeExport: Bool                 // AVAssetExportSession vs SJSAssetExportSession
}

Clip count scales automatically with video duration. Use extractCount(forVideoDuration:) to inspect the calculated value:

let count = config.extractCount(forVideoDuration: video.duration)
print("Clips to extract: \(count)")

Resolution Cap (macOS 26+ / iOS 26+)

On macOS 26 and iOS 26, you can cap the output resolution to reduce file size:

if #available(macOS 26, iOS 26, *) {
    config.exportMaxResolution = ._1080p   // or ._720p, ._4K, etc.
}

On earlier OS versions the setting is silently ignored and the full source resolution is used.

Preview with Progress Tracking

let coordinator = PreviewGeneratorCoordinator()

let playerItem = try await coordinator.generatePreviewComposition(
    for: video,
    config: config
) { progress in
    print("\(Int(progress.progress * 100))% — \(progress.status.displayLabel)")
}

Batch Preview Generation

let coordinator = PreviewGeneratorCoordinator(concurrencyLimit: 2)

let results = try await coordinator.generatePreviewCompositionsForBatch(
    videos: [video1, video2, video3],
    config: config
) { progress in
    print("\(progress.video.filename): \(progress.status.displayLabel)")
}

let succeeded = results.filter(\.isSuccess)
print("Generated \(succeeded.count)/\(results.count) previews")

if let first = succeeded.first, let playerItem = first.playerItem {
    AVPlayer(playerItem: playerItem).play()
}

Layout Algorithm Details

Custom Layout (Recommended)

Three-zone layout with small thumbnails at top/bottom and large thumbnails in the center:

config.layout.useCustomLayout = true
// Automatically calculates optimal grid based on:
// - Target aspect ratio
// - Video aspect ratio
// - Thumbnail count
// - Density settings

Classic Layout

Traditional grid layout with uniform thumbnail sizes:

config.layout.useCustomLayout = false
config.layout.useAutoLayout = false
// Simple rows × columns grid

Auto Layout

Adapts to your display size for optimal viewing:

config.layout.useAutoLayout = true
// Calculates based on:
// - Screen resolution
// - DPI/scaling factor
// - Minimum readable thumbnail size

Dynamic Layout

Center-emphasized layout with variable thumbnail sizes:

config.layout.useDynamicLayout = true
// Larger thumbnails in center, smaller at edges

Examples

Example 1: High-Quality Mosaic

var config = MosaicConfiguration(
    width: 10000,
    density: .xs,
    format: .heif,
    layout: .default,
    includeMetadata: true,
    useAccurateTimestamps: true,
    compressionQuality: 0.6
)
config.outputdirectory = outputDir

let mosaicURL = try await generator.generate(for: video, config: config)

Example 2: Fast Preview Mosaic

var config = MosaicConfiguration(
    width: 2000,
    density: .xl,
    format: .jpeg,
    layout: .default,
    includeMetadata: false,
    useAccurateTimestamps: false,
    compressionQuality: 0.4
)
config.outputdirectory = outputDir

let mosaicURL = try await generator.generate(for: video, config: config)

Example 3: Square Social Media Mosaic

var config = MosaicConfiguration.default
config.width = 3000
config.layout.aspectRatio = .square
config.density = .m
config.format = .jpeg
config.compressionQuality = 0.8
config.outputdirectory = outputDir

let mosaicURL = try await generator.generate(for: video, config: config)

Example 4: Fully Annotated Mosaic

var config = MosaicConfiguration(
    width: 5120, density: .m, format: .heif, includeMetadata: true
)
config.outputdirectory = outputDir
config.overlay = OverlayConfiguration(
    frameLabel: FrameLabelConfig(format: .timestamp, position: .bottomRight, backgroundStyle: .pill),
    header:     HeaderConfig(
        fields: [.title, .duration, .resolution, .codec, .colorPalette(swatchCount: 8)],
        height: .fixed(80)
    ),
    watermark:  WatermarkConfig(content: .text("© My Studio"), position: .bottomRight, opacity: 0.35, scale: 0.10),
    colorDNA:   ColorDNAConfig(show: true, height: 24, position: .bottom, style: .gradient)
)

let mosaicURL = try await generator.generate(for: video, config: config)

Performance Tips

1. Use HEIF format - Best compression with good quality
2. Start with medium density - Adjust based on video length
3. Disable accurate timestamps for faster processing when precision isn't critical
4. Use batch processing for multiple videos to leverage concurrency
5. Consider screen size - Match output width to your display for optimal viewing
6. Monitor memory usage - Very high densities or large widths can use significant memory

Frame Extraction Strategy

MosaicKit uses an intelligent frame extraction strategy:

• Skips first 5% and last 5% of video (avoid fade in/out)
• First third: 20% of frames (opening scenes)
• Middle third: 60% of frames (main content)
• Last third: 20% of frames (ending)
• Hardware accelerated using VideoToolbox
• Concurrent extraction based on available CPU cores

Error Handling

do {
    let mosaicURL = try await generator.generate(for: video, config: config)
} catch MosaicError.metalNotSupported {
    print("Metal is not available on this device")
} catch MosaicError.invalidVideo(let message) {
    print("Invalid video: \(message)")
} catch MosaicError.layoutCreationFailed(let error) {
    print("Layout creation failed: \(error)")
} catch MosaicError.saveFailed(let url, let error) {
    print("Failed to save mosaic to \(url): \(error)")
} catch {
    print("Unexpected error: \(error)")
}

System Requirements for Best Performance

• Apple Silicon (M1/M2/M3) - Optimal performance with unified memory
• 16GB+ RAM - For processing large videos or high densities
• Intel Mac with dedicated GPU - Good performance with AMD/NVIDIA GPUs
• Fast SSD - For quick frame extraction and mosaic saving

Concurrency Management

Batch processing automatically adjusts concurrency based on:

• CPU cores: max(2, processorCount - 1)
• Available memory: max(2, physicalMemory / 4GB)
• Final limit: min(cpu_limit, memory_limit, configured_limit)

// Configure custom concurrency limit
let generator = try MetalMosaicGenerator()
let coordinator = MosaicGeneratorCoordinator(
    mosaicGenerator: generator,
    concurrencyLimit: 8  // Max 8 videos processed simultaneously
)

Troubleshooting

"Metal is not supported"

• Ensure you're running on a Metal-capable device
• Check minimum OS requirements (macOS 15+ / iOS 15+)

Out of memory errors

• Reduce mosaic width
• Lower density setting
• Process videos in smaller batches
• Close other applications

Slow processing

• Check if accurate timestamps are needed (slower but more precise)
• Verify Metal is being used (check device capabilities)
• Consider reducing frame count for long videos
• Use batch processing for multiple videos

Quality issues

• Increase compression quality (0.6-0.8 for HEIF/JPEG)
• Use higher density settings
• Increase output width
• Try PNG format for lossless output

Contributing

Contributions are welcome! Please feel free to submit a Pull Request.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• Built with Swift 6's modern concurrency features
• Uses Metal for GPU-accelerated processing
• VideoToolbox for hardware-accelerated frame extraction
• swift-log for structured logging
• DominantColors for color analysis

Support

For issues, questions, or feature requests, please open an issue on GitHub.

Package Metadata

Repository: fdenis75/mosaickit

Default branch: main

README: README.md