ryanfrancesconi/spfk-image-analysis

Features

• Built-in Classification — Classify still images using Vision's built-in classifier (VNClassifyImageRequest), covering a broad taxonomy of everyday subjects (1303 identifiers as of this writing).
• Custom ML Models — Supply your own Core ML image-classification model via VNCoreMLRequest.
• Video Classification — Classify a video's visual content by sampling frames (via spfk-video) and aggregating results, closing a real gap for silent/muted video that produces no keywords from audio-only classification.
• Async/Await API — Async interface with Swift concurrency, matching the shape of spfk-sound-analysis's SoundClassification.
• Confidence Filtering — Configurable minimum confidence threshold to filter low-quality results.

Architecture

┌──────────────────────────────────────────────────┐
│                ImageClassification                │
│      (Public API — async analyze)                 │
├──────────────────────────────────────────────────┤
│  VNClassifyImageRequest   or   VNCoreMLRequest    │
│     (built-in classifier)   (custom Core ML model)│
├──────────────────────────────────────────────────┤
│         VNImageRequestHandler                     │
│  (Single-shot request against one still image)    │
└──────────────────────────────────────────────────┘

Unlike SoundAnalysis, which delivers classifications progressively across overlapping audio windows (requiring an aggregator to keep the best result per identifier), VNClassifyImageRequest performs one classification pass over a still image and returns one complete, already-deduplicated result set. There is no windowed-results aggregator for still images in this package for that reason. Video is different: Vision has no native video-classification API, so analyzeVideo samples still frames (via spfk-video's VideoFrameExtractor), classifies each with the same engine as analyze(url:), and aggregates across frames — keeping the highest confidence seen per identifier, the video-frame analog of SoundAnalysis's windowed aggregation.

• ImageClassification — Entry point enum with static analyze/analyzeVideo methods. Handles request creation, the request handler, confidence filtering, and (for video) frame sampling and cross-frame aggregation.

Usage

Classify an Image

import SPFKImageAnalysis

let url = URL(fileURLWithPath: "/path/to/photo.jpg")
let results = try await ImageClassification.analyze(url: url)

for classification in results ?? [] {
    print("\(classification.identifier): \(classification.confidence)")
}

Classify with a Custom Confidence Threshold

let results = try await ImageClassification.analyze(
    url: url,
    minimumConfidence: 0.8
)

Classify with a Custom Core ML Model

let model = try MLModel(contentsOf: modelURL)
let results = try await ImageClassification.analyze(
    using: model,
    url: url
)

Query Known Classification Identifiers

let identifiers = try ImageClassification.knownClassifications()
print("Available identifiers: \(identifiers.count)")
// abacus, accordion, acorn, acrobat, adult, ...

Classify a Video's Visual Content

let videoURL = URL(fileURLWithPath: "/path/to/video.mp4")
let results = try await ImageClassification.analyzeVideo(
    url: videoURL,
    sampling: .fixedInterval(step: 2.0)
)

for classification in results ?? [] {
    print("\(classification.identifier): \(classification.confidence)")
}

sampling uses a SamplingStrategy enum rather than a bare interval so a future, smarter sampling mode (e.g. perceptual-difference-based frame selection) can be added later without an API-breaking change. .fixedInterval is the only strategy for now. There is no maximum-frame-count safety cap: a long video sampled at a small step interval means one real classification call per sampled frame, so a caller exposing step as a UI control should make that cost visible (or bound it) itself.

Dependencies

| Package | Description | |---------|--------------| | spfk-base | Core utilities and extensions | | spfk-video | Video frame extraction (VideoFrameExtractor), used by analyzeVideo | | spfk-testing | Test infrastructure (test target only) |

Requirements

• Platforms: macOS 13+, iOS 16+
• Swift: 6.2+

About

Spongefork is the personal software projects of musician and developer Ryan Francesconi. Dedicated to creative sound manipulation, his first application, Spongefork, was released in 1999 for macOS 8. From 2026, Spongefork returns as his software container for more musical experimentation. In addition to software releases, open source components can be found on his GitHub page.

Package Metadata

Repository: ryanfrancesconi/spfk-image-analysis

Default branch: main

README: README.md