---
title: ryanfrancesconi/spfk-image-analysis
framework: Swift Package Catalog
role: article
path: packages/ryanfrancesconi/spfk-image-analysis
---

# ryanfrancesconi/spfk-image-analysis

ML image classification for Swift, built on Apple's [Vision](https://developer.apple.com/documentation/vision) framework. The image-domain sibling of [spfk-sound-analysis](https://github.com/ryanfrancesconi/spfk-sound-analysis).

## Features

- **Built-in Classification** — Classify still images using Vision's built-in classifier (`VNClassifyImageRequest`), covering a broad taxonomy of everyday subjects (1303 identifiers as of this writing). - **Custom ML Models** — Supply your own Core ML image-classification model via `VNCoreMLRequest`. - **Video Classification** — Classify a video's visual content by sampling frames (via `spfk-video`) and aggregating results, closing a real gap for silent/muted video that produces no keywords from audio-only classification. - **Async/Await API** — Async interface with Swift concurrency, matching the shape of `spfk-sound-analysis`'s `SoundClassification`. - **Confidence Filtering** — Configurable minimum confidence threshold to filter low-quality results.

## Architecture

``` ┌──────────────────────────────────────────────────┐ │                ImageClassification                │ │      (Public API — async analyze)                 │ ├──────────────────────────────────────────────────┤ │  VNClassifyImageRequest   or   VNCoreMLRequest    │ │     (built-in classifier)   (custom Core ML model)│ ├──────────────────────────────────────────────────┤ │         VNImageRequestHandler                     │ │  (Single-shot request against one still image)    │ └──────────────────────────────────────────────────┘ ```

Unlike `SoundAnalysis`, which delivers classifications progressively across overlapping audio windows (requiring an aggregator to keep the best result per identifier), `VNClassifyImageRequest` performs one classification pass over a still image and returns one complete, already-deduplicated result set. There is no windowed-results aggregator for still images in this package for that reason. Video is different: Vision has no native video-classification API, so `analyzeVideo` samples still frames (via `spfk-video`'s `VideoFrameExtractor`), classifies each with the same engine as `analyze(url:)`, and aggregates across frames — keeping the highest confidence seen per identifier, the video-frame analog of `SoundAnalysis`'s windowed aggregation.

- **`ImageClassification`** — Entry point enum with static `analyze`/`analyzeVideo` methods. Handles request creation, the request handler, confidence filtering, and (for video) frame sampling and cross-frame aggregation.

## Usage

### Classify an Image

```swift import SPFKImageAnalysis

let url = URL(fileURLWithPath: "/path/to/photo.jpg") let results = try await ImageClassification.analyze(url: url)

for classification in results ?? [] {     print("\(classification.identifier): \(classification.confidence)") } ```

### Classify with a Custom Confidence Threshold

```swift let results = try await ImageClassification.analyze(     url: url,     minimumConfidence: 0.8 ) ```

### Classify with a Custom Core ML Model

```swift let model = try MLModel(contentsOf: modelURL) let results = try await ImageClassification.analyze(     using: model,     url: url ) ```

### Query Known Classification Identifiers

```swift let identifiers = try ImageClassification.knownClassifications() print("Available identifiers: \(identifiers.count)") // abacus, accordion, acorn, acrobat, adult, ... ```

### Classify a Video's Visual Content

```swift let videoURL = URL(fileURLWithPath: "/path/to/video.mp4") let results = try await ImageClassification.analyzeVideo(     url: videoURL,     sampling: .fixedInterval(step: 2.0) )

for classification in results ?? [] {     print("\(classification.identifier): \(classification.confidence)") } ```

`sampling` uses a `SamplingStrategy` enum rather than a bare interval so a future, smarter sampling mode (e.g. perceptual-difference-based frame selection) can be added later without an API-breaking change. `.fixedInterval` is the only strategy for now. There is no maximum-frame-count safety cap: a long video sampled at a small step interval means one real classification call per sampled frame, so a caller exposing `step` as a UI control should make that cost visible (or bound it) itself.

## Dependencies

| Package | Description | |---------|--------------| | [spfk-base](https://github.com/ryanfrancesconi/spfk-base) | Core utilities and extensions | | [spfk-video](https://github.com/ryanfrancesconi/spfk-video) | Video frame extraction (`VideoFrameExtractor`), used by `analyzeVideo` | | [spfk-testing](https://github.com/ryanfrancesconi/spfk-testing) | Test infrastructure (test target only) |

## Requirements

- **Platforms:** macOS 13+, iOS 16+ - **Swift:** 6.2+

## About

Spongefork is the personal software projects of musician and developer [Ryan Francesconi](https://spongefork.com). Dedicated to creative sound manipulation, his first application, Spongefork, was released in 1999 for macOS 8. From 2026, Spongefork returns as his software container for more musical experimentation. In addition to [software releases](https://spongefork.com/shadowtag/), open source components can be found on his [GitHub page](https://github.com/ryanfrancesconi).

## Package Metadata

Repository: ryanfrancesconi/spfk-image-analysis

Default branch: main

README: README.md