soundanalysis

Safe Rust bindings for Apple's SoundAnalysis framework on macOS.

Status: v0.6.4 keeps full public macOS SoundAnalysis.framework coverage against the Xcode 26.5 SDK and expands the Tier-1 async layer with both AsyncAudioFileAnalyzer and AsyncAudioStreamAnalyzer. See COVERAGE.md for complete API mapping.

Quick start

use soundanalysis::prelude::*;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let results = classify_file("target/utterance.aiff")?;
    for result in &results {
        if let Some(top) = result.top() {
            println!(
                "[{:.2}s+{:.2}s] {} ({:.2})",
                result.time_start,
                result.time_duration,
                top.identifier,
                top.confidence
            );
        }
    }

    println!("known classes: {}", known_classifications()?.len());
    Ok(())
}

Low-level framework surface

use soundanalysis::prelude::*;

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut request = ClassifySoundRequest::version1()?;
    request.set_overlap_factor(0.25)?;

    if let TimeDurationConstraint::Range(range) = request.window_duration_constraint()? {
        request.set_window_duration(range.start_seconds.max(0.5))?;
    }

    let mut analyzer = AudioFileAnalyzer::new("target/utterance.aiff")?;
    analyzer.add_request(
        &request,
        ResultsObserverFns::new(|_request, result| {
            if let Some(top) = result.top() {
                println!("{} {:.2}", top.identifier, top.confidence);
            }
        }),
    )?;
    analyzer.analyze()?;
    Ok(())
}

For real-time use, AudioStreamAnalyzer accepts interleaved or planar PCM buffers (f32, f64, i16, i32) and start_live_classification() keeps the high-level microphone convenience API. If you prefer Apple naming, the crate now re-exports SNClassifySoundRequest, SNClassifierIdentifier, SNAudioFileAnalyzer, SNAudioStreamAnalyzer, SNClassificationResult, SNClassification, SNTimeDurationConstraint, SNTimeRange, SNRequest, SNResult, SNResultsObserving, SNErrorDomain(), and SNErrorCode alongside the ergonomic Rust names. The raw SDK error identifiers are also available as error_domain() and ErrorCode while SAError remains the higher-level operational error type.

Examples

• cargo run --example 01_classify_file
• cargo run --example 02_known_classes
• cargo run --all-features --example 03_smoke_surface
• cargo run --example 04_apple_aliases
• cargo run --example 05_custom_model_request
• cargo run --features async --example 04_async_classify_file

Async API

The async feature now provides:

• AsyncAudioFileAnalyzer for non-blocking audio file analysis
• AsyncAudioStreamAnalyzer + AudioStreamAnalysisStream for bounded async event streams over SNAudioStreamAnalyzer

AsyncAudioFileAnalyzer looks like this:

use soundanalysis::async_api::AsyncAudioFileAnalyzer;

#[tokio::main]
async fn main() -> Result<(), Box<dyn std::error::Error>> {
    let analyzer = AsyncAudioFileAnalyzer::new("path/to/audio.mp3")?;
    analyzer.analyze().await?;
    println!("Analysis complete");
    Ok(())
}

The async API is executor-agnostic — it works with any async runtime (Tokio, async-std, smol, etc.). For streaming analysis, poll AudioStreamAnalysisStream::next() / try_next() and handle AudioStreamAnalysisEvent::{Result, Error, Complete} values.

Pipeline composition

screencapturekit-rs ──► system audio ──► soundanalysis ──► event timeline
                                                             │
                                                             ▼
                                                    speech ──► transcript
                                                             │
                                                             ▼
                                                    naturallanguage ──► entities
                                                             │
                                                             ▼
                                                    foundation-models
                                                    ("summarise the meeting,
                                                     flag the cough at 3:42")

Pairs naturally with screencapturekit (system audio capture) and speech for full audio-understanding pipelines.

Roadmap

• File-based classification (SNAudioFileAnalyzer)
• Request tuning (overlapFactor, windowDuration, windowDurationConstraint)
• Built-in classifier metadata (knownClassifications, ClassifierIdentifier::Version1)
• Custom Core ML request creation (SNClassifySoundRequest(mlModel:))
• File analyzer request management (add/remove/removeAll, sync + completion-handler analysis, cancel)
• Stream analyzer request management (add/remove/removeAll, raw PCM analyzeAudioBuffer, completeAnalysis)
• Results observing callbacks and ClassificationResult::classification_for_identifier
• High-level microphone convenience (start_live_classification)

License

Licensed under either of Apache-2.0 or MIT at your option.