DenoiseStream removes background noise from a live microphone in real time. A mic-side client streams 20 ms audio chunks to a server; the server suppresses noise and streams the cleaned audio back — low enough latency to monitor while you speak, with optional before/after recording.

The denoising backend is pluggable. A dependency-free spectral-gate runs on any CPU, and an optional DeepFilterNet backend plugs in for learned, GPU-accelerated suppression — without changing the client, protocol, or endpoints.

Why client + server

Heavy denoising wants a capable (or GPU) machine, while the microphone lives on a laptop. DenoiseStream splits the two and connects them over a small, explicit binary protocol, so the same client can drive a local CPU server for development and a remote GPU server in production.

        ┌──────────────┐    20 ms float32 chunks     ┌─────────────────────┐
 mic ─▶ │    client    │ ──────────────────────────▶ │       server        │
        │  capture &   │      WebSocket  ·  HTTP      │  denoise (CPU/GPU)   │
 out ◀─ │   playback   │ ◀────────────────────────── │  spectral-gate  or  │
        └──────────────┘      cleaned chunks back     │   DeepFilterNet     │
                                                      └─────────────────────┘

Features

Capability	Detail
Pluggable backends	numpy spectral-gate (no GPU) or DeepFilterNet (GPU), selected with --enhancer
Low-latency streaming	full-duplex WebSocket, 20 ms (320-sample) frames at 16 kHz
Batch enhancement	one-shot POST /api/enhance: raw PCM in, denoised WAV out
Real DSP	decision-directed (Ephraim–Malah) Wiener filter on a streaming overlap-add STFT
Bring your own server	client talks to any server that implements the documented protocol
Light core	depends only on numpy; the server and learned backend are optional extras

Quickstart

Offline demo (no microphone)

pip install -e ".[dev]"
python scripts/demo.py

The demo synthesizes a noisy clip, runs the spectral-gate enhancer offline, writes samples/before.wav and samples/after.wav, and reports the measured reduction:

backend         : spectral-gate
clip            : 4.0s @ 16000 Hz
noise floor     : 0.0598 -> 0.0226 rms
noise reduction : 8.4 dB

Live streaming (with a microphone)

pip install -e ".[server,client]"

# terminal 1 — start the server (GPU-free spectral-gate backend)
denoisestream-server --enhancer spectral-gate

# terminal 2 — stream mic -> denoise -> speakers
denoisestream-client --url http://127.0.0.1:8765 stream --play

# capture before/after WAVs while streaming (Ctrl+C to stop)
denoisestream-client stream --record --before before.wav --after after.wav

Batch mode

Record a fixed segment, enhance it in one request, then save or play it back:

denoisestream-client --url http://127.0.0.1:8765 batch --seconds 3 --play

DeepFilterNet backend

pip install -e ".[server,deepfilternet]"   # adds torch (large)
denoisestream-server --enhancer deepfilternet

DeepFilterNet is a learned model that handles non-stationary noise — other voices, keyboard, traffic — that a stationary spectral gate cannot. The client, protocol, and endpoints are unchanged.

Use your own server

The bundled server is a reference default, not a requirement. The client and server are fully decoupled: the client speaks a small, documented protocol (docs/protocol.md) and will talk to any server that implements it — for example a GPU box already running DeepFilterNet, RNNoise, or a custom model.

Install only the client and point it at your host:

pip install -e ".[client]"     # installs the mic client only — no aiohttp, no server
denoisestream-client --url http://YOUR-SERVER:PORT stream --play

A compatible server only needs to honor the contract:

Endpoint	Purpose
GET /health	report sample_rate_hz and chunk_samples
GET /ws/stream	JSON handshake, then binary float32 chunk in, denoised chunk out
POST /api/enhance	raw float32 PCM in, denoised WAV out (batch)

How it works

Signal processing — src/denoisestream/dsp.py

The default backend is a numpy-only speech enhancer on a streaming overlap-add STFT (50% overlap, Hann window):

1. Frame the audio into overlapping windows.
2. Estimate the noise spectrum with minima-controlled recursive averaging and a speech-presence guard, so sustained speech is not absorbed into the noise floor.
3. Apply a gain mask from a decision-directed (Ephraim–Malah) Wiener filter: bins dominated by noise are attenuated toward a floor; bins above it pass through.
4. Reconstruct with weighted overlap-add, dividing out the window energy so a unity mask reproduces the input exactly. Algorithmic latency is one hop (20 ms).

Transport — src/denoisestream/protocol.py, server/app.py

• Streaming — a WebSocket duplex. A JSON handshake negotiates the chunk size, then the client sends raw float32 chunks and the server returns one denoised chunk for each.
• Batch — POST /api/enhance with raw float32 PCM returns a denoised WAV.
• Raw-TCP framing — a self-contained length-prefixed format (magic header, uint32 length, payload, 0 for end-of-stream) for the low-level transport.

See docs/architecture.md and docs/protocol.md for the full design.

Concurrency — src/denoisestream/client/stream.py

The client runs three cooperating threads — a mic-input callback, a network worker, and a speaker-output callback — bridged by two queues. In live mode it drops the oldest chunk under backpressure to stay low-latency; in record mode it keeps every chunk so the saved file is complete.

Backends

--enhancer	Dependencies	Removes	Notes
spectral-gate	numpy	stationary noise (hiss, fans, hum)	default, GPU-free
deepfilternet	torch + model	non-stationary noise too	learned, GPU-friendly
passthrough	numpy	nothing (identity)	baseline / transport testing

Add your own by implementing the Enhancer interface (src/denoisestream/enhancers/base.py) and registering it in the factory.

Project layout

src/denoisestream/
  config.py        client/server contract (sample rate, chunk size, routes)
  protocol.py      length-prefixed binary framing
  audio_io.py      WAV encode/decode
  dsp.py           streaming STFT + spectral-gate Wiener filter
  enhancers/       pluggable backends and factory
  server/          aiohttp app: WebSocket stream, HTTP batch, health
  client/          streaming and batch clients, CLI
tests/             protocol, DSP, enhancers, end-to-end server
scripts/demo.py    offline before/after demo
docs/              architecture and protocol specs

Development

pip install -e ".[dev]"
pytest

The suite exercises the binary framing, WAV I/O, the STFT/Wiener DSP, and an end-to-end round-trip against the live aiohttp server.

Limitations

• The spectral gate targets stationary noise; use the DeepFilterNet backend for non-stationary sources.
• The DeepFilterNet streaming path runs per chunk; a frame-synchronous integration would lower its latency further.
• The server has no authentication or TLS — run it on a trusted network or behind a reverse proxy.

License

Released under the MIT License.