[dependabot]

Bumps transformers from 5.5.4 to 5.21.0.

Release notes

Sourced from transformers's releases.

Release: v5.21.0

Release v5.21.0

New Model additions

MiniCPM-OCR

MiniCPM-OCR is a document-understanding model from OpenBMB's MiniCPM-V family. It keeps the MiniCPM-V 4.6 architecture, which pairs a SigLIP vision encoder and a window-attention merger with a Qwen3.5 language model. It is specialized for reading text-dense images such as scanned pages, screenshots, receipts and tables.

Links: Documentation | Paper

• Feat test (#49920) by @​guarin in #49920

Flint

Flint is Recursive's mixture-of-experts language model. Compared to a standard SwiGLU MoE transformer such as Qwen3-MoE, it uses hybrid attention (sliding-window attention with rotary embeddings over the current token and the previous 512 tokens in most layers, and full attention without positional encoding in the remaining layers). It also has learned attention sinks, as in GPT-OSS, with per-head RMS-normalized queries and keys. Each sparse layer routes a token to 6 of 256 experts by sigmoid score and adds an ungated shared expert, and the first two layers are dense.

Links: Documentation

• Add Flint model support. (#49824) by @​lovishmadaan in #49824

UltraVBert

UltraVBert extends ModernVBert by pairing the UltraBERT text encoder with a SigLIP vision encoder for visual document understanding and retrieval. Its image processor splits each page into 512-pixel tiles plus a global view. The connector merges each 4 x 4 block of vision patches into one token, so a tile becomes 64 image tokens, and RMS-normalizes them to the scale of UltraBERT's token embeddings. The max_image_tiles setting (27 by default) caps the number of tiles per page.

Links: Documentation

• Add UltraVBERT (#49685) by @​athrael-soju in #49685

Gradium Phonon TTS

Gradium Phonon TTS is Gradium's family of on-device text-to-speech models, trained with the Pocket TTS recipe. A causal transformer reads a voice prompt and the text, then emits one continuous 32-dimensional audio latent per step at 12.5 Hz through a one-step flow head, and an end-of-speech head decides when the utterance ends. The latents are decoded to 24 kHz audio by a Pocket TTS audio tokenizer. Unlike Pocket TTS, voice conditioning uses 512-dimensional latents from a separate, unreleased speaker codec, so voices come as precomputed latents and cannot be cloned from a recording.

Links: Documentation

• Add GradiumPhononTTS (#49791) by @​eustlb in #49791

Pocket TTS

Pocket TTS is Kyutai's 100M-parameter text-to-speech model, built as a Continuous Audio Language Model that runs faster than real time on CPU. A causal transformer emits one continuous audio latent per frame (12.5 Hz) through a one-step flow head, and an end-of-speech head decides when to stop. A Mimi-based audio tokenizer encodes voice prompts and decodes the generated latents to 24 kHz audio. Two flow heads are released, Lagrangian self-distillation (default) and drifting, and voices can be cloned from a recording, passed as precomputed latents, or loaded from cached KV states.

Links: Documentation | [Paper](https://huggingface.co/papers/2509.06926 (the Continuous Audio Language Models paper, arXiv 2509.06926). It is the only paper the documentation links, and it describes the architecture family Pocket TTS belongs to. The docs do not cite a separate Pocket TTS technical report.) | [Blog Post](NONE. The documentation links no official blog post. It points only to the original code repository at https://github.com/kyutai-labs/pocket-tts.)

• add PocketTTS (#49589) by @​eustlb in #49589

Pocket TTS Audio Tokenizer

The Pocket TTS audio tokenizer is the codec used by Pocket TTS. It has Mimi's architecture (SEANet convolutions plus a small causal transformer on each side), but its residual vector quantizer is replaced by a continuous 32-dimensional bottleneck, the VAE-GAN from Continuous Audio Language Models. It converts 24 kHz audio into latents at 12.5 Hz and decodes them back to audio. It serves as the audio_tokenizer of PocketTTSProcessor, encoding voice prompts and decoding generated latents, and its encoder and decoder can also be loaded on their own.

Links: Documentation | Paper

• add PocketTTSAudioTokenizer (#49588) by @​eustlb in #49588

... (truncated)

Commits

• dded62b Release: v5.21
• 6c5f8f4 [Fix] Fix CSM codebook embedding tying (#49197)
• b41ca9c [FA] Fix FA with static cache (#49998)
• 2e12b32 [tests_fetcher] fix missing .as_posix() in preprocessor_patterns path keys (#...
• fadef79 fix(ci): skip bot suggestion if a maintainer already posted run-slow: (#49992)
• d0103c9 [loading] Allow Renaming and Converter to work on the sume subkeys (#49928)
• 7725fb8 [Fix] Fix NUMA and cpuset detection in CB (#49980)
• 2711ad2 [Refactor] Make CB utils depend on no CB import (#49968)
• c9b377c Composite transparency in shared RGB conversion and image processors (#49079)
• b3d3a61 fix: prevent empty frame sampling for short videos (#49908)
• Additional commits viewable in compare view

Note
Automatic rebases have been disabled on this pull request as it has been open for over 30 days.