pocket-tts-onnx (upstream export)

ONNX graphs for Kyutai's Pocket TTS, exported with the export_onnx.py script that ships with VolgaGerm/PocketTTS.cpp, for use with that C++/ONNX Runtime runtime.

Six languages: english_2026-04, french_24l, german_24l, spanish_24l, italian_24l, portuguese_24l.

Why this exists

The widely used KevinAHM/pocket-tts-onnx export predates PocketTTS.cpp's own exporter. The newer exporter changes the graph, not the weights:

• the Mimi decoder's KV cache is a circular buffer of 250 positions instead of 1000, matching the model's actual attention context — 4× less masked-out work per decode call;
• K and V caches are separate tensors, removing a torch.stack copy per step;
• KV caches are stored in fp16 and bound as both input and output, so ONNX Runtime scatters in place instead of copying the cache every step.

Measured on french_24l, same runtime binary, same reference voice, INT8 backbone + FP32 flow and decoder ("mixed" precision), Apple M-series, median of 5 draws over 3 sentences:

	KevinAHM export	this export
RTFx	2.03	2.91
first-chunk latency	723 ms	294 ms
peak RSS	1172 MB	842 MB
Whisper WER (large-v3-turbo, fr)	0.0 %	0.0 %

Same intelligibility, ~1.4× faster, ~330 MB less resident memory.

Contents, per language folder

file	precision	notes
flow_lm_main_int8.onnx	INT8	backbone; only its MatMuls are quantized, as in Kyutai's own RECOMMENDED_CONFIG
flow_lm_flow.onnx	FP32	flow-matching net
mimi_decoder.onnx	FP32	codec decoder
flow_lm_flow_int8.onnx, mimi_decoder_int8.onnx	INT8	compact variants, for A/B only
mimi_encoder.onnx, text_conditioner.onnx	FP32	never quantized upstream
bos_before_voice.npy	—	learned [1,1,1024] vector; the runtime must prepend it to the encoded reference voice before conditioning
bundle.json	—	runtime contract: model_recommended_frames_after_eos, max_token_per_chunk, remove_semicolons
tokenizer.model	—	SentencePiece

flow_lm_main.onnx (FP32, 1.2 GB per language) is not published — it is only needed to validate the INT8 quantization, which was done at export time.

Quantizing INT8 Mimi is not free

The INT8 Mimi decoder is included for comparison, not recommended. Against the FP32 decoder on identical latents it scores ~16.5 dB SNR, and the error is chunk-size dependent (ONNX Runtime derives the activation scale from the chunk), correlated with the signal, and concentrated in 2–5 kHz — audible roughness. Kyutai's own runtime never quantizes Mimi or the flow net.

Reproducing

The upstream script is hardcoded to the 6-layer b6369a24 English checkpoint, whose config no longer ships with pocket_tts (upstream issue #12). Two changes were needed:

1. derive FlowLMMainWrapper.NUM_LAYERS from len(flow_lm.transformer.layers) — the *_24l checkpoints have 24 layers, and a hardcoded 6 silently drops 18 layers' worth of state;
2. select the config by language and let pocket_tts resolve the hf://…@rev weights.

Every export was validated against PyTorch, FP32 and INT8, with the script's own run_validation.

Licence and attribution

Weights © Kyutai, released under CC-BY-4.0; these are derived works (ONNX conversions) and carry the same licence. Source model: https://huggingface.co/kyutai/pocket-tts. Export tooling: VolgaGerm/PocketTTS.cpp (MIT).