sucrette/chatterbox-s3gen-coreai

Chatterbox S3Gen flow and vocoder for Core AI

The speech-token-to-waveform half of Resemble AI's Chatterbox (MIT), converted to Apple's Core AI (.aimodel, macOS 27) and cut into fixed streaming windows: the S3Gen flow (speech tokens to mel, 2-step MeanFlow) and the HiFT vocoder (mel to 24 kHz audio), split into an F0 graph and a body graph. Weights from s3gen_meanflow.safetensors in ResembleAI/chatterbox-nano; Nano and Turbo share them, so this repository serves both chatterbox-nano-t3-coreai and chatterbox-turbo-t3-coreai. Converted for Asa (AsaTTS.SpeechSynthesizer). The voice prompt is an input, so one graph serves every voice.

Files, inputs and outputs

file	inputs	outputs
flowp_48_fp16.aimodel	ptok int32 [1, 250] prompt tokens, pfeat fp16 [1, 500, 80] prompt mel, spks fp16 [1, 80] (spk_embed_affine(normalize(x-vector))), tokens int32 [1, 48], z fp16 [1, 80, 596] noise	mel fp16 [1, 80, 96] (the new tokens' frames only)
vocf0_58.aimodel	mel float32 [1, 80, 58]	f0 [1, 58]
vocbody_58.aimodel	mel float32 [1, 80, 58], src float32 [1, 1, 27840]	wav float32 [1, 27840] (24 kHz)

The window sizes come from the streaming parameters: 15 committed tokens, 25 tokens of left context and 8 of lookahead (48 tokens into the flow); 30 frames plus 16 frames of overlap and 12 of context (58 frames into the vocoder). The harmonic source excitation (SineGen and SourceModuleHnNSF) is not in the graphs: the host makes it from f0 with a running phase accumulator so the phase stays continuous across windows, using the constants of Chatterbox's m_source (Asa writes them to engine.json with tools/build_assets.py).

Checked on

Apple silicon Mac, macOS 27, Core AI GPU, coreai-core 1.0.0b3, after saving: flow mel mean absolute error 0.0020 against PyTorch eager (fp16 graph, fp32 reference); the host source plus vocoder body against Chatterbox's own hg.inference: max error 1.27e-6, mean 6.8e-8 (mean |signal| 0.019). End to end through Asa's swift test and asa-tts.

How it was made

uv run tools/export_flow.py        # flowp_48_fp16.aimodel
uv run tools/export_vocoder.py     # vocf0_58.aimodel, vocbody_58.aimodel

from Asa's tools/. Pins: chatterbox-tts @ 5de7a54aa4e5, coreai-core 1.0.0b3, coreai-torch 0.4.3, torch 2.13.0, torchaudio 2.11.0.

License

MIT, as Chatterbox (LICENSE, Copyright (c) 2025 Resemble AI). S3Gen's flow and the HiFT vocoder derive from CosyVoice (Apache 2.0).