Most of the voice AI you hear online still lives on somebody else’s server. You type a line, it ships off to a cloud API, and a few hundred milliseconds later a voice comes back along with a usage bill. Zyphra is betting that doesn’t have to be the default anymore. On November 9 the San Francisco lab released ZONOS3-Mini, an open text-to-speech model small enough to run in a browser tab, and it put the weights on Hugging Face under the Apache 2.0 license.
That license matters. Apache 2.0 is about as permissive as open weights get: you can build it into a commercial app, a game mod or a streaming tool without asking permission or paying royalties. Plenty of recent “open” releases come with regional bans, revenue caps or research-only clauses attached. This one doesn’t.
The headline number is the size. ZONOS3-Mini is a sparse mixture-of-experts model with 800 million parameters in total, but only about 100 million of them do work on any given step. According to the model card, it routes each token through 2 of 64 experts across an 11-layer backbone. For comparison, its predecessor ZONOS2 weighed in at 8 billion parameters, and Zyphra admits in its release post that the bulk, plus a high-framerate audio codec, limited where people could actually deploy it.
What ZONOS3-Mini actually does
At its core it is a voice-cloning TTS engine. Feed it a reference clip of up to 60 seconds and it will speak new text in that voice, and the SGLang-Omni serving path doesn’t even need a transcript of the reference audio. You can also skip cloning entirely and pick from 27 bundled default voices, or “design” a voice by nudging sliders for age, gender presentation, accent and vocal texture.
The control layer is where it gets interesting for anyone making games, videos or narration. Zyphra’s demo page walks through the same sentence delivered with warm relief, restrained anger and nervous apprehension, the same voice aged from roughly 25 to 75, and a single line where only a few words get whispered. That last trick is word-level steering: the model watches its own attention alignment while it streams audio so it knows exactly which words it’s saying, then applies the delivery change only there. There is also a separate edit model for fixing a phrase after the fact without regenerating the whole take.
Zyphra is careful in its own wording here, and that’s worth repeating. The release post calls the emotion and age settings “targets” and notes that the controls “do not guarantee a specific emotion.” Translation: it’s a steering wheel, not an autopilot, and you will still be doing retakes.
Language coverage is broad on paper. The model card lists 43 supported languages, from English, Spanish and Japanese to Hindi, Tagalog and Georgian. The fine print is that automatic text normalization (turning “4:30” or “$20” into speakable words) only covers nine of them: English, Mandarin, Japanese, Korean, German, Spanish, French, Italian and Portuguese. Everything else passes through as written, so numbers and abbreviations in other languages may need cleanup before you hit generate.
Runs on phones, laptops and your own GPU
The deployment story is the strongest part of the release. Zyphra shipped several runtimes on day one, all collected in its GitHub repository (the runtime code itself is MIT-licensed):
- A WebGPU build that runs entirely in a supported browser, with no install and no server. The first visit downloads the model and the browser caches it.
- A C++/ggml server and command-line tool that targets CPU, CUDA, Metal or Vulkan, with no Python required at runtime.
- An MLX server for Apple Silicon Macs and a CUDA server for NVIDIA cards.
- Day-zero support in SGLang-Omni, which exposes an OpenAI-compatible
/v1/audio/speechendpoint with streaming.
That Vulkan path is a quiet win for Linux users and anyone not running NVIDIA hardware, since Vulkan is the cross-vendor option that works on most modern GPUs. And the OpenAI-compatible endpoint means tools already wired up for cloud TTS can, in theory, be pointed at a local server instead with minimal changes.
Zyphra’s latency claims are specific: under 570 milliseconds to first audio in the browser on a Pixel 9 phone, and under 150 ms on an M3 MacBook Air. On a single NVIDIA H100 data-center GPU, the model card reports a median 80 ms to first audio with eight requests running at once, and peak throughput of 457 seconds of audio generated per second. Those are big numbers, but they are Zyphra’s own measurements. MarkTechPost flags the benchmark ranking as self-reported, Neural Brew attributes the speed figures to the company, and I haven’t seen independent testing yet.
The same goes for quality. Zyphra built a new benchmark for this launch, ZTTS2-Eval, covering read speech in 35 languages and real-world recordings in 20, and its capability index ranks ZONOS3-Mini first among open models, ahead of Fish Audio’s S2 Pro and MOSS TTS v1.5. A company releasing its own scoreboard alongside its own model is not unusual in AI, but it does mean you should treat the ranking as a claim to verify. Zyphra’s own chart is honest in one respect: on the quality axis, OmniVoice actually scores a bit higher. ZONOS3-Mini’s pitch is that it’s on the efficiency frontier, offering near-top quality at a fraction of the latency, not that it sounds best outright.
Under the hood, the speedup comes from a few places. Zyphra trained its own audio codec, ZAC-1, which runs at 15 frames per second instead of the 86 Hz codec ZONOS2 used, so the model has far fewer audio tokens to predict. The backbone borrows ideas from recent language models, mixing Gated DeltaNet layers with global attention in a 3-to-1 ratio, and the whole thing was trained on 112 NVIDIA H100s in two and a half days. A single generation can run up to 10 minutes of audio at 44.1 kHz.
Early interest is real but still modest: the Hugging Face page showed about 4,170 downloads and 112 likes when I checked on Wednesday morning, less than two days after launch.
Cody’s take
I’ve been waiting for local TTS to get here. When I was a kid wearing out my PS2, voiced dialogue felt like a luxury, and now a 100-million-active-parameter model promises expressive, cloneable voices on a phone with no cloud bill. For indie devs who need placeholder dialogue, YouTubers who want narration without a subscription, or streamers building local overlays and alert voices, an Apache-licensed engine that runs on your own hardware is a genuinely useful tool.
But I’d be doing you a disservice if I skipped the uncomfortable part. Voice cloning from a short clip, with no gatekeeper, is exactly the capability scammers and impersonators want. The model card I read doesn’t describe built-in watermarking or consent checks, which means responsibility lands on whoever builds with it. If you ship ZONOS3-Mini in an app, build in consent for cloned voices and make it clear when a voice is synthetic. And please don’t clone anyone who hasn’t said yes.
The other caveat is simply time. Every number in this story comes from Zyphra, and the real test is what happens when people run it on mid-range laptops, older phones and non-NVIDIA cards over the next few weeks. If the browser demo holds up on everyday hardware, this could be one of the more practical open AI releases of the fall. If you want to judge for yourself, the Lite browser demo is the fastest way to hear it, as long as your browser supports WebGPU.
Featured image: Image: Zyphra.
Author
