Add S3Gen v3 weights (batch 1/1)

This commit is contained in:
systemd
2026-06-11 12:43:33 +00:00
parent fecca7cade
commit e5729f4973
4 changed files with 55 additions and 5 deletions

3
.gitattributes vendored
View File

@ -53,3 +53,6 @@ Cangjie5_TC.json filter=lfs diff=lfs merge=lfs -text
t3_mtl23ls_v2.safetensors filter=lfs diff=lfs merge=lfs -text
t3_mtl23ls_v3.safetensors filter=lfs diff=lfs merge=lfs -text
s3gen_v3.safetensors filter=lfs diff=lfs merge=lfs -text
s3gen_v3.pt filter=lfs diff=lfs merge=lfs -text

View File

@ -55,6 +55,29 @@ library_name: chatterbox
<img width="100" alt="resemble-logo-horizontal" src="https://github.com/user-attachments/assets/35cf756b-3506-4943-9c72-c05ddfa4e525" />
</div>
## Latest Release: Chatterbox Multilingual V3
**Chatterbox Multilingual V3** is the latest general-purpose multilingual TTS model in the Chatterbox family. It keeps the same 0.5B model size while improving speaker similarity, reducing hallucinations, and producing more natural, conversational speech across languages.
V3 is designed for broad language coverage like V2, but with stronger stability and more expressive generation. It is the recommended multilingual model for users who want one voice cloning model that works across many languages. Try it in the [Chatterbox Multilingual TTS V3 Space](https://huggingface.co/spaces/ResembleAI/Chatterbox-Multilingual-TTS-V3).
Alongside V3, we are releasing the **Single Language Pack**: dedicated finetunes for priority languages where tighter quality control, stronger language-specific behavior, and more specialized speech generation are valuable.
- **Broad Multilingual Coverage:** Designed as the main general-purpose multilingual Chatterbox model, supporting wide language coverage similar to V2.
- **Single Language Pack:** Dedicated single-language models provide stronger specialization and quality control where language- and regional-dialect-specific performance matters most.
- **More Consistent Speaker Similarity:** Improves voice identity and accent preservation across languages, making cross-language voice cloning more stable and reliable.
- **Reduced Hallucination:** V3 is optimized to reduce unwanted continuation, repetition, and off-prompt speech, especially in cases where earlier multilingual models were less stable.
## Model Zoo
Choose the right Chatterbox model for your application.
| Model | Size | Languages | Key Features | Best For | Demo |
| --- | --- | --- | --- | --- | --- |
| **Chatterbox Multilingual V3** | **500M** | **23+** | Improved speaker similarity, reduced hallucinations, more natural multilingual speech | Global applications, localization, cross-language voice cloning | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox-Multilingual-TTS-V3) |
| **Single Language Pack** | 500M each | 6 dedicated finetunes | Language- and region-specific quality control | Priority languages and dialect-sensitive applications | [Demos](#single-language-pack) |
| Chatterbox | 500M | English | CFG and exaggeration tuning | General zero-shot TTS with creative controls | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox) |
**09/04 🔥 Introducing Chatterbox Multilingual in 23 Languages!**
We're excited to introduce **Chatterbox** and **Chatterbox Multilingual**, [Resemble AI's](https://resemble.ai) production-grade open source TTS models. Chatterbox Multilingual supports **Arabic, Danish, German, Greek, English, Spanish, Finnish, French, Hebrew, Hindi, Italian, Japanese, Korean, Malay, Dutch, Norwegian, Polish, Portuguese, Russian, Swedish, Swahili, Turkish, Chinese** out of the box. Licensed under MIT, Chatterbox has been benchmarked against leading closed-source systems like ElevenLabs, and is consistently preferred in side-by-side evaluations.
@ -64,7 +87,8 @@ Whether you're working on memes, videos, games, or AI agents, Chatterbox brings
If you like the model but need to scale or tune it for higher accuracy, check out our competitively priced TTS service (<a href="https://resemble.ai">link</a>). It delivers reliable performance with ultra-low latency of sub 200ms—ideal for production use in agents, applications, or interactive media.
# Key Details
- Multilingual, zero-shot TTS supporting 23 languages
- Chatterbox Multilingual V3, a 0.5B general-purpose multilingual TTS model supporting 23 languages
- Dedicated single-language finetunes for Chinese, LatAm Spanish, Brazilian Portuguese, Spain Spanish, Portugal Portuguese, and Hindi
- SoTA zeroshot English TTS
- 0.5B Llama backbone
- Unique exaggeration/intensity control
@ -116,18 +140,35 @@ ta.save("test-2.wav", wav, model.sr)
import torchaudio as ta
from chatterbox.mtl_tts import ChatterboxMultilingualTTS
multilingual_model = ChatterboxMultilingualTTS.from_pretrained(device="cuda")
multilingual_model = ChatterboxMultilingualTTS.from_pretrained(
device="cuda",
t3_model="v3",
)
# To use the legacy V2 multilingual checkpoint, omit t3_model or pass t3_model="v2".
french_text = "Bonjour, comment ça va? Ceci est le modèle de synthèse vocale multilingue Chatterbox, il prend en charge 23 langues."
wav_french = multilingual_model.generate(french_text, language_id="fr")
ta.save("test-french.wav", wav_french, model.sr)
ta.save("test-french.wav", wav_french, multilingual_model.sr)
chinese_text = "你好,今天天气真不错,希望你有一个愉快的周末。"
wav_chinese = multilingual_model.generate(chinese_text, language_id="zh")
ta.save("test-chinese.wav", wav_chinese, model.sr)
ta.save("test-chinese.wav", wav_chinese, multilingual_model.sr)
```
See `example_tts.py` for more examples.
## Single Language Pack
The Single Language Pack provides dedicated finetunes for priority languages and regional variants. Use these when you want stronger language-specific behavior, tighter quality control, or dialect-aware generation beyond the general multilingual model.
| Language | Model Card | Demo Space |
| --- | --- | --- |
| Chinese | [ResembleAI/Chatterbox-Multilingual-zh-cmn](https://huggingface.co/ResembleAI/Chatterbox-Multilingual-zh-cmn) | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox-Multilingual-TTS-zh-cmn) |
| LatAm Spanish | [ResembleAI/Chatterbox-Multilingual-es-mx-latam](https://huggingface.co/ResembleAI/Chatterbox-Multilingual-es-mx-latam) | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox-Multilingual-TTS-es-mx-latam) |
| Brazilian Portuguese | [ResembleAI/Chatterbox-Multilingual-pt-br](https://huggingface.co/ResembleAI/Chatterbox-Multilingual-pt-br) | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox-Multilingual-TTS-pt-br) |
| Spain Spanish | [ResembleAI/Chatterbox-Multilingual-es-es](https://huggingface.co/ResembleAI/Chatterbox-Multilingual-es-es) | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox-Multilingual-TTS-es-es) |
| Portugal Portuguese | [ResembleAI/Chatterbox-Multilingual-pt-pt](https://huggingface.co/ResembleAI/Chatterbox-Multilingual-pt-pt) | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox-Multilingual-TTS-pt-pt) |
| Hindi | [ResembleAI/Chatterbox-Multilingual-hi](https://huggingface.co/ResembleAI/Chatterbox-Multilingual-hi) | [Demo](https://huggingface.co/spaces/ResembleAI/Chatterbox-Multilingual-TTS-hi) |
# Acknowledgements
- [Cosyvoice](https://github.com/FunAudioLLM/CosyVoice)

BIN
s3gen_v3.pt (Stored with Git LFS) Normal file

Binary file not shown.

BIN
s3gen_v3.safetensors (Stored with Git LFS) Normal file

Binary file not shown.