Skip to content
Docs
TTS
OmniTTS

OmniTTS

vLLM‑Omni is an inference service system that supports multiple TTS models, including Qwen3‑TTS, CosyVoice and VoxCPM2. For more models, refer to https://docs.vllm.ai/projects/vllm-omni/en/latest/serving/speech_api/. It exposes a unified TTS API service compliant with the OpenAI specification, and supports uploading, deleting and viewing cloned voices.

1. Install Dependencies 

uv venv --python 3.12 --seed
source .venv/bin/activate
#uv pip install vllm==0.22.0 --torch-backend=auto
uv pip install vllm==0.27.0 --extra-index-url https://wheels.vllm.ai/0.27.0/cu129 --extra-index-url https://download.pytorch.org/whl/cu129 --index-strategy unsafe-best-match #Select the corresponding version from https://vllm.ai/releases based on your CUDA version

2. Install vllm‑omni 

git clone https://github.com/vllm-project/vllm-omni.git
cd vllm-omni
uv pip install -e .

3. Start the API Service

3.1 Qwen3

Requires 12 GB of VRAM

vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-CustomVoice --omni --trust-remote-code --port 8091  #Pre‑defined voices: vivian, aiden, etc.
vllm serve Qwen/Qwen3-TTS-12Hz-1.7B-Base --omni --trust-remote-code --port 8091  #Cloned voices need to be uploaded manually

3.2 VoxCPM

Requires 13 GB of VRAM

uv pip install voxcpm>=2.0
vllm serve openbmb/VoxCPM2 --omni --trust-remote-code --port 8091  #Default preset voice; custom cloned voices can be uploaded

3.3 Fish Speech

Requires 18 GB of VRAM

uv pip install fish_speech
vllm serve fishaudio/s2-pro --omni --trust-remote-code --port 8091

3.4 IndexTTS

Requires 17 GB of VRAM, latency ~4 seconds

vllm serve IndexTeam/IndexTTS-2 --omni --trust-remote-code --port 8091

3.5 CosyVoice

Requires 14 GB of VRAM

uv pip install s3tokenizer
vllm serve FunAudioLLM/Fun-CosyVoice3-0.5B-2512 --omni --trust-remote-code --port 8091

4. Voice Cloning

  1. Start the LiveTalking service
python app.py --transport webrtc --model wav2lip --avatar_id wav2lip256_avatar1 --tts omnitts --TTS_SERVER http://127.0.0.1:8091
  1. Open the webpage: http://<serverip>:8010/tts/index.html

Fill in your TTS server address, e.g. http://<tts-serverip>:8091, then click Test Connection. You will see the list of available voices.

Upload your audio file in the voice‑cloning upload box. Once cloning succeeds, the new voice will appear in the voice list. This cloned voice is available for all models.

  1. Open the webpage: http://<serverip>:8010

Enter the name of your newly‑cloned voice (for example, voxpcm_edge) into the reference‑audio input box, then click Connect Video.

Legacy Guide