VibeVoice

The API

Two ways in: a WebSocket that returns punctuated text while somebody is still talking, and an HTTP pipeline for files. Every example below was taken from the running server rather than written to describe it.

What can I build on the VibeVoice API?

Two things. The WebSocket at wss://vibevoice.net/api/stream takes 16 kHz mono PCM — that is what the desktop client uses. The HTTP endpoints transcribe files you already have. Both authenticate with an API key from your account.

There is no local or on-premise deployment: audio is transcribed on our servers, so an integration needs a network connection. If your constraint is that audio must never leave the machine, a self-hosted transcriber is the right tool and this is not one — we would rather say so here than after you have built against us.

Authentication

HTTP requests carry the key in an X-API-Key header, or the same key as Authorization: Bearer. Either works on every HTTP endpoint here. The WebSocket does neither: it takes the key in the first text frame after the connection opens, and there is no query-parameter form of it — connecting with ?api_key= fails the handshake without explanation, which is the single most common way a first integration goes wrong.

X-API-Key: 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef      # HTTP
Authorization: Bearer 0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef   # HTTP, same key
{"api_key": "0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"}          # first WebSocket frame

Keys are per-device rather than per-account: treat a key as a password, as it is not scoped. Unverified accounts return 403 until the link in the confirmation email is clicked.

What comes back

Text arrives in pieces while the speaker is still going. Each piece is a JSON frame carrying the transcript in text and its position in idx, counting from 1. The pieces never overlap: append them in order and you have the transcript.

{"text": "Wo kann man ", "idx": 1}
{"text": "am besten kitesurfen?", "idx": 2}
{"text": "", "is_final": true, "idx": 2}    # end of stream, idx = pieces sent

Reach for idx if your client can deliver a piece more than once — writing into two places, retrying, or resuming an insertion. Keep the indices you have applied and skip the ones you have seen. The number on the final frame says how many there were, so a short transcript can be told apart from a lost frame.

Added 2026-09-02. Clients written before it are unaffected: an unknown JSON key is ignored, and text and is_final keep their meaning.

Each piece also carries ack_ms: the point in your audio, in milliseconds from the first byte you sent, before which every word has now been delivered. It only ever grows. It is what to resend from if a connection is lost.

If the connection drops

A phone switching from Wi-Fi to mobile data loses its socket mid-sentence. Ask for a resume token when you connect, and the next connection carries on the same stream: the text from before the break is delivered, numbering continues, and the server tells you how much of your audio it actually received.

{"api_key": "...", "resumable": true}                         # connect
{"status": "authenticated", ..., "resume": "<token>"}

{"api_key": "...", "resumable": true,
 "resume": "<token>", "last_idx": 3}                            # reconnect
{"status": "authenticated", ..., "resumed": true, "received_ms": 9024}
{"text": "...", "idx": 4, "ack_ms": 9000}                       # what you missed

Send last_idx, the last piece you received, and resend your audio from received_ms onwards — that is what was in flight when the line went down. The session is held for 30 seconds — also after you sent END_STREAM, so a drop right after stopping does not lose the last piece: that reply also says "replay_only": true, you receive what you missed and the final frame, and you send nothing more. If it is gone, the reply says "resumed": false and the stream starts fresh: resend from the last ack_ms and drop the words that repeat the end of the last piece you have — the offset is safe, not exact.

Added 2026-10-03. Optional: a client that does not send resumable sees no change at all.

A working client, in four languages

# Install: pip install websockets
import asyncio
import json
import websockets

API_KEY = "your_api_key_here"
URL = "wss://vibevoice.net/api/stream"


async def stream_audio(pcm_chunks):
    """pcm_chunks yields raw int16 PCM bytes, 16 kHz mono."""
    async with websockets.connect(URL) as ws:
        # 1. Authenticate. The key goes in a text frame, not the URL.
        await ws.send(json.dumps({"api_key": API_KEY}))
        ack = json.loads(await ws.recv())
        if ack.get("status") != "authenticated":
            raise RuntimeError(ack)

        # 2. Read transcript pieces while audio is still going out.
        async def reader():
            async for message in ws:
                frame = json.loads(message)
                if frame.get("is_final"):
                    break
                if frame.get("text"):
                    print(frame["text"], end="", flush=True)

        reading = asyncio.create_task(reader())

        # 3. Send audio as binary frames, then close the input explicitly.
        for chunk in pcm_chunks:
            await ws.send(chunk)
        await ws.send("END_STREAM")

        await reading


asyncio.run(stream_audio(my_pcm_source()))

Endpoints

WSSwss://vibevoice.net/api/stream

Bidirectional streaming for live dictation.

Text returns while the speaker is still going · /stream is the frozen legacy alias and behaves identically

POST/api/transcribe

Synchronous transcription of a single WAV file. Nothing transcodes first, so other formats belong on /api/jobs/submit.

Optional prompt field biases vocabulary · up to 30MB, the same for every plan

POST/api/jobs/submit

The queued pipeline, and the one to use for anything long or not already a WAV. Returns a job id immediately; poll GET /api/jobs/{id} for status and the finished transcript. Optional word-level timestamps (Pro and Ultra) and speaker diarization (Ultra).

Accepts audio and video · include_timestamps, word_level_timestamps, enable_diarization · up to 100MB free, 1GB on Pro, 10GB on Ultra

POST/oauth/device/code

Device Code Grant, for a client with no browser of its own — the desktop app and CLI tools use it. Pair with /oauth/device/token to exchange an approved code for a key.

Same flow the desktop client runs on first launch

The complete parameter and response reference is in the Redoc explorer (opens in a new tab), generated from openapi.json (opens in a new tab).

30min / month

Included every month on every account, Free plan and up — no credit card needed to test the API.

A glass housing with an orb feeding threads into one side and structured output emerging from the other.

What the API does not do

  • It does not run on your infrastructure. There is no self-hosted build, no container and no on-premise licence — audio is transcribed on our servers.
  • It does not work offline, and it has no local fallback. A dropped connection can be resumed for 30 seconds if the client asked for it; after that, or without it, the stream is gone.
  • POST /api/transcribe takes WAV only. Other formats belong on /api/jobs/submit, which transcodes; sending an mp3 to the synchronous endpoint currently returns a 500 rather than a helpful error.
  • There is no custom vocabulary or terminology training. You can bias a single request with the prompt field, which is a hint and not a dictionary.
  • Speaker diarization runs on the job pipeline only, not on the live stream, and requires Ultra.
  • There is no webhook or push callback yet. Job completion is polled.

Common questions

Which audio format does the stream expect?+

Raw int16 PCM, 16 kHz, mono, sent as binary WebSocket frames — no container, no header, no WAV wrapper. Float32 samples are the usual mistake: they are accepted without error and transcribe as noise, because the bytes are read as int16 regardless.

How is the WebSocket authenticated?+

With a JSON text frame, {"api_key": "..."}, sent as the first message after the connection opens. Wait for {"status": "authenticated"} before sending audio. There is no query-parameter or header form of this — a URL with ?api_key= fails.

What is the difference between /stream and /api/stream?+

Nothing functional. /stream is the original path and remains supported because desktop clients going back to version 0.1.0 still use it and cannot be force-updated. New integrations should use /api/stream.

Is there an official SDK?+

No. The protocol is a WebSocket and a multipart POST, which every language handles in its standard library or one common package, so a wrapper would mostly be maintenance. The four request examples on this page are the reference implementation.

What are the rate limits?+

100 requests an hour on POST /api/transcribe and the same on job submission, per key. Separately, your plan caps monthly transcription minutes and maximum file size — those are the limits most integrations meet first.

Can I transcribe a file that is hours long?+

Yes, through /api/jobs/submit rather than the synchronous endpoint. It queues the work and returns an id to poll, so the request does not sit open for the duration. File-size caps still apply by plan.

Last updated .