# Tone Media Streams

Source: https://docs.usetone.ai/tone-media-streams

> The WebSocket protocol Tone speaks to your own voice stack.

*The tier-2 integration contract: you buy a +91 number from Tone, Tone handles
TRAI compliance and the carrier, and every call's audio is handed to YOUR
server over a WebSocket. You run STT, LLM and TTS; Tone runs everything else.
Protocol first, then setup, then the local test loop.*

## What you get

- Every call on a BYO number (outbound via `POST /v1/calls`, inbound from the
  PSTN) opens one WebSocket from Tone to your endpoint, carrying the caller's
  audio and accepting yours.
- The protocol is **Twilio Media Streams-shaped** — same seven events, same
  camelCase fields, same base64 payloads — so code and framework serializers
  written for Twilio (Pipecat's `TwilioFrameSerializer` in particular) work
  with at most a config change. Pipecat also ships an `ExotelFrameSerializer`;
  prefer the Twilio one against Tone.
- The carrier behind the number is Tone's concern, not yours. Tone may change
  carriers; this protocol will not change under you.

## Wire protocol

All frames are JSON text messages. Every frame Tone sends carries a
`sequenceNumber` (string, monotonic). Audio is base64 in `media.payload`, no
headers, no containers.

### Tone → you

```jsonc
{"event":"connected","protocol":"Call","version":"1.0.0","sequenceNumber":"1"}

{"event":"start","sequenceNumber":"2","streamSid":"MZ…",
 "start":{"streamSid":"MZ…","callSid":"<Tone call id (uuid)>","tracks":["inbound"],
   "mediaFormat":{"encoding":"audio/l16","sampleRate":8000,"channels":1},
   "customParameters":{"callId":"<uuid>","direction":"outbound","from":"+91…","to":"+91…", "…":"…"}}}

{"event":"media","sequenceNumber":"3","streamSid":"MZ…",
 "media":{"track":"inbound","chunk":"1","timestamp":"20","payload":"<base64>"}}

{"event":"dtmf","streamSid":"MZ…","dtmf":{"track":"inbound_track","digit":"5"}}

{"event":"mark","streamSid":"MZ…","mark":{"name":"turn-3"}}   // playback reached your mark

{"event":"stop","streamSid":"MZ…","stop":{"callSid":"<uuid>","reason":"completed"}}
```

`customParameters` is three layers, later wins: static pairs you configured on
the number, the `variables` map passed to `POST /v1/calls` (so your dialer can
hand your bot per-call context — order id, customer name), then the identity
facts `callId`, `direction`, `from`, `to`. All values are strings.

`mediaFormat.encoding` is `audio/l16` (16-bit signed little-endian PCM, mono)
or `audio/x-mulaw` (G.711 μ-law, always 8000 Hz) — whichever you configured.

### You → Tone

```jsonc
{"event":"media","streamSid":"MZ…","media":{"payload":"<base64 audio in the same format>"}}

{"event":"mark","streamSid":"MZ…","mark":{"name":"turn-3"}}   // echoed back when played

{"event":"clear","streamSid":"MZ…"}   // barge-in: drop audio Tone has queued but not yet played
```

Three rules that decide whether your bot feels right on a phone:

1. **Send audio in small chunks** (20–100 ms). `clear` cannot recall audio the
   carrier is already playing; what you have queued ahead is what a barge-in
   costs you.
2. **Use `mark`.** It is the only reliable "the caller has now heard it"
   signal — TTS lands far faster than it plays, and speaking again at "TTS
   finished" talks over yourself.
3. **Hang up by closing the socket.** Tone closes the carrier leg, bills the
   call, and sends your webhook.

### Audio

| Setting | Values | Notes |
|---|---|---|
| `format` | `linear16` (default) \| `mulaw8k` | μ-law is the Twilio-compat mode |
| `sampleRate` | `8000` (default) \| `16000` | linear16 only; 16k is better for ASR |

Tone resamples between the carrier's rate and yours.

## Setting up a number

1. Declare your sender classification once, in the **dashboard** under
   Compliance → profile. It is the `purpose` the pre-dial gate uses for every
   BYO dial — there is no agent to read one from. (`PATCH /v1/compliance/profile`
   backs that screen, but it takes a dashboard session, not an API key — a key
   gets `403`.)
2. Put the credential your endpoint expects into Tone's secret store, in the
   **dashboard** under Secrets — the same store tool credentials use. Tone sends
   it as the `Authorization` header value, verbatim. (`POST /v1/secrets` is
   session-only too: minting credentials is deliberately closed to API keys.)
3. Switch the number:

```http
PATCH /v1/numbers/{id}
{
  "routingMode": "byo_ws",
  "mediaEndpoint": {
    "type": "static",
    "url": "wss://bot.example.com/media",
    "format": "linear16",
    "sampleRate": 8000,
    "authSecretId": "<secret id>",
    "customParameters": { "team": "support" }
  }
}
```

`type: "webhook"` instead POSTs `{callId, from, to, direction,
customParameters}` to an `https://` URL at call time and expects
`{"url":"wss://…"}` back — per-call routing for servers that cannot carry
session state in a static URL. Static URLs get `?callId=` appended.

Endpoints must be public `wss://` (or `https://` for the webhook). Hosts that
resolve to private address space are refused, both when you save the config
and again when Tone connects.

### Placing a call

```http
POST /v1/calls
{ "numberId": "<byo number>", "toE164": "+91…", "variables": { "order_id": "A-1042" } }
```

No `agentId` — the number's mode forbids one. The compliance gate runs exactly
as it does for a Tone-agent call (calling window, your DNC list, consent), and
a blocked dial is a 403 with the audit rows already written.

## SIP trunking (for hosted platforms)

If your voice AI is a hosted platform — Retell, Vapi, ElevenLabs Agents,
LiveKit, OpenAI Realtime — it consumes a SIP trunk, not a WebSocket. Tone
provisions one per number on its SIP-capable carrier:

```http
POST /v1/numbers/{id}/sip
{ "sipUri": "sip:abc123@sip.retellai.com;transport=tls", "transport": "tls" }
```

The response carries, **once**, the termination credentials your platform
uses to dial out through Tone:

```json
{ "termination": { "domain": "…sip.vobiz.ai", "username": "tone_…", "password": "…", "realm": "…" } }
```

- **Inbound**: calls to the number are sent to `sipUri` (your platform's
  ingress). Tone adds the platform-standard headers; your platform sees the
  caller as `From`.
- **Outbound**: your platform dials the termination domain with digest auth
  and the Tone number as caller id.
- `DELETE /v1/numbers/{id}/sip` tears the trunk pair down (and is how you
  rotate credentials).
- SIP trunking is available on numbers bought on Tone's SIP-capable carrier;
  the purchase flow tells you which.

**Compliance on SIP calls is post-hoc, not pre-dial.** Your platform dials the
carrier directly, so Tone cannot stand in front of the call the way it does
for WebSocket numbers. Pre-flight recipients with `POST /v1/compliance/check`
(or the batch form) before dialling; Tone records every SIP call from the
carrier's CDR — priced at the same ₹2.50/min, with MOS/jitter/packet-loss in
the call record — and fires `call.completed`. The evidence pack for a number
includes those calls.

## Billing

A BYO minute is **₹2.50** — telephony plus platform (gate, CDR, recording,
webhooks, the bridge); the agent component is yours to run. Plus the number's
monthly rental. A call that never connects, or that Tone refused, bills nothing.

## Webhooks

Subscribe to `call.completed`. You get it for every finished call — answered,
no answer, busy, failed — with the disposition in the payload, so your dialer
can react to the ones that did not connect.

## Testing without a phone

Point the fake caller at your server:

```bash
cd voice && pnpm test:byo -- wss://bot.example.com/media --wav hello.wav --format linear16 --rate 8000
```

It performs the handshake, streams the WAV at real time as a caller would,
prints every frame you send, echoes your `mark`s, and writes what a caller
would have heard to `byo-reply.wav`. With no `--wav` it streams silence —
enough to prove the handshake and your `mark`/`clear` plumbing.

For a full carrier loop locally, run the fake carrier (`skrull/`) with
`BYO_ALLOW_PRIVATE_TARGETS=true` on the voice service and a BYO number on a
`tone_test_` key: the dial, the bridge, billing and the webhook all execute
against your endpoint on localhost with no money and no phone ringing.

## Refusals you may see in the call log

| `endReason` | Meaning |
|---|---|
| `refused_no_endpoint` | Number is BYO but has no media endpoint configured |
| `refused_endpoint_unreachable` | Connect (or webhook fetch) failed or timed out |
| `refused_endpoint_forbidden` | Endpoint resolved to a private address or was not `wss://` |
| `refused_media_format` | Carrier rate and your configured rate cannot be bridged |
| `max_duration` | The call hit the platform ceiling (60 min) |

Each is a `failed` disposition, reported — never a silent drop.