> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pyai.com/llms.txt
> Use this file to discover all available pages before exploring further.

# Live Translation API: languages, voices, and calls

> Translate a live conversation between two people, with separate audio, source transcripts, translated captions and spoken output.

Live Translation connects two human participants speaking different languages.
Send each person's separate microphone audio and deliver the translated speech
to the other person. Choose any two different languages from the available roster below.

For customer integrations, start with the [calling platform guide](https://docs.pyai.com/guides/live-translation-telephony.md)
and a funded Live project key. Bring your own Twilio or Plivo account and run
the media bridge on your server. Use the [call-history API](https://docs.pyai.com/guides/live-translation-call-history.md)
if you want to keep transcripts, transfer events or recordings.

## Settings and customization

| Change | Where to configure it |
| - | - |
| Each participant's language and voice | `session.start.legs`; select from the [live language roster](#available-languages) |
| Browser or server authentication | [Connection setup](#connect-from-your-server) |
| Input audio and leg routing | [Conversation frames](#stream-a-conversation) |
| Playback cancellation and echo behavior | [Playback and echo checks](#playback-and-echo-checks) |
| Stored transcripts, recordings, and transfers | [Call history and recordings](https://docs.pyai.com/guides/live-translation-call-history.md) |
| Carrier media and phone transfers | [Calling-platform setup](https://docs.pyai.com/guides/live-translation-telephony.md) |

Live Translation connects two human participants. To customize an AI voice
agent instead, use [Agent settings](https://docs.pyai.com/agents/settings.md).

## Recordings and transcripts in Console

Open **Live Translation → Call history** in the selected live project to review
saved audio and source/translated transcripts. Enable recording deliberately
and wait for saving to finish after ending the conversation. Access, incomplete
uploads, channel layout, and export behavior are documented in
[Call history and recordings](https://docs.pyai.com/guides/live-translation-call-history.md).

## Available languages

Arabic (`ar`), Bengali (`bn`), English (`en`), French (`fr`), German (`de`), Gujarati (`gu`),
Hindi (`hi`), Italian (`it`), Japanese (`ja`), Kannada (`kn`), Korean (`ko`),
Mandarin Chinese (`zh`), Portuguese (`pt`), Punjabi (`pa`), Russian (`ru`),
Spanish (`es`), Telugu (`te`) and Urdu (`ur`). Both people can use any two
different languages from this list; English is optional.

The WebSocket `hello.languages` list provides accepted voices and a default
voice for each language. Use a voice belonging to that participant's language.
`hello.synthetic_audio_tested_pairs` records completed synthetic audio checks:
Hindi ↔ English and Punjabi ↔ English. Other pairs are available to try; their
audio and translation quality still need review. General speech recognition or
voice catalogs do not establish additional Live Translation language support.

## Try it in the Console

Open [Live Translation](https://console.pyai.com/live-translation). The initial
tester requires a verified `saaslabs.co` login and an active live project in your
workspace. Workspace credits apply. Use headphones, select each person’s language,
and switch the speaker when testing with one microphone. The tester displays
both the source transcript and translation.

## Price

See [PyAI pricing](https://pyai.com/pricing#rate-card) for the Live Translation
rate. One connected call-minute includes both directions, recognition,
translation and generated speech. Carrier charges are separate. Silence counts while connected.
Rejected WebSocket upgrades are free. Duration is accumulated as fractional
minutes; the standard per-second billing pulse applies to the invoice aggregate.
The interpretation request does not add a separate Speak charge to your project.

## Connect from your server

Your application owns its calling provider. Open the Console's
[Integration tab](https://console.pyai.com/live-translation#integration) to compare
the shared PyAI code with Twilio and Plivo transport excerpts. Provider setup
lives in your server code; the Translation API, language configuration and
transcript events use the same contract for both. See the
[calling integration guide](https://docs.pyai.com/guides/live-translation-telephony.md) for the complete
media, playback and transfer flow.

Create a live key in [API Keys](https://console.pyai.com/keys) with the
`interpret:session` permission. Connect to:

```text theme={null}
wss://api.pyai.com/v1/audio/interpretations/stream
```

Server clients send `Authorization: Bearer <live-key>`. Browser clients must use
an ephemeral, origin-locked token created by your server:

```bash theme={null}
curl --fail-with-body https://api.pyai.com/v1/audio/interpretations/sessions \
  -H "Authorization: Bearer $PYAI_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"allowed_origins":["https://your-app.example"],"ttl_seconds":60}'
```

Connect using the returned `url` and the subprotocols `pyai.v1` and
`pyai-key.<token>`. Keep tokens in memory. Never send a secret server key to a
browser or place credentials in a URL. Tokens default to 60 seconds and permit
one concurrent connection. Keys with a monthly spending limit use direct server
connections; they cannot delegate browser tokens.

## Stream a conversation

Wait for `hello`, inspect `supported_languages`, then send:

```json theme={null}
{
  "type": "session.start",
  "external_call_id": "your-call-id",
  "sample_rate": 16000,
  "legs": {
    "customer": {"participant_id":"customer-1","language":"hi","voice":"hi1"},
    "agent": {"participant_id":"agent-1","language":"en","voice":"stock_dorit_en_us"}
  }
}
```

Use `pa` and `pa1` for Punjabi. A leg's voice speaks **to** that participant.
Adopt the acknowledged epochs from `session.started`. Its server-assigned
`request_id` identifies the authenticated connection for usage reconciliation;
retain it with your session record. Send 16 kHz mono signed
little-endian PCM16 as base64, without a WAV header, in frames up to 200 ms.
A 20 ms frame contains 640 bytes. Sequence numbers start at zero for each leg's
epoch and increase by one:

```json theme={null}
{"type":"audio.append","leg":"customer","epoch":1,"sequence":0,"audio":"<base64 PCM16>"}
```

Source and translated captions arrive in `transcript.committed` and
`translation.committed`. Preserve their independent offsets and join them using
`source_context_id`. Route each `audio.chunk` only to its `target_leg`, respecting
its epoch, output generation and increasing audio sequence. Translation can
begin before the source speaker finishes; `input.commit` flushes the remaining
source context.

## Playback and echo checks

Feed dry participant audio into the API. Generated translation must never be
mixed back into the input. The service gates a listener's microphone while
translated speech plays, then waits for terminal playback receipts and a 250 ms
tail before resuming input. This per-listener half-duplex policy intentionally
prevents speech played to one person from being translated back.

`audio.completed` means generation finished. Report `played` only after the
actual playback device finishes. For interrupted or failed playback, report the
actual partial sample count. Clear queued sound on `output.cancelled` and send
a matching `playback.cleared` acknowledgement after a physical flush. Delivery
receipts are required to safely resume gated input.

`input.speech_started` cancels speech addressed to that listener and advances
their output generation once. Track this transition: `output.cancelled` still
identifies the cancelled utterance's older generation. Clear acknowledgements
must carry the current leg, epoch and output generation. Do not replay an
interruption control after an uncertain acknowledgement.

Send `session.end` and close the socket when the call ends. Sessions are bounded
to 30 minutes, with 60 seconds of client-input inactivity and a 15-second start
deadline. There is no automatic replay after reconnect.

## Calling-platform integration

Use the [calling-platform guide](https://docs.pyai.com/guides/live-translation-telephony.md) for
Twilio, Plivo, Telnyx and Vonage. Each requires a media bridge that keeps the
two human call legs separate and returns actual playback receipts.

Your application owns both call legs, the customer journey, recording consent,
transcript storage and internal transfers. `leg.replace` fences the old
participant and updates interpreter state; your calling platform performs the
actual transfer. Preserve already emitted source/translation records on errors.

The initial service permits four concurrent sessions. Ten concurrent calls,
language-pair quality and telephone latency need separate qualification. No production latency SLA is implied by the tester.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.