Skip to main content
Live Translation connects two human participants speaking different languages. Send each person’s separate microphone audio and deliver the translated speech to the other person. Choose any two different languages from the available roster below. For customer integrations, start with the calling platform guide and a funded Live project key. Bring your own Twilio or Plivo account and run the media bridge on your server. Use the call-history API if you want to keep transcripts, transfer events or recordings.

Settings and customization

Live Translation connects two human participants. To customize an AI voice agent instead, use Agent settings.

Recordings and transcripts in Console

Open Live Translation → Call history in the selected live project to review saved audio and source/translated transcripts. Enable recording deliberately and wait for saving to finish after ending the conversation. Access, incomplete uploads, channel layout, and export behavior are documented in Call history and recordings.

Available languages

Arabic (ar), Bengali (bn), English (en), French (fr), German (de), Gujarati (gu), Hindi (hi), Italian (it), Japanese (ja), Kannada (kn), Korean (ko), Mandarin Chinese (zh), Portuguese (pt), Punjabi (pa), Russian (ru), Spanish (es), Telugu (te) and Urdu (ur). Both people can use any two different languages from this list; English is optional. The WebSocket hello.languages list provides accepted voices and a default voice for each language. Use a voice belonging to that participant’s language. hello.synthetic_audio_tested_pairs records completed synthetic audio checks: Hindi ↔ English and Punjabi ↔ English. Other pairs are available to try; their audio and translation quality still need review. General speech recognition or voice catalogs do not establish additional Live Translation language support.

Try it in the Console

Open Live Translation. The initial tester requires a verified saaslabs.co login and an active live project in your workspace. Workspace credits apply. Use headphones, select each person’s language, and switch the speaker when testing with one microphone. The tester displays both the source transcript and translation.

Price

See PyAI pricing for the Live Translation rate. One connected call-minute includes both directions, recognition, translation and generated speech. Carrier charges are separate. Silence counts while connected. Rejected WebSocket upgrades are free. Duration is accumulated as fractional minutes; the standard per-second billing pulse applies to the invoice aggregate. The interpretation request does not add a separate Speak charge to your project.

Connect from your server

Your application owns its calling provider. Open the Console’s Integration tab to compare the shared PyAI code with Twilio and Plivo transport excerpts. Provider setup lives in your server code; the Translation API, language configuration and transcript events use the same contract for both. See the calling integration guide for the complete media, playback and transfer flow. Create a live key in API Keys with the interpret:session permission. Connect to:
Server clients send Authorization: Bearer <live-key>. Browser clients must use an ephemeral, origin-locked token created by your server:
Connect using the returned url and the subprotocols pyai.v1 and pyai-key.<token>. Keep tokens in memory. Never send a secret server key to a browser or place credentials in a URL. Tokens default to 60 seconds and permit one concurrent connection. Keys with a monthly spending limit use direct server connections; they cannot delegate browser tokens.

Stream a conversation

Wait for hello, inspect supported_languages, then send:
Use pa and pa1 for Punjabi. A leg’s voice speaks to that participant. Adopt the acknowledged epochs from session.started. Its server-assigned request_id identifies the authenticated connection for usage reconciliation; retain it with your session record. Send 16 kHz mono signed little-endian PCM16 as base64, without a WAV header, in frames up to 200 ms. A 20 ms frame contains 640 bytes. Sequence numbers start at zero for each leg’s epoch and increase by one:
Source and translated captions arrive in transcript.committed and translation.committed. Preserve their independent offsets and join them using source_context_id. Route each audio.chunk only to its target_leg, respecting its epoch, output generation and increasing audio sequence. Translation can begin before the source speaker finishes; input.commit flushes the remaining source context.

Playback and echo checks

Feed dry participant audio into the API. Generated translation must never be mixed back into the input. The service gates a listener’s microphone while translated speech plays, then waits for terminal playback receipts and a 250 ms tail before resuming input. This per-listener half-duplex policy intentionally prevents speech played to one person from being translated back. audio.completed means generation finished. Report played only after the actual playback device finishes. For interrupted or failed playback, report the actual partial sample count. Clear queued sound on output.cancelled and send a matching playback.cleared acknowledgement after a physical flush. Delivery receipts are required to safely resume gated input. input.speech_started cancels speech addressed to that listener and advances their output generation once. Track this transition: output.cancelled still identifies the cancelled utterance’s older generation. Clear acknowledgements must carry the current leg, epoch and output generation. Do not replay an interruption control after an uncertain acknowledgement. Send session.end and close the socket when the call ends. Sessions are bounded to 30 minutes, with 60 seconds of client-input inactivity and a 15-second start deadline. There is no automatic replay after reconnect.

Calling-platform integration

Use the calling-platform guide for Twilio, Plivo, Telnyx and Vonage. Each requires a media bridge that keeps the two human call legs separate and returns actual playback receipts. Your application owns both call legs, the customer journey, recording consent, transcript storage and internal transfers. leg.replace fences the old participant and updates interpreter state; your calling platform performs the actual transfer. Preserve already emitted source/translation records on errors. The initial service permits four concurrent sessions. Ten concurrent calls, language-pair quality and telephone latency need separate qualification. No production latency SLA is implied by the tester.