The SDK provides the two-party translation room and Twilio media transport.
Your server remains responsible for owning both calls, validating Twilio’s
signed WebSocket upgrade, storing transcripts and call events, and allowlisting
internal contacts. Bounded synthetic browser-call tests passed for Hindi ↔ English
and Punjabi ↔ English, including both directions, echo blocking, saved transcripts
and replacement-agent transfers. Validate your own call setup and human speech
quality before production.
interpret:session key on the
bridge server.
The repository includes a bounded
twilio-live-translation test host
for dedicated customer, agent and replacement-agent browser calls. Build the
current recipe and SDK from the same repository revision. The host validates
webhook signatures, issues single-use participant links, persists transcripts
to private storage and applies carrier time limits. Tests used real Twilio calls
and the deployed PyAI API; they did not certify PSTN routing, physical speaker
audibility or JustCall integration.
Recordings and transcripts in Console
Open Live Translation in Console, select your team’s live project, and start a new conversation or call. Confirm that everyone agrees to recording before starting. For a carrier call, the Console explicitly shows whether the connected host supports recording. Choose End conversation or End call and wait for saving to finish. In Call history, select the call to play its recording, download the audio, or read and download the original and translated transcripts. Other signed-in members of the same project can open the same history. Invitation links and upload tokens do not grant access to stored recordings. For a per-person recording, use Listening view to select Person A or Person B. The left channel contains that person’s original speech; the right contains the incoming translation sent to that person. The agent view includes the replacement agent after a transfer, within the same call record. Browser recordings contain microphone audio and translations rendered by the browser. Carrier recordings contain admitted source audio and translations sent to the call. Cancelled playback can differ from what reached the listener; recordings do not prove that someone heard every sent sample. Older recordings may have a single combined view. Read original speech beside its translation using Combined or By language. Search or filter Call history by language pair to find earlier calls. Both orientations of a pair appear together when history is grouped by pair. Calls made before recording was enabled have no recoverable audio. An incomplete upload is labelled Incomplete, and any successfully saved transcript remains available. Recordings stay private; playback and download require current project access.Test a conversation in Console
Open Live Translation, select a live project and choose Phone call test. The internal tester uses its configured calling backend automatically; there is no provider selector. Hosted test access requires a verified saaslabs.co account. Choose the customer and agent languages, create a test room, and share their separate invitations. Use separate devices and headphones. Select Transfer to invite a replacement agent, then End call and wait for cleanup before opening Call history for available recordings and transcripts. Your own Twilio application is configured in your code. Open the Console’s Integration tab for the shared PyAI room code and Twilio transport excerpt, then follow the complete webhook and media setup below. The internal tester does not configure your Twilio account or certify your application’s call routing.Connect the customer call
Return this TwiML from your authenticated answer webhook. The WebSocket URL belongs to your bridge, not the PyAI API:leg=agent. Pair both calls
in trusted server state. The SDK’s TwilioStreamAdmission issues a short-lived,
single-use path and validates X-Twilio-Signature; use that binding together
with start.accountSid, start.callSid and start.streamSid before creating
the media transport.
Twilio supports one bidirectional stream per call and delivers its inbound
track. <Connect><Stream> blocks following TwiML while connected. A
<Start><Stream> or ordinary conference audio tap cannot supply the required
listener-specific bidirectional output. See
Media Streams and
Stream parameters.
Authenticate and convert the audio
ValidateX-Twilio-Signature on webhooks and the WebSocket upgrade using
Twilio’s supported validation library and the correct public request URL behind
your proxy. Bind the expected account, start.callSid, start.streamSid and
room/leg to the authenticated call record. Stream URLs do not support query
parameters; use nested Parameter elements for pairing metadata.
Twilio media uses base64 headerless μ-law at 8 kHz. Decode and statefully
resample to PyAI’s 16 kHz PCM16 input. Convert output back to μ-law at 8 kHz,
with bounded, paced queues. The existing @pyai/twilio codec and streaming
resampler helpers are used by TwilioInterpretationTransport; pair it with
InterpretationRoom and a durable record callback.
On the intended listener’s stream, send:
Map marks without inventing playback
After the final media for an utterance, send an identity-bound mark:clear. Those marks
must never become status: "played". See
Twilio’s mark and clear semantics.
Twilio has no distinct documented clear acknowledgement. The SDK sends a fresh
sentinel mark after clear on the same ordered socket and waits for that barrier
before reporting playback.cleared. Carrier transfer tests returned the barrier
and resumed the replacement participant. Unit tests separately reject invalidated
old marks; the carrier tests did not naturally produce a late stale mark.
Handle transfers and hangup
Perform transfers with Twilio call control and the replacement sequence in the shared guide. Clear both playback queues and fence the previous agent’s call/stream before admitting the new agent. DTMF on bidirectional Media Streams is inbound-only; keep outbound DTMF in your call-control design. When either call ends, apply your room’s waiting/hangup policy, sendsession.end and close the PyAI socket promptly. Do not replay uncertain
speech after reconnect. Validate both translation directions, late marks after
clear, echo blocking and a real transfer before enabling customer traffic.