Voices, languages, and controls

Choose a configuration in this order: product → language → accent → voice → delivery → controls. Product compatibility is not implied by a voice name or by a successful request. Read the catalog, then verify the effective configuration returned by the runtime.

Download the dated JSON reference pack. It is designed for code generation and local validation; it is not a substitute for GET /v1/voices, account entitlement checks, or the Omni configured acknowledgement.

Pick a product before a language

Product	How language is selected	Runtime evidence
Hear streaming STT	`language=auto`, omitted, or an explicit supported code	Transcript events; no language-ID field is returned
Speak TTS	The selected voice	Catalog `available_on`, response headers, and successful synthesis
Omni	Agent or `configure.language`, plus a compatible voice	`configured.language_active`, `language_fallback`, canonical `voice_id`, and `voice_tier`
Cast	Query Cast capabilities	The capabilities response, not the Speak or Omni catalog

Hear supports automatic detection only inside its published language set. Omni accepts en, es, fr, de, and hi, while its served set is currently en, es, fr, and hi; German is an English fallback. Speak catalog groups are broader, so a Speak voice must never be presented as proof that the same language works end to end in Omni.

English is a language; accent is a voice choice

Selecting en-IN does not transform an American voice into Indian English. Use an accent-specific voice ID, then listen to the exact configuration before you ship it.

Customer choice	Omni language	Example voice	Accent
English — US	`en`	`stock_dorit_en_us`	American
English — UK	`en`	`stock_amelia_en_gb`	British
English — Australia	`en`	`stock_elias_en_au`	Australian
English — India	`en`	`stock_arjun_en_in`	Indian

Evaluate names, numbers, dates, phone numbers, interruptions, and short conversational replies. Evaluate Hindi and Hinglish separately from Indian English; a catalog identifier is not a quality result.

Copy a complete Omni configuration

This is the body of a tagged 0x03 configure frame, not a REST request or a complete browser client:

{
  "type": "configure",
  "language": "en",
  "voice_id": "stock_elias_en_au",
  "voice_instruct": "Speak calmly and clearly. Leave space between steps.",
  "persona": "You are a concise support assistant. Ask one question at a time.",
  "greeting": "Hi, thanks for calling. How can I help?"
}

voice_instruct is a delivery request, not an acoustic guarantee or an emotion enum. It is limited to 200 characters and works only on compatible tiers. Speak uses its own conditional emotion field; Cast exposes a separate capabilities response with emotion and intensity. Do not copy controls between these products. For Omni, check configured.voice_instruct_supported === true when delivery direction is a requirement. Standard Elise and Gwen do not support this control; English Natural does. An absent field means the server has not reported the capability. A later voice_capabilities event can revoke support after synthesis fallback. Keep prompt guidance for wording separate from this acoustic capability. After sending configure, require the session to confirm any capabilities your application needs:

function onConfigured(event, requested) {
  if (event.event !== "configured") return;
  if (event.language_fallback || event.language_active !== requested.language) {
    throw new Error("Required language was not served; do not start the call.");
  }
  console.info({ requested, served: event });
}

Conversation timing and silence

Omni has a session endpointing_ms control and an Agent/session idle_check_in preset (auto, patient, off). The deprecated barge_sensitivity and ack_mode Agent fields are stored for compatibility and do not tune the runtime. Do not map a responsiveness slider to an invented scale or present a saved value as evidence of application. See the controls contract and effective acknowledgement and machine-readable matrix.

Delivery is an observed behavior

Surface	Transport	What the client receives	Common integration error
Hear streaming	WebSocket	JSON transcript events	Appending `speech_final` and `final` as two rows
Speak streaming	HTTP response body	Audio bytes	Waiting for the full body before beginning playback
Omni	WebSocket	Tagged audio, text, and control frames	Playing a control frame as audio or retaining audio after `flush`

Speak stream: true means the request accepts streamed delivery. If the server returns x-pyai-stream: buffered, it completed successfully but audio was buffered before delivery. Record that mode when measuring time to first audible audio.

Launch check

Query GET /v1/voices and retain the canonical ID, aliases, available_on, and delivery metadata you selected.
Mint a short-lived, origin-bound browser session token on your backend; do not expose a PyAI API key in the browser.
Configure Omni, wait for configured, and compare requested versus served language, voice, and tier.
Start microphone capture and playback only after required configuration is active. On flush, stop local playback and clear queued audio immediately.
Test microphone denial, cancellation while connecting, duplicate start/stop, language fallback, failed configuration, interruptions, tool timeout, and duplicate tool delivery.

For a customer-owned write such as booking, retain the conversation ID, tool invocation call_id, and callback-delivery identity separately. Authorize the customer tenant in the customer backend and return a durable prior result when the same invocation is delivered again. See Omni tools.

Language support

Accepted, served, and fallback language behavior.

Omni protocol

Tagged frames and the configured acknowledgement.

Browser voice agent

A complete microphone-to-playback client.

Speak

Voice catalog, delivery modes, and audio formats.

Get started

Agents

Speech To Text (Hear)

Text To Speech (Speak)

Clone

Cast

Dub

Calls & intelligence

Frameworks

Integrations

Use cases

Reference

Voices, languages, and controls

Pick a product before a language

English is a language; accent is a voice choice

Copy a complete Omni configuration

Conversation timing and silence

Delivery is an observed behavior

Launch check

Language support

Omni protocol

Browser voice agent

Speak

​Pick a product before a language

​English is a language; accent is a voice choice

​Copy a complete Omni configuration

​Conversation timing and silence

​Delivery is an observed behavior

​Launch check

Language support

Omni protocol

Browser voice agent

Speak

Pick a product before a language

English is a language; accent is a voice choice

Copy a complete Omni configuration

Conversation timing and silence

Delivery is an observed behavior

Launch check