> ## Documentation Index > Fetch the complete documentation index at: https://docs.pyai.com/llms.txt > Use this file to discover all available pages before exploring further. # Update an agent > Partial update: present fields are set, `null` clears a field, absent fields are untouched. Config edits are live on the agent's next call. ## OpenAPI ````yaml https://api.pyai.com/openapi.json post /v1/agents/{id} openapi: 3.1.0 info: title: PyAI API version: 2.9.0 description: >- Telephony-native Voice AI behind one bearer key: - **Hear**, speech-to-text · `POST /v1/audio/transcriptions` (streaming + batch) - **Speak**, text-to-speech · `POST /v1/audio/speech`, `GET /v1/voices` - **Clone**, custom voices from a short clip · `/v1/voice/clones` - **Cast**, auto-directed expressive voiceovers · `/v1/cast` - **Dub**, asynchronous audio and video dubbing · `POST /v1/dub` ([guide](https://docs.pyai.com/guides/dub-overview)) - **Cue configuration frames (reserved)**; Hear streams skip knowledge-base retrieval by default - **Omni**, full-duplex agentic voice (speech-to-speech, grounded in your knowledge bases + tools) · `/v1/omni` - **Knowledge Bases**, hosted grounding for Omni: create bases, add documents (file, URL, or text), crawl a public website, bind to agents or org defaults · `/v1/knowledgebases` - **AMD API**, answering-machine detection: know *who or what* answered a call (human, voicemail, IVR, iPhone/Google screening, dead number) with the reason it decided · `wss …/v1/amd/stream` (Twilio Media Streams drop-in), `POST /v1/amd/config`, `GET /v1/amd/calls/{id}` - **Agents Beta**, the live console feature to create, configure, test, and connect Omni voice agents without code. Beta features and limits may change. ## Authentication Create a key in the [console](https://console.pyai.com) (it is shown once) and send it as a bearer token: ``` Authorization: Bearer pyai_live_... ``` Keys are environment-scoped: `pyai_live_...` (production) and `pyai_test_...` (sandbox). `POST /v1/sandbox/keys` creates an instant, short-lived test key without login or billing. Account signup also creates a sandbox key. Live keys consume prepaid credit; phone verification may unlock promotional credit under graduated-signup rules, but credit is not guaranteed at signup. Keys are self-validating signed tokens: they work on every PyAI surface the instant they are created, no activation or propagation delay. Treat them as opaque strings (up to 512 chars) and never parse their contents. WebSocket endpoints can't use request headers from a browser, so pass the key as a **subprotocol** instead: ``` Sec-WebSocket-Protocol: pyai.v1, pyai-key.pyai_live_... ``` (server-side clients may instead append `?api_key=...` to the URL). Your key is authenticated on the upgrade and never reaches the model. ## Quickstart, Hear (speech-to-text) ``` curl https://api.pyai.com/v1/audio/transcriptions \ -H "Authorization: Bearer $PYAI_API_KEY" \ -F file=@audio.wav -F model=pyai-hear # -> { "text": "..." } ``` ## Quickstart, Speak (text-to-speech) ``` curl https://api.pyai.com/v1/audio/speech \ -H "Authorization: Bearer $PYAI_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model":"pyai-speak","input":"Hello from PyAI.","voice":"voice_abc"}' \ --output speech.wav ``` `voice` is a stock voice id from `GET /v1/voices` (the curated prebuilt catalog with personas and avatars) or a cloned voice id from `/v1/voice/clones`. Omit it to use the platform default voice (`stock_dorit_en_us`). ## Quickstart, Omni (realtime voice agent) Omni is **zero-state, there is nothing to create first.** Open a WebSocket, pass your key as a subprotocol, and send the agent's behavior (voice, persona, knowledge endpoint) in the first `configure` frame: ``` wss://api.pyai.com/v1/omni?session_label=support&format=pcm16&rate=24000 Sec-WebSocket-Protocol: pyai.v1, pyai-key.$PYAI_API_KEY ``` The session is authorized by your key's **organization**; `session_label` is an **optional, opaque** tag (echoed to your own knowledge endpoint for correlation), omit it or use any value. When `session_label` equals a **`/v1/agents` profile id**, the engine loads persona, voice, and **greeting message** from that profile (turn-0 playback). `format` and `rate` are load-bearing on the connect URL (the SDK sets them). Prefix each PCM16 frame with byte `0x01`; prefix control JSON with byte `0x03`. **Optional convenience:** pre-store config via `POST /v1/agents` (including `greeting`, `consent_line`, `recordings_enabled`) and pass its id as `session_label`, or send everything inline in the post-handshake `configure` frame. Not required to connect. For reproducible eval runs, determinism controls (`seed`/`temperature`) ride the Omni session's `configure` frame, which the gateway passes through unchanged, they are honored once the engine supports them; no platform change is required. ## Scopes | Scope | Grants | | --- | --- | | `hear:transcribe` | `POST /v1/audio/transcriptions` | | `hear:stream` | `GET /v1/audio/transcriptions/stream` (WebSocket) | | `hear:configure` | `GET`/`PUT /v1/hear/vocabulary` | | `speak:synthesize` | `POST /v1/audio/speech` (Speak) | | `speak:clone` | `/v1/voice/clones` (Clone) | | `speak:design` | `/v1/voice/design` (Speak) | | `omni:session` | `/v1/omni` and `POST /v1/omni/sessions` (mint a browser session token) | | `omni:read` | `/v1/omni/calls` (Omni post-call records) | | `kb:manage` | `/v1/knowledgebases/*` (hosted knowledge bases for Omni grounding) | | `transcribe:jobs` | `/v1/transcription/jobs` | | `trace:configure` | `/v1/trace/config`, `/v1/trace/rule-packs` (Trace management) | | `trace:read` | `/v1/trace/interactions`, `/violations`, `/findings`, `/exposure` (Trace reads) | | `recap:configure` | `/v1/recap/config` (Recap management) | | `recap:configure` | `/v1/recap/crm-config` (Salesforce field mapping) | | `recap:read` | `/v1/recap/calls` (Recap reads and speaker-role corrections) | | `amd:detect` | `wss …/v1/amd/stream` (AMD realtime detection, Twilio drop-in) | | `amd:configure` | `/v1/amd/config` (AMD operating-point dial + webhook) | | `amd:read` | `/v1/amd/calls` (AMD decision records) | | `telephony:manage` | `/v1/telephony/*` (managed numbers) | `GET /v1/models`, `GET /v1/voices`, and `GET /v1/me` need no specific scope, any active key may call them. Wildcards (`hear:*`, `speak:*`, …, and the global `*`) grant every scope in their family. ## Canonical endpoints One row per product surface, endpoint, auth, required scope, and lifecycle status. **live** = generally available; **beta** = available now with features or limits that may change; **unavailable** = reserved in the contract but not active on the serving route. | Product | Endpoint | Auth | Scope | Status | | --- | --- | --- | --- | --- | | Identity | `GET /v1/me` | Bearer | _any active key_ | live | | Models | `GET /v1/models` | Bearer | _any active key_ | live | | Voices | `GET /v1/voices`, `GET /v1/voices/{id}` | Bearer | _any active key_ | live | | Hear (batch) | `POST /v1/audio/transcriptions` | Bearer | `hear:transcribe` | live | | Hear (vocabulary settings) | `GET`/`PUT /v1/hear/vocabulary` | Bearer | `hear:configure` | live | | Hear (streaming) | `GET /v1/audio/transcriptions/stream` (WS) | Subprotocol | `hear:stream` | live | | Cue | `GET /v1/audio/transcriptions/stream` + grounding (WS) | Subprotocol | `hear:stream` | unavailable | | Hear (async batch) | `POST`/`GET /v1/transcription/jobs` | Bearer | `transcribe:jobs` | live | | Speak (TTS) | `POST /v1/audio/speech` | Bearer | `speak:synthesize` | live | | Clone | `GET`/`POST /v1/voice/clones` | Bearer | `speak:clone` | live | | Speak (design) | `/v1/voice/design` | Bearer | `speak:design` | live | | Omni | `wss …/v1/omni?session_label=` | Subprotocol | `omni:session` | live | | Agent profiles (optional config) | `/v1/agents`, `/v1/agents/{id}` | Bearer | `omni:session` | live | | Knowledge Bases (hosted grounding) | `/v1/knowledgebases/*`, `PUT /v1/agents/{id}/knowledgebases` | Bearer | `kb:manage` (`omni:session` for the binding) | live | | Trace (config) | `/v1/trace/config`, `/v1/trace/rule-packs` | Bearer | `trace:configure` | beta | | Trace (reads) | `/v1/trace/interactions`, `/violations`, `/findings`, `/exposure` | Bearer | `trace:read` | beta | | Recap (config) | `/v1/recap/config` | Bearer | `recap:configure` | live | | Recap (CRM) | `/v1/recap/crm-config` | Bearer | `recap:configure` | live | | Integrations (Zapier) | `/v1/integrations/events`, `/v1/integrations/zapier/hooks` | Bearer | _any active key_ | live | | Recap (reads) | `/v1/recap/calls` | Bearer | `recap:read` | live | | Omni call records | `/v1/omni/calls`, `/v1/omni/calls/{id}` | Bearer | `omni:read` | live | | AMD (stream) | `wss …/v1/amd/stream` (Twilio Media Streams drop-in) | TwiML `` (from Twilio) or subprotocol (server-side) | `amd:detect` | live | | AMD (config) | `GET`/`POST /v1/amd/config` | Bearer | `amd:configure` | live | | AMD (reads) | `GET /v1/amd/calls`, `/v1/amd/calls/{id}` | Bearer | `amd:read` | live | | Telephony | `/v1/telephony/*` | Bearer | `telephony:manage` | live | | Agents (console builder) | `https://console.pyai.com/agents` | Console session |, | beta | WebSocket surfaces authenticate with the `Sec-WebSocket-Protocol: pyai.v1, pyai-key.` subprotocol pair (or `?api_key=` server-side); everything else takes the `Authorization: Bearer` key. Managed-number calls return 404 until the PyAI network is enabled for the account. ## Rate limits & billing Every key has a per-second rate limit (with burst) and a cap on concurrent realtime sessions. Exceeding either returns `429` with a `Retry-After` header. Usage is metered per minute of audio, transcription minutes (Hear), synthesized audio minutes (Speak), and realtime session minutes (Omni), and billed against your plan and credits. List prices: Hear $0.001/min (async Transcribe $0.0005/min), Speak $0.04/min, Omni $0.05/min including speech plus brain, and Agents Live Beta $0.08/min. Managed telephony is separate at $0.01/min. English Natural (`en1`) is available on Speak (streaming and buffered) and Omni; Omni acknowledges the canonical id and `voice_tier: natural`. Hindi uses the Standard-tier voices `hi1`–`hi4` on Omni; the former Hindi Natural aliases (`hi5`–`hi8`) are retired and no longer in the catalog. Standard and Natural voices are included in their product's base rate with no voice-tier add-on. The AMD API bills per **answered** call, the first 5,000 answered calls each month are free, then $0.004/answered call (no-answers, busies, and failed calls are free; AMD bundled with PyAI telephony/Omni is included at no charge). AI products (Hear, Speak, Omni) bill **per second by default**, the pulse is applied once to each meter's invoice-period total, so many short sessions are summed and rounded a single time (never minute-rounded per call), and an empty/failed call bills nothing. Coarser pulses are available as an optional enterprise override. Managed telephony minutes keep a 1-minute pulse. Per-character Speak billing is available on enterprise contracts. contact: name: PyAI url: https://pyai.com servers: - url: https://api.pyai.com description: Production security: - apiKey: [] - xApiKey: [] tags: - name: Dub description: >- Asynchronous dubbing: submit a recording, choose the languages, poll the job and download the output. - name: Omni description: >- The flagship: build an AI voice agent with one WebSocket (`GET /v1/omni`) and one `configure` frame, nothing to pre-create. This group also holds the optional browser-token mint and the post-call records. - name: Knowledge Bases description: >- Hosted knowledge bases for Omni grounding: create a base, add documents (file upload, URL fetch, or pasted text), then bind it to agent profiles or set org-wide defaults. Bound bases are retrieved per turn, no `kb_endpoint` of your own required. - name: Identity description: >- Introspect the calling key: org/project, env, granted scopes, and limits/credit posture. Use it to self-diagnose a 401/403/402. - name: Speech To Text (Hear) description: Speech-to-text (streaming + batch) - name: Text To Speech (Speak) description: Text-to-speech, stock voices, and prompt-to-voice design - name: Clone description: Enroll, list, and delete custom voices from a short reference clip - name: Models description: Model catalog - name: Sandbox description: >- Zero-friction onboarding for coding agents: mint a free, instant, no-card sandbox key with no human steps. - name: Startup Program description: >- PyAI for Startups: $20k to $100k in PyAI credit for early-stage voice teams. Public application endpoint; review and activation happen out of band. - name: Cast description: >- Auto-directed, expressive multi-line voiceover projects and asynchronous renders. - name: Transcription Jobs description: Async batch transcription - name: Agents description: >- Agent profiles used by the live Agents Beta console and available directly through the API. Store Omni session config (persona, greeting, voice, conversation knobs) and reference it by id instead of sending a full `configure` frame each call. Profiles remain optional for direct `/v1/omni` integrations. - name: Trace description: >- Compliance & guardrails: per-agent config, rule packs, and the exposure / violations / interaction-evidence read views - name: AMD description: >- Answering-machine detection: know who or what answered a call (human, voicemail, IVR, iPhone/Google screening, dead number), with the reason it decided. Twilio Media Streams drop-in over `wss …/v1/amd/stream`; one operating-point dial; billed per answered call. - name: Telephony description: >- Managed phone numbers: search, provision, route to an agent, and release. Call minutes bill on telephony.minutes ($0.01/min). - name: WhatsApp description: >- WhatsApp Business Calling: register a WhatsApp Business number, enable calling, and let an Omni agent answer (and, with the user's permission, place) WhatsApp voice calls. Requires the `telephony:manage` scope. - name: Call Integrations description: >- Signed provider webhooks that import completed calls into Hear, Recap, and offline Trace. paths: /v1/agents/{id}: post: tags: - Agents summary: Update an agent description: >- Partial update: present fields are set, `null` clears a field, absent fields are untouched. Config edits are live on the agent's next call. operationId: updateAgent parameters: - name: id in: path required: true schema: type: string requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/AgentConfig' responses: '200': description: Updated agent content: application/json: schema: $ref: '#/components/schemas/Agent' '400': description: Invalid field content: application/json: schema: $ref: '#/components/schemas/Error' '401': $ref: '#/components/responses/Unauthorized' '404': description: No such agent components: schemas: AgentConfig: type: object description: >- Writable agent fields. On update, present fields are set, `null` clears, absent fields are untouched. properties: name: type: string maxLength: 200 description: Display name. Required on create. persona_system_prompt: type: - string - 'null' maxLength: 32000 description: The agent's entire character, role, policies, and business context. role: type: - string - 'null' enum: - receptionist - support - sales - collections - ea - concierge - custom - null description: >- Role archetype. Adds PyAI's role operating standard beneath your persona and derives the runtime mode (`receptionist`, `sales`, and `collections` enable their matching deterministic conversation guards; everything else runs the default mode). Your stored persona is unchanged and remains authoritative for identity and business policy. Null clears the role. greeting: type: - string - 'null' maxLength: 1000 description: >- Greeting message, opening line spoken at turn 0 when a call connects (before the caller speaks). Stored on the agent profile; played automatically when connecting with `session_label={agent_id}`. May also be sent inline in the Omni `configure` frame. greeting_variants: type: - array - 'null' maxItems: 15 items: type: string maxLength: 1000 description: >- Approved opening-line variants. PyAI selects one for each newly resolved call profile; null or empty falls back to greeting. Recording consent is fixed and never rotated. voice_id: type: - string - 'null' description: >- A stock `voice_id`, one of its permanent `aliases` from `GET /v1/voices`, or a cloned voice id. Omni reports the canonical served stock id in `configured.voice_id`. voice_instruct: type: - string - 'null' maxLength: 200 description: >- Stored delivery direction for instruct-capable voice tiers. Current Omni routes support neutral delivery only: omit or set null for neutral speech. A saved custom value is rejected when configuring Omni; an inline empty `configure.voice_instruct` resets it to neutral for that session. brain_model: type: - string - 'null' description: Per-agent model selection. Omit for the platform default. barge_sensitivity: type: - string - 'null' deprecated: true description: >- Deprecated compatibility field. Stored values are not applied to Omni runtime. ack_mode: type: - string - 'null' deprecated: true description: >- Deprecated compatibility field. Stored values are not applied to Omni runtime. idle_check_in: type: - string - 'null' enum: - auto - patient - 'off' - null description: >- How patient the agent is before checking in on a silent caller ("Sorry, are you still there?"). `auto` checks in after a few seconds of silence; `patient` waits far longer, for callers who routinely think, read, or look something up mid-call; `off` disables the check-in entirely, so the agent stays silent until the caller speaks. When unset, most roles render `auto`; support agents render `patient`. Independent of `ack_mode`. May also be sent inline in the Omni `configure` frame, which wins for that session. persona_perspective: type: - string - 'null' enum: - agent - caller - null description: >- Which side of the call the persona is on. `agent` (default) means the persona is the business being called, so PyAI adds its conversation layer for handling a caller (capability honesty, handoffs, turn discipline). `caller` means the persona is the individual on the call instead, as in QA and simulation callers, mystery shopping, or training partners; PyAI drops that operator-voice layer so it cannot contradict an inverted persona. May also be sent inline in the Omni `configure` frame, which wins for that session. recordings_enabled: type: - boolean - 'null' description: Enable stereo call recordings. Default false. consent_line: type: - string - 'null' minLength: 10 maxLength: 500 description: >- Recording disclosure spoken before recording starts when `recordings_enabled` is true. Required for compliance when recordings are on and must contain a meaningful spoken phrase (at least 10 characters and 5 letters). Playback order: consent_line, then greeting, then conversation. language: type: - string - 'null' enum: - en - fr - es - de - hi - zh - null x-staged-language-ids: - ht - zh-CN - ar-MSA - ar-Gulf description: >- Requested language for this Agent's Omni sessions. `null` or absent means `en`. Availability is staged: public serving is `en`, `fr`, `es`, and `hi`; `de` and the legacy `zh` alias fall back to English unless explicitly enabled with qualified endpoints. Canonical staged route identifiers are exposed in `x-staged-language-ids` for internal integration only and are rejected by the public control plane until qualified. Inspect the session's `configured.language_active` and `language_fallback` fields before assuming the requested language is active. See the Language support reference. metadata: type: - object - 'null' additionalProperties: type: string description: Up to 16 key/value annotations (keys ≤64 chars, values ≤512 chars). vocabulary: type: - array - 'null' maxItems: 5 items: type: string minLength: 4 maxLength: 64 description: >- Optional custom vocabulary for this Agent's Omni speech recognition. A non-empty list is the opt-in. PyAI keeps at most five effective terms, with at most five words per term. It trims whitespace, deduplicates without regard to case while preserving the first spelling and order, and drops common-only phrases. Set `[]` or `null` to turn it off. The list is fixed when a session starts. Organization Hear vocabulary is never applied to Omni. keyterms: type: - array - 'null' items: type: string maxItems: 100 deprecated: true description: >- Deprecated stored compatibility field. It does not affect speech recognition. Use `vocabulary`. goals: type: - array - 'null' items: type: string maxItems: 20 description: >- Goal checklist for post-call outcome scoring (stored now; scoring ships with summaries). extraction_schema: type: - object - 'null' additionalProperties: true description: >- JSON Schema of fields to capture from each completed call's transcript. When set with extraction_webhook_url, PyAI runs a post-call extraction pass and POSTs the structured JSON to your webhook (signed with X-PyAI-Signature). Null disables extraction. extraction_webhook_url: type: - string - 'null' format: uri description: >- HTTPS URL that receives the signed post-call extraction result (event `omni.call.extracted`). Requires extraction_schema. The call's agent is resolved from the connect-URL session_label when it equals this agent's id. tools: type: - array - 'null' items: $ref: '#/components/schemas/AgentToolBinding' description: >- Tool bindings for this agent profile (references tools from `GET /v1/tools`). Omni `configure.tools[]` may list hosted catalog names or client-loop schemas; server webhooks must be registered here. An inline configure `endpoint` is not supported. continuity: type: - boolean - 'null' description: >- Reuse a tiny caller card on the next call when PyAI can resolve the caller (phone number or a signed customer id). The card is advisory — the agent may recall a name or open thread, and must not refund, book, or transfer from it. Default false. Agent: type: object properties: object: type: string example: agent agent_id: type: string example: agent_7f3a0b12 name: type: string persona_system_prompt: type: - string - 'null' greeting: type: - string - 'null' greeting_variants: type: array maxItems: 15 items: type: string voice_id: type: - string - 'null' voice_instruct: type: string description: >- Effective TTS delivery direction for instruct-capable voice tiers. Renders PyAI's natural conversational pace when no override is stored. brain_model: type: string example: default barge_sensitivity: type: - string - 'null' deprecated: true ack_mode: type: - string - 'null' deprecated: true idle_check_in: type: string enum: - auto - patient - 'off' description: >- Idle check-in patience. Renders `auto` when unset, except support agents render `patient`. persona_perspective: type: string enum: - agent - caller description: >- Which side of the call the persona is on. Renders the effective default (`agent`) when unset. mode: type: string enum: - default - receptionist - sales - collections readOnly: true example: default description: >- Read-only runtime mode derived from the agent's role (`default`, `receptionist`, `sales`, or `collections`). Set indirectly via the writable `role` field. role: type: - string - 'null' description: >- Role archetype the agent runs as, or null. Writable on create and update. continuity: type: boolean description: >- Whether this agent reuses a prior-call caller card when a caller key is available. Default false. recordings_enabled: type: boolean consent_line: type: - string - 'null' language: type: string enum: - en - fr - es - de - hi - zh x-staged-language-ids: - ht - zh-CN - ar-MSA - ar-Gulf description: >- Session language for this agent's calls. Renders the effective default (`en`) when unset. Canonical staged route identifiers remain excluded from the public enum until qualification. vocabulary: type: array maxItems: 5 items: type: string minLength: 4 maxLength: 64 description: >- Sanitized Agent vocabulary. An empty list means speech-recognition biasing is off. keyterms: type: array items: type: string deprecated: true goals: type: array items: type: string metadata: type: object additionalProperties: type: string extraction_schema: type: - object - 'null' additionalProperties: true description: Post-call extraction JSON Schema, or null. extraction_webhook_url: type: - string - 'null' description: Signed delivery target for post-call extraction, or null. tools: type: array items: $ref: '#/components/schemas/AgentToolBinding' created_at: type: integer description: Unix seconds. Error: type: object description: >- OpenAI-compatible error envelope returned by the gateway data plane (401/402/403/429). Control-plane request/resource errors use Problem (application/problem+json) instead. required: - error properties: error: type: object required: - message properties: message: type: string description: Human-readable explanation. type: type: string description: Error category, e.g. rate_limit_error. code: $ref: '#/components/schemas/ErrorCode' param: type: string nullable: true description: Offending parameter when applicable, else null. AgentToolBinding: type: object required: - tool_id properties: tool_id: type: string enabled: type: boolean default: true config: type: object additionalProperties: true description: >- Customer SETTINGS for this tool on this agent, keyed by the tool's config_schema field keys (e.g. { from_number, provider_api_key }). Fields marked secret are encrypted at rest and returned masked ('********'); re-send the mask (or omit) to keep the stored value. name: type: string description: Present on agent reads when the tool resolves. description: type: string input_schema: type: object additionalProperties: true description: >- Per-call model argument schema. Present on agent reads when the tool resolves. execution: type: string enum: - hosted - server - engine - client description: >- Resolved execution mode. Present on agent reads when the tool resolves. ErrorCode: type: string description: >- Stable, machine-readable error code. Branch on this rather than the human `message`. enum: - invalid_request_error - invalid_session_label - unauthorized - forbidden - origin_not_allowed - credit_exhausted - key_budget_exceeded - insufficient_quota - rate_limit_exceeded - concurrency_limit_exceeded - daily_cap_exceeded responses: Unauthorized: description: 'Missing or invalid API key (`code: unauthorized`)' content: application/json: schema: $ref: '#/components/schemas/Error' securitySchemes: apiKey: type: http scheme: bearer description: 'Use `Authorization: Bearer pyai_live_...` (or `pyai_test_...`).' xApiKey: type: apiKey in: header name: x-api-key description: >- Header alias for bearer auth on HTTP endpoints. WebSocket auth uses the subprotocol pair `pyai.v1, pyai-key.`. ```` This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.