-
Notifications
You must be signed in to change notification settings - Fork 65
docs: add websocket authentication and generic handler documentation #746
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
mattshep
wants to merge
6
commits into
main
Choose a base branch
from
ms/generic-websockets-handler-hmac-auth
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
50c5ef1
docs: add websocket authentication and generic handler documentation
mattshep 523a4b0
docs: replace bot-starter with Pipecat Cloud in user-facing text
mattshep 7eb6ebe
docs: update websocket auth default to none, add Twilio Function example
mattshep f9acee3
docs: use wsUrl from /start response, don't hardcode endpoint paths
mattshep c12f017
docs: move generic-websocket out of telephony, add telephony overview
markbackman d51ecfa
docs: clean up generic websocket and auth pages
mattshep File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,49 @@ | ||
| --- | ||
| title: Generic WebSocket | ||
| description: "Connect any WebSocket client — telephony, server-to-server, or custom integrations — to your Pipecat Cloud agents" | ||
| --- | ||
|
|
||
| The generic WebSocket endpoint is a protocol-agnostic way to connect a WebSocket client to your Pipecat Cloud agent. Unlike the provider-specific endpoints (`/ws/twilio`, `/ws/plivo`, etc.), it imposes no protocol requirements. Messages are relayed directly between the client and your bot without validation or transformation. | ||
|
|
||
| ## When to Use | ||
|
|
||
| - **Telephony providers**: Connect any telephony provider that supports WebSocket audio streaming. | ||
| - **Server-to-server integrations**: Connect your own backend or a third-party platform over WebSocket. | ||
| - **Custom WebSocket clients**: Any client that can establish a WebSocket connection. | ||
|
|
||
| <Note> | ||
| **Web and mobile clients**: Browsers and mobile apps can connect via | ||
| WebSocket, but we recommend WebRTC instead. WebRTC handles varying network | ||
| conditions, media quality, and reconnect behavior better than raw WebSocket. | ||
| See the [Daily WebRTC guide](/pipecat-cloud/guides/daily-webrtc) or the | ||
| SmallWebRTC [client SDKs](/client/js/introduction) for recommended web and | ||
| mobile options. | ||
| </Note> | ||
|
|
||
| ## Endpoint | ||
|
|
||
| ``` | ||
| wss://{region}.api.pipecat.daily.co/ws/generic/{agentName}.{organizationName} | ||
| ``` | ||
|
|
||
| The service host (`{agentName}.{organizationName}`) can also be provided as a query parameter: | ||
|
|
||
| ``` | ||
| wss://{region}.api.pipecat.daily.co/ws/generic?serviceHost={agentName}.{organizationName} | ||
| ``` | ||
|
|
||
| ## Authentication | ||
|
|
||
| The generic endpoint supports optional HMAC token authentication, controlled by the `websocket_auth` setting on your agent. See the [WebSocket Authentication guide](/pipecat-cloud/guides/websocket-authentication) for details on how to configure and use token authentication. | ||
|
|
||
| When `websocket_auth` is set to `none`, clients can connect directly without a token: | ||
|
|
||
| ```bash | ||
| wscat -c "wss://us-west.api.pipecat.daily.co/ws/generic/my-agent.my-org" | ||
| ``` | ||
|
|
||
| ## Message Format | ||
|
|
||
| The generic endpoint supports both text and binary WebSocket frames. Your bot receives messages exactly as sent by the client — no protocol-specific parsing or transformation is applied. | ||
|
|
||
| Your bot code is responsible for handling the client's message format. Use the appropriate [serializer](/api-reference/server/services/serializers/introduction) for your client's protocol, or implement custom message handling for non-standard clients. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,87 @@ | ||
| --- | ||
| title: Telephony Overview | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Added this new guide to help tie together the telephony section. It's not related to the change, but was a good opportunity to add it. |
||
| sidebarTitle: Overview | ||
| description: "Connect telephony providers to your Pipecat Cloud agents" | ||
| --- | ||
|
|
||
| Pipecat Cloud supports telephony through two broad approaches: **WebSocket streaming** from a telephony provider, or **Daily PSTN**. Pick the approach that matches how your phone numbers are provisioned and what call-control features you need. | ||
|
|
||
| ## WebSocket Providers | ||
|
|
||
| Telephony providers stream real-time audio to and from your Pipecat Cloud agent over a WebSocket connection. Pipecat Cloud ships dedicated endpoints for the major providers, each with protocol validation for added reliability. | ||
|
|
||
| All provider integrations follow a common setup pattern: | ||
|
|
||
| 1. Get your organization name with `pipecat cloud organizations list`. | ||
| 2. Configure the provider's call flow to connect to `wss://{region}.api.pipecat.daily.co/ws/{provider}?serviceHost={agentName}.{organizationName}`. | ||
| 3. Assign the configuration to your phone number. | ||
|
|
||
| <CardGroup cols={2}> | ||
| <Card | ||
| title="Twilio" | ||
| icon="phone" | ||
| href="/pipecat-cloud/guides/telephony/twilio-websocket" | ||
| > | ||
| Connect Twilio numbers via Media Streams and a TwiML Bin. | ||
| </Card> | ||
| <Card | ||
| title="Telnyx" | ||
| icon="phone" | ||
| href="/pipecat-cloud/guides/telephony/telnyx-websocket" | ||
| > | ||
| Connect Telnyx numbers via a TeXML Application. | ||
| </Card> | ||
| <Card | ||
| title="Plivo" | ||
| icon="phone" | ||
| href="/pipecat-cloud/guides/telephony/plivo-websocket" | ||
| > | ||
| Connect Plivo numbers via XML responses from your own server. | ||
| </Card> | ||
| <Card | ||
| title="Exotel" | ||
| icon="phone" | ||
| href="/pipecat-cloud/guides/telephony/exotel-websocket" | ||
| > | ||
| Connect Exotel numbers via an App Bazaar Voicebot flow. | ||
| </Card> | ||
| </CardGroup> | ||
|
|
||
| ## Custom Telephony Providers | ||
|
|
||
| If your provider isn't listed above, use the **generic WebSocket endpoint**. It relays messages directly without protocol validation, so your bot is responsible for parsing the provider's format. | ||
|
|
||
| <Card | ||
| title="Generic WebSocket" | ||
| icon="plug" | ||
| href="/pipecat-cloud/guides/generic-websocket" | ||
| > | ||
| Protocol-agnostic WebSocket endpoint for custom telephony providers and other | ||
| integrations. | ||
| </Card> | ||
|
|
||
| ## Daily PSTN | ||
|
|
||
| Daily provisions phone numbers and delivers calls to your Pipecat Cloud agent through a Daily room — no WebSocket integration required. This approach uses WebRTC under the hood and supports advanced call-control features like transfers. | ||
|
|
||
| <CardGroup cols={2}> | ||
| <Card | ||
| title="Daily Dial-In" | ||
| icon="phone" | ||
| href="/pipecat-cloud/guides/telephony/daily-dial-in" | ||
| > | ||
| Receive incoming PSTN calls on a Daily-provisioned number. | ||
| </Card> | ||
| <Card | ||
| title="Daily Dial-Out" | ||
| icon="phone" | ||
| href="/pipecat-cloud/guides/telephony/daily-dial-out" | ||
| > | ||
| Have your agent place outbound PSTN calls. | ||
| </Card> | ||
| </CardGroup> | ||
|
|
||
| ## Common Topics | ||
|
|
||
| - **[Regional endpoints](/pipecat-cloud/guides/regions#regional-websocket-endpoints)**: Reduce latency by connecting to the region closest to your agent. | ||
| - **[WebSocket authentication](/pipecat-cloud/guides/websocket-authentication)**: Require HMAC tokens to prevent unauthorized connections to your agent. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,153 @@ | ||
| --- | ||
| title: WebSocket Authentication | ||
| description: "Secure your WebSocket connections with HMAC token authentication" | ||
| --- | ||
|
|
||
| Pipecat Cloud supports optional HMAC token authentication for WebSocket connections. When enabled, clients must obtain a short-lived session token via the `/start` endpoint before connecting, preventing unauthorized session starts. | ||
|
|
||
| ## How It Works | ||
|
|
||
| <Steps> | ||
| <Step title="Client calls /start"> | ||
| Your client calls the `/start` endpoint with `transport: "websocket"` and | ||
| your public API key. Pipecat Cloud generates a signed HMAC token and returns | ||
| it along with the WebSocket URL. | ||
| </Step> | ||
| <Step title="Client connects with token"> | ||
| The client connects to the returned WebSocket URL with the token included as | ||
| a header, query parameter, or URL path segment. No bot is started until the | ||
| client connects. | ||
| </Step> | ||
| <Step title="Pipecat Cloud validates"> | ||
| The token is validated before the WebSocket connection is accepted. Invalid, | ||
| expired, or replayed tokens are rejected with an HTTP 403 — no bot resources | ||
| are allocated. | ||
| </Step> | ||
| </Steps> | ||
|
|
||
| ## Configuration | ||
|
|
||
| WebSocket authentication is controlled by the `websocket_auth` setting in your deployment config. Valid values are: | ||
|
|
||
| | Value | Behavior | | ||
| | ------- | ---------------------------------------------------- | | ||
| | `none` | No authentication required (default) | | ||
| | `token` | Requires an HMAC session token obtained via `/start` | | ||
|
|
||
| Set it in your `pcc-deploy.toml`: | ||
|
|
||
| ```toml pcc-deploy.toml | ||
| agent_name = "my-agent" | ||
| websocket_auth = "token" | ||
|
|
||
| [scaling] | ||
| min_agents = 1 | ||
| ``` | ||
|
|
||
| ## Getting a Token | ||
|
|
||
| Call the `/start` endpoint with `transport` set to `"websocket"`: | ||
|
|
||
| ```bash | ||
| curl -X POST "https://api.pipecat.daily.co/v1/public/my-agent/start" \ | ||
| -H "Authorization: Bearer pk_your_public_key" \ | ||
| -H "Content-Type: application/json" \ | ||
| -d '{"transport": "websocket"}' | ||
| ``` | ||
|
|
||
| Response: | ||
|
|
||
| ```json | ||
| { | ||
| "token": "eyJzaCI6Im15LWFnZW50Lm15LW9yZy...", | ||
| "wsUrl": "wss://us-west.api.pipecat.daily.co/ws/generic/my-agent.my-org", | ||
| "sessionId": "639f91d8-d511-4677-a83b-bd7564d5d92f" | ||
| } | ||
| ``` | ||
|
|
||
| <Info> | ||
| Unlike other transport types, `transport: "websocket"` does not start a bot | ||
| immediately. The bot starts when the client connects to the WebSocket URL. | ||
| </Info> | ||
|
|
||
| ## Connecting with a Token | ||
|
|
||
| Use the `wsUrl` from the `/start` response as your WebSocket endpoint. Always use this URL rather than constructing one manually — the format may change in the future. | ||
|
|
||
| The token can be attached to the `wsUrl` in three ways, checked in this order: | ||
|
|
||
| ### Authorization Header | ||
|
|
||
| ``` | ||
| Authorization: Bearer <token> | ||
| ``` | ||
|
|
||
| ### Query Parameter | ||
|
|
||
| Append `?token=<token>` to the `wsUrl`: | ||
|
|
||
| ``` | ||
| {wsUrl}?token=<token> | ||
| ``` | ||
|
|
||
| ### URL Path | ||
|
|
||
| Append `/<token>` to the `wsUrl`: | ||
|
|
||
| ``` | ||
| {wsUrl}/<token> | ||
| ``` | ||
|
|
||
| This is the recommended method for telephony providers that don't support query | ||
| parameters (such as Twilio). | ||
|
|
||
| ## Token Details | ||
|
|
||
| - **Lifetime**: 5 minutes from generation. Connect promptly after calling `/start`. | ||
| - **One-time use**: Each token can only be used once. Replayed tokens are rejected. | ||
| - **Scoped**: Tokens are bound to a specific agent and organization. They cannot be used with a different agent. | ||
|
|
||
| ## Using with Telephony Providers | ||
|
|
||
| Telephony providers like Twilio, Plivo, and Telnyx initiate WebSocket connections | ||
| directly — they can't call `/start` themselves. To use token authentication with | ||
| these providers, call `/start` from your existing webhook handler and embed the | ||
| token in the WebSocket URL that you return to the provider. | ||
|
|
||
| ### Example: Twilio Function | ||
|
|
||
| This Twilio Function calls `/start` to get a token, then returns TwiML with the | ||
| authenticated WebSocket URL: | ||
|
|
||
| ```javascript | ||
| exports.handler = async function (context, event, callback) { | ||
| // Call PCC /start to get a websocket auth token | ||
| const startResponse = await fetch( | ||
| "https://api.pipecat.daily.co/v1/public/my-agent/start", | ||
| { | ||
| method: "POST", | ||
| headers: { | ||
| Authorization: `Bearer ${context.PCC_PUBLIC_KEY}`, | ||
| "Content-Type": "application/json", | ||
| }, | ||
| body: JSON.stringify({ transport: "websocket" }), | ||
| }, | ||
| ); | ||
| const { token, wsUrl } = await startResponse.json(); | ||
|
|
||
| // Return TwiML with the authenticated WebSocket URL | ||
| const twiml = new Twilio.twiml.VoiceResponse(); | ||
| const connect = twiml.connect(); | ||
| connect.stream({ url: `${wsUrl}/${token}` }); | ||
| callback(null, twiml); | ||
| }; | ||
| ``` | ||
|
|
||
| <Tip> | ||
| Store your Pipecat Cloud public API key as an environment variable in your | ||
| Twilio Function settings (e.g., `PCC_PUBLIC_KEY`) rather than hardcoding it. | ||
| </Tip> | ||
|
|
||
| The same pattern works for any provider — call `/start` from your webhook handler, | ||
| get the token, and include it in the provider-specific XML response. The token | ||
| goes in the URL path, which is supported by all providers. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a new top-level page highlighting the generic websocket endpoint.
I'm keeping this separate from the auth page since we're linking to the auth page from a few places.