Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
252 changes: 252 additions & 0 deletions client/concepts/events-and-callbacks.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,21 @@ You are currently viewing the React version of this page. Use the dropdown to th
You are currently viewing the JavaScript version of this page. Use the dropdown to the right to customize this page for your client framework.
</Callout>
</View>
<View title="React Native" icon="mobile">
<Callout icon="mobile" color="#FFC107">
You are currently viewing the React Native version of this page. Use the dropdown to the right to customize this page for your client framework.
</Callout>
</View>
<View title="iOS" icon="apple">
<Callout icon="apple" color="#FFC107">
You are currently viewing the iOS version of this page. Use the dropdown to the right to customize this page for your client framework.
</Callout>
</View>
<View title="Android" icon="android">
<Callout icon="android" color="#FFC107">
You are currently viewing the Android version of this page. Use the dropdown to the right to customize this page for your client framework.
</Callout>
</View>

The Pipecat client emits events throughout the session lifecycle — when the bot connects, when the user speaks, when a transcript arrives, and more.

Expand Down Expand Up @@ -69,6 +84,82 @@ Callbacks and event listeners are equivalent — use whichever pattern fits your

</View>

<View title="React Native" icon="mobile">

Use `useEffect` with `.on()` and `.off()` to subscribe and clean up within a component:

```tsx
import { RTVIEvent, TranscriptData } from "@pipecat-ai/client-js";
import { useEffect } from "react";

function TranscriptDisplay() {
useEffect(() => {
const handler = (data: TranscriptData) => {
if (data.final) setTranscript(data.text);
};
client.on(RTVIEvent.UserTranscript, handler);
return () => client.off(RTVIEvent.UserTranscript, handler);
}, []);
}
```

Always return a cleanup function from `useEffect` to remove the listener when the component unmounts.

</View>

<View title="iOS" icon="apple">

Conform your model to `PipecatClientDelegate` and assign it as the delegate after creating the client. All delegate methods are optional — implement only what you need:

```swift
let client = PipecatClient(options: PipecatClientOptions(
transport: SmallWebRTCTransport(),
enableMic: true
))
client.delegate = self

extension MyModel: PipecatClientDelegate {
func onBotReady(botReadyData: BotReadyData) {
Task { @MainActor in
print("Bot is ready")
}
}

func onUserTranscript(data: Transcript) {
Task { @MainActor in
if data.final ?? false { setTranscript(data.text) }
}
}
}
```

Delegate callbacks arrive on a background thread — always use `Task { @MainActor in }` before updating `@Published` properties or any UI state.

</View>

<View title="Android" icon="android">

Extend `PipecatEventCallbacks` and pass it in `PipecatClientOptions`. Override only the methods you need:

```kotlin
val callbacks = object : PipecatEventCallbacks() {
override fun onBotReady(data: BotReadyData) {
Log.d("Bot", "Bot is ready")
}

override fun onUserTranscript(data: Transcript) {
if (data.final) setTranscript(data.text)
}
}

val options = PipecatClientOptions(callbacks = callbacks)
val client = PipecatClient(DailyTransport(context), options)
```

Callbacks run on the main thread and can update Compose `mutableStateOf` directly.

</View>

---

## Event reference
Expand Down Expand Up @@ -157,6 +248,54 @@ client.on(RTVIEvent.UserTranscript, (data) => {

</View>

<View title="React Native" icon="mobile">

```tsx
useEffect(() => {
const handler = (data: TranscriptData) => {
if (data.final) {
addMessage(data.text); // committed
} else {
updatePartial(data.text); // still in progress
}
};
client.on(RTVIEvent.UserTranscript, handler);
return () => client.off(RTVIEvent.UserTranscript, handler);
}, []);
```

</View>

<View title="iOS" icon="apple">

```swift
func onUserTranscript(data: Transcript) {
Task { @MainActor in
if data.final ?? false {
addMessage(data.text) // committed
} else {
updatePartial(data.text) // still in progress
}
}
}
```

</View>

<View title="Android" icon="android">

```kotlin
override fun onUserTranscript(data: Transcript) {
if (data.final) {
addMessage(data.text) // committed
} else {
updatePartial(data.text) // still in progress
}
}
```

</View>

`BotOutput` is the recommended way to display the bot's response text. It provides the best possible representation of what the bot is saying — supporting interruptions and unspoken responses. By default, Pipecat aggregates output by sentences and words (assuming your TTS supports streaming), but custom aggregation strategies are supported too - like breaking out code snippets or other structured content:

<View title="React" icon="react">
Expand Down Expand Up @@ -186,6 +325,48 @@ client.on(RTVIEvent.BotOutput, (data) => {

</View>

<View title="React Native" icon="mobile">

```tsx
useEffect(() => {
const handler = (data: BotOutputData) => {
if (data.aggregated_by === "sentence") {
appendSentence(data.text);
}
};
client.on(RTVIEvent.BotOutput, handler);
return () => client.off(RTVIEvent.BotOutput, handler);
}, []);
```

</View>

<View title="iOS" icon="apple">

The iOS SDK exposes `onBotTranscript` for the bot's LLM text output:

```swift
func onBotTranscript(data: BotLLMText) {
Task { @MainActor in
appendSentence(data.text)
}
}
```

</View>

<View title="Android" icon="android">

`onBotTranscript` receives the bot's text output as it accumulates:

```kotlin
override fun onBotTranscript(text: String) {
appendSentence(text)
}
```

</View>

### Errors

| Event | Callback | When it fires |
Expand Down Expand Up @@ -226,6 +407,47 @@ client.on(RTVIEvent.Error, ({ data }) => {

</View>

<View title="React Native" icon="mobile">

```tsx
useEffect(() => {
const handler = ({ data }) => {
if (data.fatal) {
showReconnectPrompt(data.message);
} else {
showToast(data.message);
}
};
client.on(RTVIEvent.Error, handler);
return () => client.off(RTVIEvent.Error, handler);
}, []);
```

</View>

<View title="iOS" icon="apple">

```swift
func onError(message: RTVIMessageInbound) {
Task { @MainActor in
// message.data contains the error description string
showError(message.data ?? "Unknown error")
}
}
```

</View>

<View title="Android" icon="android">

```kotlin
override fun onBackendError(message: String) {
showError(message)
}
```

</View>

### Devices and tracks

| Event | Callback | When it fires |
Expand Down Expand Up @@ -285,3 +507,33 @@ For custom server\<-\>client messaging, see [Custom Messaging](/client/guides/cu
</CardGroup>

</View>

<View title="React Native" icon="mobile">

<CardGroup cols={1}>
<Card title="JavaScript SDK Callbacks" icon="js" href="/api-reference/client/js/callbacks">
Complete callback signatures, data types, and transport compatibility
</Card>
</CardGroup>

</View>

<View title="iOS" icon="apple">

<CardGroup cols={1}>
<Card title="iOS SDK Reference" icon="apple" href="/api-reference/client/ios/overview">
Full `PipecatClientDelegate` protocol and API reference
</Card>
</CardGroup>

</View>

<View title="Android" icon="android">

<CardGroup cols={1}>
<Card title="Android SDK Reference" icon="android" href="/api-reference/client/android/overview">
Full `PipecatEventCallbacks` class and API reference
</Card>
</CardGroup>

</View>
Loading
Loading