Skip to content
62 changes: 62 additions & 0 deletions docs/chat/streaming.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,68 @@ TanStack AI implements the [AG-UI Protocol](https://docs.ag-ui.com/introduction)

> **Tip:** Some models expose their internal reasoning as thinking content that streams before the response. See [Thinking & Reasoning](./thinking-content).

### Type-Safe Tool Call Events

When you pass typed tools (defined with `toolDefinition()` and Zod schemas) to `chat()`, the stream chunks automatically carry type information for tool call events. The `toolName` field narrows to the union of your tool name literals, and the `input` field on `TOOL_CALL_END` events is typed as the union of your tool input schemas:

```typescript
import { chat, toolDefinition } from "@tanstack/ai";
import { openaiText } from "@tanstack/ai-openai";
import { z } from "zod";

const weatherTool = toolDefinition({
name: "get_weather",
description: "Get weather for a location",
inputSchema: z.object({
location: z.string(),
unit: z.enum(["celsius", "fahrenheit"]).optional(),
}),
});

const stream = chat({
adapter: openaiText("gpt-5.2"),
messages,
tools: [weatherTool],
});

for await (const chunk of stream) {
if (chunk.type === "TOOL_CALL_END") {
chunk.toolName; // ✅ typed as "get_weather" (not string)
chunk.input; // ✅ typed as { location: string; unit?: "celsius" | "fahrenheit" }
}
}
```

Without typed tools, `toolName` defaults to `string` and `input` defaults to `unknown` — the same behavior as before. The type narrowing is automatic when you use `toolDefinition()` with Zod schemas.

When multiple tools are provided, tool call events form a **discriminated union** — checking `toolName` narrows `input` to that specific tool's type:

```typescript
const searchTool = toolDefinition({
name: "search",
inputSchema: z.object({ query: z.string() }),
});

const stream = chat({
adapter: openaiText("gpt-5.2"),
messages,
tools: [weatherTool, searchTool],
});

for await (const chunk of stream) {
if (chunk.type === "TOOL_CALL_END") {
if (chunk.toolName === "get_weather") {
chunk.input; // ✅ { location: string; unit?: "celsius" | "fahrenheit" }
}
if (chunk.toolName === "search") {
chunk.input; // ✅ { query: string }
}
}
}
```

> **Tip:** The typed stream chunk type is exported as `TypedStreamChunk<TTools>` if you need to annotate variables or function parameters. When used without type arguments, `TypedStreamChunk` is equivalent to `StreamChunk`.

### Thinking Chunks

Thinking/reasoning is represented by AG-UI events `STEP_STARTED` and `STEP_FINISHED`. They stream separately from the final response text:
Expand Down
37 changes: 36 additions & 1 deletion docs/reference/type-aliases/StreamChunk.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,42 @@ title: StreamChunk
type StreamChunk = AGUIEvent;
```

Defined in: [types.ts:976](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L976)
Defined in: [types.ts:989](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L989)

Chunk returned by the SDK during streaming chat completions.
Uses the AG-UI protocol event format.

# Type Alias: TypedStreamChunk

```ts
type TypedStreamChunk<TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<Tool<any, any, any>>>
```

Defined in: [types.ts:1033](https://github.com/TanStack/ai/blob/main/packages/typescript/ai/src/types.ts#L1033)
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Outdated

A variant of `StreamChunk` parameterized by the tools array. When specific tool types are provided (e.g. from `chat({ tools: [myTool] })`):

- `TOOL_CALL_START` and `TOOL_CALL_END` events form a **discriminated union** over tool names.
- Checking `toolName === 'x'` narrows `input` to that specific tool's input type.
- `TOOL_CALL_END` events have `input` typed per-tool via Zod schema inference.

When tools are untyped or absent, `TypedStreamChunk` degrades to the same type as `StreamChunk`.

This is the type returned by `chat()` when streaming is enabled (the default). You don't typically need to reference it directly unless annotating function parameters or return types.

```ts
import type { TypedStreamChunk } from "@tanstack/ai";
import { toolDefinition } from "@tanstack/ai";

// Given tools created with toolDefinition():
const weatherTool = toolDefinition({ name: "get_weather", description: "...", inputSchema: /* Zod schema */ });
const searchTool = toolDefinition({ name: "search", description: "...", inputSchema: /* Zod schema */ });

// Without type args — equivalent to StreamChunk
type Chunk = TypedStreamChunk;

// With specific tools — tool call events are typed
type TypedChunk = TypedStreamChunk<[typeof weatherTool, typeof searchTool]>;
```

See [Streaming - Type-Safe Tool Call Events](../../chat/streaming) for a practical walkthrough.
2 changes: 2 additions & 0 deletions docs/tools/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,6 +78,8 @@ const inputSchema: JSONSchema = {

> **Note:** When using JSON Schema, TypeScript will infer `any` for input/output types since JSON Schema cannot provide compile-time type information. Zod schemas are recommended for full type safety.

> **Tip:** Type safety from Zod schemas extends beyond tool execution — when you iterate over the stream returned by `chat()`, tool call events have typed `toolName` and `input` fields too. See [Type-Safe Tool Call Events](../chat/streaming#type-safe-tool-call-events).

## Tool Definition

Tools are defined using `toolDefinition()` from `@tanstack/ai`:
Expand Down
66 changes: 66 additions & 0 deletions examples/ts-react-chat/src/routes/api.tanchat.ts
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,72 @@ const loggingMiddleware: ChatMiddleware = {
},
}

// ===========================
// TypedStreamChunk showcase — type-safe tool call events
// ===========================
//
// When `chat()` receives tools with typed schemas, the returned stream
// carries type information on TOOL_CALL_START and TOOL_CALL_END events.
// No casts, no `as any` — just narrow by `chunk.type` and everything is typed.

const tools = [
getGuitars,
recommendGuitarToolDef,
addToCartToolServer,
addToWishListToolDef,
getPersonalGuitarPreferenceToolDef,
compareGuitars,
calculateFinancing,
searchGuitars,
] as const

async function typedStreamShowcase() {
const stream = chat({
adapter: openaiText('gpt-4o'),
messages: [
{ role: 'user' as const, content: 'Recommend an acoustic guitar' },
],
tools,
})

for await (const chunk of stream) {
switch (chunk.type) {
case 'TOOL_CALL_START':
// ✅ chunk.toolName is typed as the union of all tool name literals:
// 'getGuitars' | 'recommendGuitar' | 'addToCart' | 'addToWishList'
// | 'getPersonalGuitarPreference' | 'compareGuitars'
// | 'calculateFinancing' | 'searchGuitars'
//
// ❌ Without TypedStreamChunk, this would just be `string`
console.log(`Tool call started: ${chunk.toolName}`)
break

case 'TOOL_CALL_END':
// ✅ Discriminated union — checking toolName narrows input to that tool's type
if (chunk.toolName === 'searchGuitars') {
// ✅ chunk.input is { query: string } (not the full union)
console.log(`Searching for: ${chunk.input?.query}`)
} else if (chunk.toolName === 'calculateFinancing') {
// ✅ chunk.input is { guitarId: number; months: number }
console.log(
`Financing guitar ${chunk.input?.guitarId} for ${chunk.input?.months} months`,
)
} else {
console.log(`Tool call ended: ${chunk.toolName}`, chunk.input)
}
break

case 'TEXT_MESSAGE_CONTENT':
// Non-tool events are unaffected — still fully typed
console.log(chunk.delta)
break
}
}
}

// Suppress unused warning — this is a showcase, not called at runtime
void typedStreamShowcase

export const Route = createFileRoute('/api/tanchat')({
server: {
handlers: {
Expand Down
39 changes: 28 additions & 11 deletions packages/typescript/ai/src/activities/chat/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -43,6 +43,7 @@ import type {
ToolCallArgsEvent,
ToolCallEndEvent,
ToolCallStartEvent,
TypedStreamChunk,
} from '../../types'
import type {
ChatMiddleware,
Expand All @@ -69,11 +70,15 @@ export const kind = 'text' as const
* @template TAdapter - The text adapter type (created by a provider function)
* @template TSchema - Optional Standard Schema for structured output
* @template TStream - Whether to stream the output (default: true)
* @template TTools - The tools array type for type-safe tool call events in the stream
*/
export interface TextActivityOptions<
TAdapter extends AnyTextAdapter,
TSchema extends SchemaInput | undefined,
TStream extends boolean,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
> {
/** The text adapter to use (created by a provider function like openaiText('gpt-4o')) */
adapter: TAdapter
Expand All @@ -87,7 +92,7 @@ export interface TextActivityOptions<
/** System prompts to prepend to the conversation */
systemPrompts?: TextOptions['systemPrompts']
/** Tools for function calling (auto-executed when called) */
tools?: TextOptions['tools']
tools?: TTools
/** Controls the randomness of the output. Higher values make output more random. Range: [0.0, 2.0] */
temperature?: TextOptions['temperature']
/** Nucleus sampling parameter. The model considers tokens with topP probability mass. */
Expand Down Expand Up @@ -125,7 +130,7 @@ export interface TextActivityOptions<
outputSchema?: TSchema
/**
* Whether to stream the text result.
* When true (default), returns an AsyncIterable<StreamChunk> for streaming output.
* When true (default), returns an AsyncIterable<TypedStreamChunk<TTools>> for streaming output.
* When false, returns a Promise<string> with the collected text content.
*
* Note: If outputSchema is provided, this option is ignored and the result
Expand Down Expand Up @@ -186,9 +191,12 @@ export function createChatOptions<
TAdapter extends AnyTextAdapter,
TSchema extends SchemaInput | undefined = undefined,
TStream extends boolean = true,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
>(
options: TextActivityOptions<TAdapter, TSchema, TStream>,
): TextActivityOptions<TAdapter, TSchema, TStream> {
options: TextActivityOptions<TAdapter, TSchema, TStream, TTools>,
): TextActivityOptions<TAdapter, TSchema, TStream, TTools> {
return options
}

Expand All @@ -200,16 +208,22 @@ export function createChatOptions<
* Result type for the text activity.
* - If outputSchema is provided: Promise<InferSchemaType<TSchema>>
* - If stream is false: Promise<string>
* - Otherwise (stream is true, default): AsyncIterable<StreamChunk>
* - Otherwise (stream is true, default): AsyncIterable<TypedStreamChunk<TTools>>
*
* When tools with typed schemas are provided, the stream chunks include
* type-safe `toolName` and `input` fields on tool call events.
*/
export type TextActivityResult<
TSchema extends SchemaInput | undefined,
TStream extends boolean = true,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
> = TSchema extends SchemaInput
? Promise<InferSchemaType<TSchema>>
: TStream extends false
? Promise<string>
: AsyncIterable<StreamChunk>
: AsyncIterable<TypedStreamChunk<TTools>>

// ===========================
// ChatEngine Implementation
Expand Down Expand Up @@ -1374,9 +1388,12 @@ export function chat<
TAdapter extends AnyTextAdapter,
TSchema extends SchemaInput | undefined = undefined,
TStream extends boolean = true,
TTools extends ReadonlyArray<Tool<any, any, any>> = ReadonlyArray<
Tool<any, any, any>
>,
>(
options: TextActivityOptions<TAdapter, TSchema, TStream>,
): TextActivityResult<TSchema, TStream> {
options: TextActivityOptions<TAdapter, TSchema, TStream, TTools>,
): TextActivityResult<TSchema, TStream, TTools> {
const { outputSchema, stream } = options

// If outputSchema is provided, run agentic structured output
Expand All @@ -1387,7 +1404,7 @@ export function chat<
SchemaInput,
boolean
>,
) as TextActivityResult<TSchema, TStream>
) as TextActivityResult<TSchema, TStream, TTools>
}

// If stream is explicitly false, run non-streaming text
Expand All @@ -1398,13 +1415,13 @@ export function chat<
undefined,
false
>,
) as TextActivityResult<TSchema, TStream>
) as TextActivityResult<TSchema, TStream, TTools>
}

// Otherwise, run streaming text (default)
return runStreamingText(
options as unknown as TextActivityOptions<AnyTextAdapter, undefined, true>,
) as TextActivityResult<TSchema, TStream>
) as TextActivityResult<TSchema, TStream, TTools>
}

/**
Expand Down
Loading
Loading