> ## Documentation Index
> Fetch the complete documentation index at: https://langchain-5e9cc07a-preview-docsdy-1782656769-5d7a089.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Trace OpenAI Realtime applications

> Trace OpenAI Realtime voice agents in LangSmith using the LangSmith SDK.

The [OpenAI Realtime API](https://platform.openai.com/docs/guides/realtime) powers low-latency speech-to-speech voice agents. This guide shows how to trace a Realtime app to LangSmith.

OpenAI Realtime is a speech-to-speech model: it processes audio natively and exchanges a continuous stream of typed JSON events with your application over a persistent WebSocket connection, rather than making discrete request/response calls. The following sections show those events and how to turn them into a LangSmith trace. For our high-level philosophy on getting the most out of your voice agent traces, see [Voice tracing fundamentals](/langsmith/trace-voice-fundamentals).

For a complete implementation, see the [voice demo repository](https://github.com/langchain-ai/voice-demo).

## The event model

Every event has a discriminated `type` string that indicates what it represents: audio, a tool call, and so on.

The client **sends** events to configure the session and stream audio. You do not trace these as spans; they are requests, and their effects come back as server events:

| Client event                | What it does                                                                                                |
| --------------------------- | ----------------------------------------------------------------------------------------------------------- |
| `session.update`            | Configure the session: instructions, voice, audio formats, transcription model, turn detection, tools.      |
| `input_audio_buffer.append` | Stream a base64 PCM16 mic chunk. Sent continuously.                                                         |
| `conversation.item.create`  | Add an item—used to return a `function_call_output` after running a tool.                                   |
| `response.create`           | Ask the model to generate a response (needed explicitly when turn detection uses `create_response: false`). |

The server **sends back** events:

| Server event                                            | What it carries                                                                                     | Traced?                  |
| ------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | ------------------------ |
| `session.created` / `session.updated`                   | Handshake / config acknowledgement.                                                                 | Yes                      |
| `input_audio_buffer.speech_started`                     | Server VAD heard the user start—the **barge-in signal**; flush the speaker buffer.                  | Yes                      |
| `input_audio_buffer.speech_stopped`                     | Server VAD heard the user stop.                                                                     | Yes                      |
| `input_audio_buffer.committed`                          | The audio buffer became a conversation item.                                                        | Yes                      |
| `conversation.item.created`                             | An item was added server-side.                                                                      | Yes                      |
| `conversation.item.input_audio_transcription.completed` | The full user transcript for the turn.                                                              | Yes                      |
| `response.created`                                      | The model started generating.                                                                       | Yes                      |
| `response.output_audio.delta`                           | One chunk of agent speech (base64 PCM16); hundreds per response.                                    | No—played, never spanned |
| `response.output_audio_transcript.delta`                | Streaming fragment of the agent's transcript.                                                       | No                       |
| `response.output_audio_transcript.done`                 | The agent's full transcript for the response.                                                       | Yes                      |
| `response.function_call_arguments.delta` / `.done`      | Streaming / final tool-call arguments.                                                              | `.done` only             |
| `response.output_item.*`, `response.content_part.*`     | Structural progress of the response.                                                                | Yes                      |
| `response.done`                                         | The complete response object: all output items (including every `function_call`), plus token usage. | Yes                      |
| `error`                                                 | Server-reported error.                                                                              | Yes                      |
| `rate_limits.updated`                                   | Quota bookkeeping.                                                                                  | Yes                      |

## How events map to LangSmith runs

We recommend tracing the whole conversation as a single trace, with one span per traced event in arrival order:

```text theme={null}
realtime_session                                 ← root run (chain)
│   metadata: thread_id, model, event_count, duration_s, ls_modality=audio
│   attachments: conversation.wav (stereo: L=user, R=agent)
│
├─ input_audio_buffer.speech_started
├─ input_audio_buffer.speech_stopped
├─ conversation.item.input_audio_transcription.completed
├─ response.created
├─ response.function_call_arguments.done
├─ response.done
│   └─ lookup_weather × N                         ← tool runs, nested under the event that announced them
├─ response.done                                  ← the spoken follow-up after tools
└─ error                                          ← only if the server sent one
```

<Note>
  **The noise rule:** we recommend skipping every event type ending in `.delta`, because the matching `.done` event repeats the complete payload. Tracing both records everything twice. `response.output_audio.delta` in particular is the agent's voice: hundreds of chunks per response that would bury the trace. Play it to the speaker, but never make it a span.
</Note>

## Installation

```bash theme={null}
pip install "langsmith>=0.4" "openai>=1.50"
```

The demo also uses `sounddevice` and `numpy` for the mic/speaker and to build the WAV attachment.

## Set up your environment

The following steps demonstrate how to trace using the LangSmith SDK. You can also trace using OpenTelemetry directly. See [Trace with OpenTelemetry](/langsmith/trace-with-opentelemetry).

```bash theme={null}
export LANGSMITH_API_KEY=...
export LANGSMITH_TRACING=true
export LANGSMITH_PROJECT=my-voice-app
export OPENAI_API_KEY=...
```

## Quickstart

<Note>
  This guide focuses on the tracing layer. It assumes you already have a working Realtime app: the WebSocket `connection`, the `session.update` configuration, and your microphone and speaker I/O. For a complete, runnable implementation, see the [voice demo repository](https://github.com/langchain-ai/voice-demo/tree/main/src/voice_demo/openai). Enable `input_audio_transcription` (and the agent transcript) in your `session.update`, or the transcription events that make the trace readable never arrive.
</Note>

### Step 1: Open the conversation root at connect time

Use one `RunTree` per conversation:

```python theme={null}
from langsmith import RunTree

root = RunTree(
    name="realtime_session",
    run_type="chain",
    inputs={},
    project_name="my-voice-app",
    extra={"metadata": {"thread_id": thread_id, "model": MODEL, "ls_modality": "audio"}},
)
root.post()
```

A stable `thread_id` you generate per conversation (for example, a UUID) groups the trace into a LangSmith [thread](/langsmith/threads); `ls_modality="audio"` marks it as a voice conversation.

### Step 2: Span each received event, skipping the noise

Define a small helper that opens a child run for one event, records the scrubbed payload as the run's input, and closes it when the block exits:

```python theme={null}
from contextlib import contextmanager

@contextmanager
def event_span(parent, event, *, name):
    """Trace one event as a child run, with its payload as the run's input."""
    payload = event.model_dump(mode="json")
    child = parent.create_child(name=name, run_type="chain", inputs={"event": payload})
    child.post()
    try:
        yield child
    finally:
        child.end()
        child.patch()
```

Then loop over the events from your open Realtime `connection`, skipping the `.delta` noise and tracing the rest:

```python theme={null}
async for event in connection:
    if event.type.endswith(".delta"):
        continue  # the matching .done event repeats the full payload

    with event_span(root, event, name=event.type) as event_run:
        ...  # your handling for this event type
```

### Step 3: Run tools nested under the announcing event

```python theme={null}
import json

from langsmith.run_helpers import tracing_context

if event.type == "response.done":
    calls = [i for i in (event.response.output or []) if i.type == "function_call"]
    for call in calls:
        with tracing_context(parent=event_run):
            result = await execute_tool(call.name, call.arguments)  # traced child
        await connection.conversation.item.create(item={
            "type": "function_call_output",
            "call_id": call.call_id,
            "output": json.dumps(result),
        })
    if calls:
        await connection.response.create()   # ask for the spoken follow-up
```

## Attach the conversation audio

To listen to a conversation alongside its transcript, attach a single combined recording of the whole conversation to the root run. Record both the user and the agent in one file (for example, a stereo WAV with the user mic on one channel and the agent on the other), captured from what was played to the client so the recording reflects what was actually heard, including speech cut off by a barge-in. The Realtime API streams agent audio as `response.output_audio.delta` events: decode and write those bytes to your output device, and tap that same output to build the recording.

For the underlying attachment API, see [Upload files with traces](/langsmith/upload-files-with-traces). For the cross-provider rationale, see [Record a single combined audio file](/langsmith/trace-voice-fundamentals#record-a-single-combined-audio-file).

When the conversation ends, finalize the root run:

```python theme={null}
root.end()
root.patch()
```

## Next steps

<CardGroup cols={2}>
  <Card title="Voice fundamentals" icon="waveform" href="/langsmith/trace-voice-fundamentals">
    Core conventions for tracing voice agents.
  </Card>

  <Card title="Upload files with traces" icon="paperclip" href="/langsmith/upload-files-with-traces">
    Attach the conversation audio recording to your trace.
  </Card>
</CardGroup>

***

<div className="source-links">
  <Callout icon="terminal-2">
    [Connect these docs](/use-these-docs) to Claude, VSCode, and more via MCP for real-time answers.
  </Callout>

  <Callout icon="edit">
    [Edit this page on GitHub](https://github.com/langchain-ai/docs/edit/main/src/langsmith/trace-openai-realtime.mdx) or [file an issue](https://github.com/langchain-ai/docs/issues/new/choose).
  </Callout>
</div>