> For the complete documentation index, see [llms.txt](https://docs.ojin.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.ojin.ai/models/build-with-python-sdk/python-sdk-best-practices.md).

# Best Practices

The key things to get right for a smooth, low-latency avatar pipeline with OjinSTVClient, whether you're prototyping straight to a browser, relaying media over WebRTC, or publishing directly into a Li

## Essentials (every setup)

* **Run it server-side.** The SDK connects to Ojin over a **server-to-server** WebSocket. Run it in a backend process, never on an end-user device, and keep your API key off the client. In production, run it in **US East**, close to Ojin's inference, for the lowest latency.
* **Feed audio as your TTS produces it.** Call `start_turn()` once per utterance, then `send_tts_audio()` for each chunk as it arrives, even tiny 40 ms fragments. The SDK shapes the feed (establishes the lead, then coalesces to ≥400 ms) so the model stays ahead while your TTS keeps producing. A pause long enough to drain the lead still runs it short. Don't batch the whole utterance yourself.
* **Present frames as they arrive. Don't re-sync.** `output_stream()` is already paced to realtime **25 fps**, with audio and video in sync and a small jitter buffer. Forward each frame the moment it arrives; never hold audio to wait for a video frame, and don't add your own A/V sync layer.
* **Handle the lifecycle.** Wait for `SESSION_READY` before relying on the stream; use `BOT_STARTED_SPEAKING` / `BOT_STOPPED_SPEAKING` for UI state; on `ERROR`, retry with backoff (e.g. `NO_BACKEND_SERVER_AVAILABLE`).
* **Barge-in with `interrupt()`.** It fades the current audio and cancels the turn server-side, just call it when the user starts talking.

{% hint style="info" %}
On [Pipecat](/models/introduction/integrations.md)? `pipecat-ojin`'s `OjinVideoService` already wires all of this into a pipeline and pushes frames into your transport, start there instead of hand-rolling the loop.
{% endhint %}

## Setup A. Local, output to a browser (prototyping)

Run the SDK in a local Python process and stream frames to a browser you control (e.g. over a WebSocket to a `<canvas>`). Great for demos and development; **not** for production over real networks.

* **Forward the JPEG straight to the browser.** Keep the default decoder and read the raw JPEG from `STVVideoFrame.source_bytes`, then draw it to a `<canvas>`/`<img>` as it arrives. It is **empty (`b""`) on a held tick** — the tick where no new server frame was ready — so skip those and keep showing the last image. Do **not** use a `PassthroughDecoder` to skip the decode: it makes the client emit no video frames at all, so the loop below never runs. (Need raw RGB in Python instead? The default decoder gives you `frame.rgb` as well.)
* **Play audio at the rate you fed.** Run the browser `AudioContext` at the rate you pass to `send_tts_audio()`. `STVAudioFrame.sample_rate` reports it per frame, but frames emitted before your first TTS audio default to 16 kHz — don't configure the context from the first frame you see. You can feed higher-quality TTS (e.g. 24 kHz) for better sound; the SDK plays back your original audio while lip-sync uses a 16 kHz copy.
* **Present as they come.** Draw each video frame and queue each audio chunk the instant it arrives, no re-sync; the stream is already aligned.

```python
client = OjinSTVClient(
    api_key=..., config_id=...,
)
async for frame in client.output_stream():
    if isinstance(frame, STVVideoFrame):
        if frame.source_bytes:                         # empty on a held tick
            await ws.send_bytes(frame.source_bytes)    # raw JPEG -> draw on a canvas
    elif isinstance(frame, STVAudioFrame):
        await ws.send_bytes(frame.pcm)                 # play at frame.sample_rate
```

## Setup B. Backend, relay to a WebRTC service (production)

Run the SDK on a backend and relay the media to your users over a **WebRTC** service (LiveKit, Daily, mediasoup, …), which absorbs packet loss, jitter, and varying networks that a raw WebSocket can't.

{% hint style="info" %}
**On LiveKit or Daily?** Skip the relay and use **Setup C** below: Ojin publishes the avatar straight into your room, for lower latency and no media through your backend. Use Setup B when you need the frames in your process or use another WebRTC service.
{% endhint %}

* **Deploy in US East**, close to Ojin's inference.
* **Push frames into the transport with their `pts`.** Each `STVAudioFrame` / `STVVideoFrame` carries a `pts` timestamp in **monotonic nanoseconds**, so convert for your transport (`frame.pts // 1000` for a microsecond API, `frame.pts / 1e9` for seconds). Hand it to your WebRTC tracks and let the transport sync them. Don't re-sync yourself.
* **Push, don't poll, when you can.** Inject a custom `STVOutput` sink to write frames straight into your transport instead of draining `output_stream()`, for lower latency and less glue. (This is exactly what `pipecat-ojin` does.)
* **Match the video track to the model's frame size**: read `STVVideoFrame.width` / `height` and configure your outgoing track to match.
* **Deliver to end users over WebRTC, not the raw WebSocket.** Keep the Ojin WebSocket strictly server-to-server.

```python
async for frame in client.output_stream():
    if isinstance(frame, STVVideoFrame):
        video_track.push(frame.rgb, pts=frame.pts)     # feed your WebRTC track
    elif isinstance(frame, STVAudioFrame):
        audio_track.push(frame.pcm, pts=frame.pts)     # transport handles A/V sync
```

## Setup C. Direct WebRTC into your LiveKit or Daily room (recommended for rooms)

If your viewers are in a **LiveKit** or **Daily** room, let Ojin publish the avatar straight into it. This gives viewers the lowest latency, and your backend relays no media: it only sends TTS audio and handles events.

* **Pass `webrtc=WebRTCSettings(...)`** with the room URL and a token for the `ojin-avatar` participant. Mint the token on your backend and keep it there.
* **Set `audio_sample_rate` to your TTS output rate** so audio reaches the room without resampling.
* **Give the join room to breathe.** Set `webrtc_join_timeout_s` to about 30 s in production; it covers the room join and the model's cold start.
* **Drive UI state from events, not frames.** `output_stream()` receives no frames in this mode; use `WEBRTC_CONNECTED`, `BOT_STARTED_SPEAKING`, and `BOT_STOPPED_SPEAKING`.
* **Treat `ERROR` with `fatal=True` as the end of the session.** There is no fallback to the WebSocket; start a new session.
* **Keep your own bot from hearing the avatar.** If your agent is also in the room, unsubscribe it from the avatar's microphone.

```python
client = OjinSTVClient(
    api_key=..., config_id=...,
    webrtc=WebRTCSettings(provider="livekit", room_url=LIVEKIT_URL, token=avatar_token,
                          audio_sample_rate=24000, webrtc_join_timeout_s=30.0),
)
```

See [Direct WebRTC (LiveKit & Daily)](/models/build-with-python-sdk/python-sdk-webrtc.md) for credentials, events, errors, and Pipecat.

## See also

* [Build with the Python SDK](/models/build-with-python-sdk.md), install, auth, events, full quickstart
* [Direct WebRTC (LiveKit & Daily)](/models/build-with-python-sdk/python-sdk-webrtc.md), publish the avatar straight into your room
* [Optimizing Performance](/guides/optimizing-performance.md), the audio-feeding contract and tuning
* [Troubleshooting](/guides/troubleshooting.md), symptoms, causes, and fixes
* [API Reference (advanced)](/models/introduction/api.md), the raw WebSocket protocol


---

# Agent Instructions
This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com.

## Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter:

```
GET https://docs.ojin.ai/models/build-with-python-sdk/python-sdk-best-practices.md?ask=<question>&goal=<endgoal>
```

`ask` is the immediate question: it should be specific, self-contained, and written in natural language.
`goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal.

The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.