Direct WebRTC (LiveKit & Daily)

Publish the avatar straight into your LiveKit or Daily room with one extra argument to OjinSTVClient. No media relay through your backend, and the lowest latency for your viewers.

By default, OjinSTVClient streams the avatar's frames back to your process over the WebSocket. If your viewers are in a LiveKit or Daily room, Ojin can publish the avatar straight into that room instead. You add one argument and the rest of your program stays the same.

Requires ojin-client 0.11.0 or later (pipecat-ojin 0.1.5 or later for Pipecat).

How it works

1. Your backend creates the room and a token for the avatar with your LiveKit or Daily account.

2. You pass the room URL and token to OjinSTVClient through WebRTCSettings.

3. Ojin joins the room as a participant named ojin-avatar and publishes the avatar's audio and video there.

4. Your viewers (browser, widget, or app) watch the avatar in the room like any other participant.

Your backend still sends TTS audio and receives events over the same WebSocket, but it carries only a lightweight control channel. No audio or video flows through your backend. The SDK doesn't join the room itself, and it doesn't create rooms or tokens.

WebRTC or WebSocket?

	WebSocket (default)	Direct WebRTC
Where the media goes	to your process, via output_stream()	straight into your LiveKit or Daily room
Latency to viewers	adds your relay hop	lowest, no relay
Backend cost	you decode and re-publish every frame	control messages only
Frames in your process	yes	no
Choose it when	you render, record, or post-process frames yourself	your viewers are already in a LiveKit or Daily room

Prerequisites

• Everything from Build with the Python SDK: an API key, a config_id, and ojin-client[stv] installed

• A LiveKit or Daily account, a room, and a token for the avatar

LiveKit

Mint an access token for the avatar. It only needs to join the room and publish:

import os
from livekit import api  # pip install livekit-api

avatar_token = (
    api.AccessToken(os.environ["LIVEKIT_API_KEY"], os.environ["LIVEKIT_API_SECRET"])
    .with_identity("ojin-avatar")
    .with_grants(api.VideoGrants(
        room_join=True, room="my-room",
        can_publish=True, can_subscribe=False, can_publish_data=False,
    ))
    .to_jwt()
)

Mint the token with the identity ojin-avatar: on LiveKit a participant's identity comes from the token, so that claim is what your viewers use to tell the avatar apart from everyone else in the room.

WebRTCSettings field	Value
provider	WebRTCProvider.LIVEKIT
room_url	your LiveKit server URL, wss://<your-project>.livekit.cloud
token	the access token above

Daily

Create a room and a meeting token for it with the Daily REST API:

curl -X POST https://api.daily.co/v1/rooms \
  -H "Authorization: Bearer $DAILY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name": "my-room"}'

curl -X POST https://api.daily.co/v1/meeting-tokens \
  -H "Authorization: Bearer $DAILY_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"properties": {"room_name": "my-room"}}'

WebRTCSettings field	Value
provider	WebRTCProvider.DAILY
room_url	the room URL, https://<your-domain>.daily.co/<room>
token	a meeting token for that room

Keep room credentials on your backend and never send the avatar's token to a browser.

Switch to direct WebRTC

Pass webrtc=WebRTCSettings(...). Leave it out and the same code runs over the WebSocket.

import os
from ojin.stv import OjinSTVClient, WebRTCProvider, WebRTCSettings

client = OjinSTVClient(
    api_key=os.environ["OJIN_API_KEY"],
    config_id=os.environ["OJIN_CONFIG_ID"],
    webrtc=WebRTCSettings(
        provider=WebRTCProvider.LIVEKIT,                # Daily: WebRTCProvider.DAILY
        room_url="wss://your-project.livekit.cloud",    # Daily: https://your-domain.daily.co/room
        token=avatar_token,                             # credential for ojin-avatar
        audio_sample_rate=24000,                        # the rate your TTS emits
        webrtc_join_timeout_s=90.0,
    ),
)

async with client:
    await client.say(pcm, sample_rate=24000, num_channels=1)  # the avatar speaks in the room

provider also accepts the plain strings "livekit" and "daily", if you carry the value in config. Everything else works as before: start(), start_turn(), send_tts_audio(), say(), interrupt(), close(), async with, and every event.

Tune two settings.

• audio_sample_rate: set it to your TTS output rate (8000 to 48000 Hz, divisible by 25; default 16000) so audio reaches the room without resampling.

• webrtc_join_timeout_s: bounds the whole wait for the session to be ready, including the room join and the model's cold start. The default is 10 s. Use about 90 s in production: a cold start alone can take more than 30 s.

output_stream() is empty in this mode. The audio and video go to the room, not to your process, so output_stream() and any custom STVOutput receive no frames. The stream ends when the session closes. Use events for speaking state and timing.

Events

Event	Fires when
SESSION_READY	the session is live and ready for audio
WEBRTC_CONNECTED	the avatar has joined the room (participant_id)
FIRST_FRAME	the first avatar frame is out (frame_type)
BOT_STARTED_SPEAKING	the avatar starts speaking a turn
BOT_STOPPED_SPEAKING	the avatar finishes a turn
INTERRUPTED	interrupt() took effect. In this mode it also fires when the barge-in only discarded input buffered before the room was open. Use interrupt()'s return value to tell the two apart
ERROR	an error occurred (message, fatal, plus code for errors that carry one)
CLOSED	the session has been torn down

FIRST_FRAME fires in WebSocket mode too.

The video track is silent until the first frame. WEBRTC_CONNECTED means the avatar joined the room and published its tracks. It does not mean video is flowing. Nothing is sent on the video track until the model renders its first frame, which is what FIRST_FRAME reports.

That gap is the model's startup. It is under a second on a warm persona and can reach a few seconds on a cold one. webrtc_join_timeout_s bounds it.

If your viewer UI reveals the avatar as soon as it subscribes to the track, it shows an empty video element for that gap. Hold your own placeholder until FIRST_FRAME, or until your client reports a decoded frame. Gate the reveal on the signal rather than on the track: a UI that reveals on its first decoded frame shows whatever the track happened to carry before the avatar started.

Errors

The client never falls back to the WebSocket silently. If the avatar can't get into the room, or later drops out of it, you get a fatal ERROR and the session closes. Each cause has its own code, so you can branch on it without reading the message:

code	What happened	What to do
WEBRTC_AUTH_FAILED	the room rejected the avatar's token	check the token is for the room in room_url, hasn't expired, and allows publishing
WEBRTC_NETWORK_FAILED	Ojin couldn't reach the room	retry the session; if it persists, check your provider's status
WEBRTC_INVALID_SETTINGS	the room URL or provider was unusable	check room_url matches the provider (wss:// for LiveKit, the room URL for Daily)
WEBRTC_JOIN_TIMEOUT	the session wasn't ready within webrtc_join_timeout_s	raise the timeout. It covers the model's cold start as well as the room join
WEBRTC_ROOM_LOST	the avatar dropped out of the room mid-session	start a new session
WEBRTC_NOT_SUPPORTED	the server returned no room result, so this session can't be published into a room	connect without webrtc to receive frames over the WebSocket, or contact support
WEBRTC_JOIN_FAILED	the join failed for a reason this SDK doesn't recognise	read message for the server's own code, retry, then contact support

@client.on(STVEvent.ERROR)
def on_error(code: str = "", message: str = "", fatal: bool = False, **_):
    if code == "WEBRTC_AUTH_FAILED":
        remint_avatar_token()
    elif code == "WEBRTC_JOIN_TIMEOUT":
        retry_with_longer_timeout()

Invalid settings, such as an unknown provider or an empty room_url or token, raise ValueError when you construct WebRTCSettings.

Pipecat

Pass the same WebRTCSettings to OjinVideoService. The service pushes no media in this mode, so turn off your transport's audio and video output:

import os
from pipecat_ojin import OjinVideoService, OjinVideoSettings, WebRTCSettings
from ojin.stv import WebRTCProvider

avatar = OjinVideoService(
    OjinVideoSettings(
        api_key=os.environ["OJIN_API_KEY"],
        config_id=os.environ["OJIN_CONFIG_ID"],
        webrtc=WebRTCSettings(
            provider=WebRTCProvider.LIVEKIT, room_url=room_url, token=avatar_token
        ),
    )
)
# Transport params: audio_out_enabled=False, video_out_enabled=False

Lifecycle frames are unchanged. See the Pipecat integration for the full pipeline.

Where to go next

Build with the Python SDK

Install, authenticate, and run the quickstart.

Best Practices

Choose the right setup for your deployment.

Troubleshooting

Join failures, timeouts, and session errors.