> For the complete documentation index, see [llms.txt](https://docs.ojin.ai/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.ojin.ai/models/build-with-python-sdk/python-sdk-webrtc.md). # Direct WebRTC (LiveKit & Daily) Publish the avatar straight into your LiveKit or Daily room with one extra argument to OjinSTVClient. No media relay through your backend, and the lowest latency for your viewers. By default, `OjinSTVClient` streams the avatar's frames back to your process over the WebSocket. If your viewers are in a **LiveKit** or **Daily** room, Ojin can publish the avatar **straight into that room** instead. You add one argument and the rest of your program stays the same. Requires `ojin-client` 0.11.0 or later (`pipecat-ojin` 0.1.5 or later for Pipecat). ## How it works 1. Your backend creates the room and a token for the avatar with your LiveKit or Daily account. 2. You pass the room URL and token to `OjinSTVClient` through `WebRTCSettings`. 3. Ojin joins the room as a participant named **`ojin-avatar`** and publishes the avatar's audio and video there. 4. Your viewers (browser, widget, or app) watch the avatar in the room like any other participant. Your backend still sends TTS audio and receives events over the same WebSocket, but it carries only a lightweight control channel. No audio or video flows through your backend. The SDK doesn't join the room itself, and it doesn't create rooms or tokens. ## WebRTC or WebSocket? | | WebSocket (default) | Direct WebRTC | | ---------------------- | --------------------------------------------------- | --------------------------------------------------- | | Where the media goes | to your process, via `output_stream()` | straight into your LiveKit or Daily room | | Latency to viewers | adds your relay hop | lowest, no relay | | Backend cost | you decode and re-publish every frame | control messages only | | Frames in your process | yes | no | | Choose it when | you render, record, or post-process frames yourself | your viewers are already in a LiveKit or Daily room | ## Prerequisites * Everything from [Build with the Python SDK](/models/build-with-python-sdk.md): an API key, a `config_id`, and `ojin-client[stv]` installed * A LiveKit or Daily account, a room, and a token for the avatar {% tabs %} {% tab title="LiveKit" %} Mint an access token for the avatar. It only needs to join the room and publish: ```python import os from livekit import api # pip install livekit-api avatar_token = ( api.AccessToken(os.environ["LIVEKIT_API_KEY"], os.environ["LIVEKIT_API_SECRET"]) .with_identity("ojin-avatar") .with_grants(api.VideoGrants( room_join=True, room="my-room", can_publish=True, can_subscribe=False, can_publish_data=False, )) .to_jwt() ) ``` Mint the token with the identity `ojin-avatar`: on LiveKit a participant's identity comes from the token, so that claim is what your viewers use to tell the avatar apart from everyone else in the room. | `WebRTCSettings` field | Value | | ---------------------- | ------------------------------------------------------------- | | `provider` | `WebRTCProvider.LIVEKIT` | | `room_url` | your LiveKit server URL, `wss://.livekit.cloud` | | `token` | the access token above | | {% endtab %} | | {% tab title="Daily" %} Create a room and a meeting token for it with the [Daily REST API](https://docs.daily.co/reference/rest-api): ```bash curl -X POST https://api.daily.co/v1/rooms \ -H "Authorization: Bearer $DAILY_API_KEY" \ -H "Content-Type: application/json" \ -d '{"name": "my-room"}' curl -X POST https://api.daily.co/v1/meeting-tokens \ -H "Authorization: Bearer $DAILY_API_KEY" \ -H "Content-Type: application/json" \ -d '{"properties": {"room_name": "my-room"}}' ``` | `WebRTCSettings` field | Value | | ---------------------- | ----------------------------------------------------- | | `provider` | `WebRTCProvider.DAILY` | | `room_url` | the room URL, `https://.daily.co/` | | `token` | a meeting token for that room | | {% endtab %} | | | {% endtabs %} | | {% hint style="warning" %} Keep room credentials on your backend and never send the avatar's token to a browser. {% endhint %} ## Switch to direct WebRTC Pass `webrtc=WebRTCSettings(...)`. Leave it out and the same code runs over the WebSocket. ```python import os from ojin.stv import OjinSTVClient, WebRTCProvider, WebRTCSettings client = OjinSTVClient( api_key=os.environ["OJIN_API_KEY"], config_id=os.environ["OJIN_CONFIG_ID"], webrtc=WebRTCSettings( provider=WebRTCProvider.LIVEKIT, # Daily: WebRTCProvider.DAILY room_url="wss://your-project.livekit.cloud", # Daily: https://your-domain.daily.co/room token=avatar_token, # credential for ojin-avatar audio_sample_rate=24000, # the rate your TTS emits webrtc_join_timeout_s=30.0, ), ) async with client: await client.say(pcm, sample_rate=24000, num_channels=1) # the avatar speaks in the room ``` `provider` also accepts the plain strings `"livekit"` and `"daily"`, if you carry the value in config. Everything else works as before: `start()`, `start_turn()`, `send_tts_audio()`, `say()`, `interrupt()`, `close()`, `async with`, and every event. {% hint style="info" %} **Tune two settings.** * **`audio_sample_rate`**: set it to your TTS output rate (8000 to 48000 Hz, divisible by 25; default 16000) so audio reaches the room without resampling. * **`webrtc_join_timeout_s`**: bounds the whole wait for the session to be ready, including the room join and the model's cold start. The default is 10 s. Use about **30 s** in production. {% endhint %} {% hint style="warning" %} **`output_stream()` is empty in this mode.** The audio and video go to the room, not to your process, so `output_stream()` and any custom `STVOutput` receive no frames. The stream ends when the session closes. Use events for speaking state and timing. {% endhint %} ## Events | Event | Fires when | | ---------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `SESSION_READY` | the session is live and ready for audio | | `WEBRTC_CONNECTED` | the avatar has joined the room (`participant_id`) | | `FIRST_FRAME` | the first avatar frame is out (`frame_type`) | | `BOT_STARTED_SPEAKING` | the avatar starts speaking a turn | | `BOT_STOPPED_SPEAKING` | the avatar finishes a turn | | `INTERRUPTED` | `interrupt()` took effect — in this mode it also fires when the barge-in only discarded input buffered before the room was open. Use `interrupt()`'s return value to tell the two apart | | `ERROR` | an error occurred (`message`, `fatal`, plus `code` for errors that carry one) | | `CLOSED` | the session has been torn down | `FIRST_FRAME` fires in WebSocket mode too. ## Errors The client never falls back to the WebSocket silently. If the avatar can't get into the room, or later drops out of it, you get a fatal `ERROR` and the session closes. Each cause has its own `code`, so you can branch on it without reading the message: | `code` | What happened | What to do | | ------------------------- | ---------------------------------------------------------------------------------- | ---------------------------------------------------------------------------------------------------------------- | | `WEBRTC_AUTH_FAILED` | the room rejected the avatar's token | check the token is for the room in `room_url`, hasn't expired, and allows publishing | | `WEBRTC_NETWORK_FAILED` | Ojin couldn't reach the room | retry the session; if it persists, check your provider's status | | `WEBRTC_INVALID_SETTINGS` | the room URL or provider was unusable | check `room_url` matches the provider (`wss://` for LiveKit, the room URL for Daily) | | `WEBRTC_JOIN_TIMEOUT` | the session wasn't ready within `webrtc_join_timeout_s` | raise the timeout — it covers the model's cold start as well as the room join | | `WEBRTC_ROOM_LOST` | the avatar dropped out of the room mid-session | start a new session | | `WEBRTC_NOT_SUPPORTED` | the server returned no room result, so this session can't be published into a room | connect without `webrtc` to receive frames over the WebSocket, or [contact support](/getting-started/support.md) | | `WEBRTC_JOIN_FAILED` | the join failed for a reason this SDK doesn't recognise | read `message` for the server's own code, retry, then [contact support](/getting-started/support.md) | ```python @client.on(STVEvent.ERROR) def on_error(code: str = "", message: str = "", fatal: bool = False, **_): if code == "WEBRTC_AUTH_FAILED": remint_avatar_token() elif code == "WEBRTC_JOIN_TIMEOUT": retry_with_longer_timeout() ``` Invalid settings, such as an unknown provider or an empty `room_url` or `token`, raise `ValueError` when you construct `WebRTCSettings`. ## Pipecat Pass the same `WebRTCSettings` to `OjinVideoService`. The service pushes no media in this mode, so turn off your transport's audio and video output: ```python import os from pipecat_ojin import OjinVideoService, OjinVideoSettings, WebRTCSettings from ojin.stv import WebRTCProvider avatar = OjinVideoService( OjinVideoSettings( api_key=os.environ["OJIN_API_KEY"], config_id=os.environ["OJIN_CONFIG_ID"], webrtc=WebRTCSettings( provider=WebRTCProvider.LIVEKIT, room_url=room_url, token=avatar_token ), ) ) # Transport params: audio_out_enabled=False, video_out_enabled=False ``` Lifecycle frames are unchanged. See the [Pipecat integration](/models/introduction/integrations.md) for the full pipeline. ## Where to go next
Build with the Python SDKInstall, authenticate, and run the quickstart.Python SDK
Best PracticesChoose the right setup for your deployment.Best Practices
TroubleshootingJoin failures, timeouts, and session errors.Troubleshooting
--- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter, and the optional `goal` query parameter: ``` GET https://docs.ojin.ai/models/build-with-python-sdk/python-sdk-webrtc.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is optional and describes the broader end goal you are ultimately trying to accomplish on behalf of the user. GitBook uses it to tailor the answer towards what is most useful for that goal. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.