Troubleshooting

Common symptoms when building real-time avatars, with their cause and fix.

🟢 SDK-handled — the Python SDK and Pipecat take care of this for you; it surfaces only on the raw WebSocket API (or if you turn a feature off). The simplest fix is usually "use the SDK."

🟠 Your setup — you own this regardless of integration: deployment, network, credentials, capacity, or raw-API tuning.

Avatar's mouth barely moves / only idle animation

🟢 SDK-handled

Cause: the model is starving. You're sending too little audio, or forwarding tiny TTS fragments one at a time, so the model can't sustain speech at 25 fps and emits idle frames in between.

Fix: feed audio with a lead — prime ~1 s, then send the largest chunks you can (~400 ms). See Optimizing Performance → Feed audio. OjinSTVClient already does this for you, so the simplest fix is to drive the model through the Python SDK or Pipecat.

Lip-sync drifts out of sync with the audio

🟢 SDK-handled

Cause: on the SDK path, you're adding your own A/V re-sync on top of an already-synced stream; or on the raw API, video is played off the audio clock or frames were reordered during async decode.

Fix: with the SDK or Pipecat, output_stream() is already aligned at 25 fps — don't re-sync it yourself. Present each frame as it arrives, or pass each frame's pts to your media transport (e.g. WebRTC) and let it handle sync. If you build your own loop on the raw API, drive playback from the audio clock and keep decode order stable — see Audio and Video Synchronization in the API Reference.

Choppy or crackling playback

🟢 SDK-handled

Cause: audio isn't playing at a steady rate, or the event loop is blocked.

Fix: play audio gaplessly at a steady rate; never stop it just because a video frame is late. Keep your frame handlers light — heavy synchronous work stalls the playback tick. See Optimizing Performance → Play back at realtime.

No video frames at all

🟢 SDK-handled

Cause: you haven't received SESSION_READY / SessionReady yet, or (SDK) you're reading frame.rgb while using a PassthroughDecoder.

Fix: wait for the session-ready signal before sending audio. With the SDK, the default decoder populates frame.rgb; a PassthroughDecoder leaves rgb as None and delivers the raw JPEG in source_bytes instead — read that, or keep the default decoder. See Optimizing Performance → Relay frames.

Growing latency over time

🟢 SDK-handled · 🟠 Your setup (deployment)

Cause: your buffer is too large, or frames backed up after a network stall. The model delivers at realtime 25 fps, so the buffer shouldn't grow on its own — keep only a small jitter buffer.

Fix: keep the buffer small (a few frames). If frames do back up on a poor connection, trim idle frames (frame_type == 0) to recover; never drop speech, start-of-speech, or fade-out frames. The SDK keeps the buffer small for you. Run the client on a backend in US East over a stable connection — see Optimizing Performance → Where to run.

Frame lag during speech (raw API)

🟠 Your setup (raw-API tuning)

Cause: the model's smoothing filter is adding responsiveness lag on the raw WebSocket protocol.

Fix: reduce the speech_filter_amount parameter — lower is more responsive but less smooth. (The SDK paces output for you, so this only applies when driving the raw API.)

"No backend servers available"

🟠 Your setup

Cause: inference capacity is momentarily exhausted.

Fix: retry shortly with backoff. On the SDK this surfaces as an ERROR event with code NO_BACKEND_SERVER_AVAILABLE; on the raw API as an ErrorResponse (or plain-text message) with BACKEND_UNAVAILABLE.

Connection or authentication fails

🟠 Your setup

Fix:

• ✓ Verify your API key and config_id, and that the config exists in your dashboard

• ✓ The Authorization header must use the raw API key (no Bearer prefix)

• ✓ Ensure your network allows WebSocket connections on port 443

Interruptions look abrupt, or the avatar keeps speaking after a barge-in

🟢 SDK-handled

Cause: frames already sent or buffered keep playing after you cancel.

Fix: the SDK's interrupt() fades audio and cancels server-side for you. On the raw API, pick an interruption strategy for your latency/smoothness trade-off — clear the buffer for an instant cut, or keep playing video while stopping audio for a smoother look. See Interruption Handling in the API Reference.

---

Still stuck? Contact support.