Concept

Voice mode

How the chat ↔ voice channel handoff works, what the widget does on each transition, and how to handle edge cases.

A single Buddy Assist conversation lives on one chat_session row for its entire lifetime. Voice is a transient attachment to that session: it occupies the session while active and drains its transcript back into the chat history when it ends, so the thread stays continuous across mode switches.

Channel state machine

      ┌─────────────────────────────────────┐
      │          chat_session               │
      │                                     │
 ─────►  current_channel = CHAT  ◄──────────┤
      │           │                         │
      │    POST /voice/connect              │  transcript drain
      │           │                         │  (bot end_conversation
      │           ▼                         │   OR POST /voice/end)
      │  current_channel = VOICE            │
      │           │                         │
      │    POST /voice/end  ─────────────►  │
      │                                     │
      │  current_channel = ENDED            │
      │  (when POST /end is called)         │
      └─────────────────────────────────────┘

CHAT → VOICE and VOICE → CHAT transitions are protected by a per-session Redis lock so concurrent requests can’t race on the same session.

Lifecycle: step by step

1. Chat is active

When the widget is open the SDK is connected to:

POST /widget/session/{id}/message   (SSE)

Each user message triggers a streaming assistant reply (text tokens + optional generative-UI blocks).

2. User taps the voice button — POST /voice/connect

The widget calls:

POST /agent/voice/breeze-buddy/widget/session/{id}/voice/connect
Authorization: Bearer <widget_token>

The server:

  1. Acquires the session lock.
  2. Verifies current_channel == CHAT (rejects with 409 if already on voice).
  3. Creates (or reuses) a lead_call_tracker row to carry the Daily session.
  4. Flips current_channel → VOICE.
  5. Calls Daily to create a room and mints a participant token.
  6. Returns { room_url, daily_token, lead_id, ttl_seconds }.

The widget then joins the Daily room using the SDK’s WebRTC transport. Chat message input is disabled while current_channel == VOICE.

Voice lead reuse

On the second and subsequent voice attachments within the same conversation, the server reuses the same **`lead_call_tracker`** row (incrementing `attempt_count`) rather than creating a new lead. This keeps the conversation backbone tidy — one chat session, one lead, N voice attempts.

3. Voice conversation runs

The Daily room TTL is 1 hour (ttl_seconds: 3600). The widget token outlives this (24 h TTL), so you can reconnect voice multiple times within the same session.

Recording is intentionally disabled for widget voice sessions. A future schema change would be needed to record per-attempt audio separately.

4. Voice ends — two paths

Path A — bot disconnects first (normal flow)

When the agent calls end_conversation (e.g. after a farewell node), the bot’s pipeline tears down, drains the voice transcript into chat_message rows, and flips current_channel back to CHAT. The widget detects the Daily session closing and re-enables the chat composer.

Path B — user ends voice from the widget

The widget calls:

POST /agent/voice/breeze-buddy/widget/session/{id}/voice/end
Authorization: Bearer <widget_token>

The server signals the bot to wind down via daily_completion_function and returns immediately (status: "end_requested"). The bot then drains the transcript asynchronously. The widget re-enables chat without waiting for the drain to complete.

Idempotent end

`POST /voice/end` is idempotent. If the channel is already `CHAT` (drain already ran), it returns { status: "not_active" } without error. Call it defensively on page unload without worrying about double-fires.

5. Session ends — POST /end

POST /agent/voice/breeze-buddy/widget/session/{id}/end
Authorization: Bearer <widget_token>

If voice is still live, the server calls /voice/end internally first, then marks the chat_session as ENDED. The voice_lead_id stays on the session row for audit and analytics.

Page reload — session resume

The widget persists { sessionId, widgetToken, expiresAt } in localStorage keyed by (baseUrl, publicWidgetKey, shopUrl). On the next page load it attempts:

GET /agent/voice/breeze-buddy/widget/session/{id}
Authorization: Bearer <widget_token>

If the session is still ACTIVE and the token hasn’t expired, the widget rehydrates the message history and resumes exactly where it left off — including any partially-drained voice transcript.

If the stored token has expired, or the GET returns 404/410, the widget silently creates a fresh session.

Handling voice-live-conflict

If a chat message arrives while current_channel == VOICE, the server returns 409. The SDK maps this to a SessionConflictError, which the widget surfaces as a breeze-buddy-assist:error event with code: "voice_live_conflict".

Listen for it to show a toast:

document.querySelector('breeze-buddy-assist').addEventListener(
  'breeze-buddy-assist:error',
  (e) => {
    if (e.detail.code === 'voice_live_conflict') {
      showToast('Please end your voice call before typing.');
    }
  }
);

Rate limits for voice

widget_config.max_voice_sessions_per_ip_hour (default 10) caps how many /voice/connect calls a single IP can make per hour. Exceeding it returns 429. If your widget sits behind a NAT that shares one IP across many users, raise this limit when creating/updating the widget config.

Was this helpful?
Edit on GitHub