Live voice conversation — setup and operations

Live voice conversation — setup and operations

The waveform button in the chat composer opens a live, back-and-forth voice session with Jordyn: the user talks, she answers out loud, and everything said also lands in the chat timeline. Until it's configured, the button shows a friendly "voice isn't switched on yet" and nothing else changes.

Switching it on

Voice runs entirely on Deepgram — streaming speech-to-text (nova-3) and Aura text-to-speech from one account. (ElevenLabs is unaffected: it still narrates presentations.)

1. Create an account at console.deepgram.com and open your project.
2. Note the Project ID (project settings).
3. Create an API key with Member or higher permissions — key creation requires it.
4. Set both environment variables and redeploy:

DEEPGRAM_API_KEY=…
DEEPGRAM_PROJECT_ID=…

How the security model works

Vercel's serverless functions can't hold a websocket open, so the user's browser connects to Deepgram directly. It never sees the master key: when a session starts, the server mints a temporary key (POST /v1/projects/{id}/keys) scoped to usage:write only, with a 2-hour time-to-live — it can transcribe and speak, nothing else, and expires on its own. The browser authenticates the STT websocket with that key via the websocket subprotocol and calls the Aura TTS endpoint with it directly.

What a session costs the business

Booked conservatively at ~$0.04 per session minute (Deepgram streaming STT + Aura TTS + Haiku conversational turns), written to ai_usage as feature voice so the cost dashboard and daily budget guard see it like everything else. Two things keep real spend well under that ceiling:

• Client-side voice activity detection: silence is never streamed to Deepgram, so dead air isn't billed.
• The slim voice brain: casual turns run on Haiku with a ~40-line prompt. Only when the user asks for real work (send, search, book, build) does the request escalate into the normal chat pipeline with the full toolset — and that path meters actions and credits exactly as typed chat does.

Allowance and overage

• Paid plans include 60 voice minutes per calendar month; trials get 5 minutes total for the month.
• Past the allowance, paid accounts continue at 2 credits per started 10-minute block (voice_minutes in feature prices — note this key is a documented margin exception, priced as a brake rather than a profit line; the reasoning lives in lib/pricing.ts).
• Trials past their 5 minutes are done for the month, with a friendly note that chat is still free.
• Metering is a 30-second heartbeat from the client into usage_events (kind voice_seconds). All reads fail open: a database blip never blocks a paying customer mid-conversation. saas_admin accounts are never limited or charged.

Session behavior worth knowing

• Sessions are enter/leave (tap in, tap out, or Esc) — not hold-to-talk.
• Barge-in: the user speaking over Jordyn stops her audio immediately.
• Idle: after 90 seconds of silence she asks "Still there?"; at 3 minutes the session ends itself.
• Every exchange lands in the chat thread, so nothing spoken is lost when the session ends.