# voice_interface memory

Non-obvious learnings, phrasing patterns that worked, sanitization gotchas, glasses-display quirks discovered in practice. Append-only.

## tmux pane reading (2026-06-13)
- The bottom `❯ ...` line, especially with `-- INSERT --` shown on the bottom
  bar, is an UNSENT DRAFT in the target agent's input box — NOT what they are
  doing. Read current activity from the body ABOVE the prompt box; never report
  draft text as live activity. (Caught on yams during bootstrap dry-run.)
- Busy/working cue = a status line like `Working… (Ns · ↑ Mk tokens)` or
  `Crunched for Ns · N shells, M monitor running`. Its presence = thinking; its
  disappearance = reply complete -> that's the poll cue for verb B (send+wait).
- A clean `❯` with no Working/Crunched line and no `-- INSERT --` = idle.

## Sanitization (2026-06-13)
- Helper: `agents/voice_interface/sanitize.py` — `sanitize(text, max_chars=300)`.
  Strips ANSI, folds smart quotes/em-dash/ellipsis/checkmarks/arrows to ASCII,
  drops remaining non-ASCII (glasses font silently eats them), caps length.
  ALWAYS run captured pane text through it before quoting to the display.

## Dispatch convention — audio-note prefix (Cameron, 2026-06-13)
EVERY message I send to another agent (verb B) must be wrapped as:
  -- This is an audio note coming from the voice interface chat so interpret
  typos as you best understand them: "<the message>"
So the target reads it as a possibly-noisy voice transcript and charitably
fixes Whisper typos. Apply to all send-keys dispatches. (The message body still
gets the draft-clear C-u first.)

## Monitor-after-dispatch loop (Cameron, 2026-06-13)
After dispatching to an agent, WATCH it on a ~12s poll and proactively push him
an update whenever there's something worth reporting -- a bug, a stuck state,
turn completion, or it's waiting for input -- even while it's still thinking.
Update shapes he wants:
  - "still thinking but it observed: <X>"
  - "now done. it said: <X>"
  - "waiting for your input: <X>"
Implementation: `agents/voice_interface/watch_agent.sh <window>` run in
background (cron/ScheduleWakeup floor at 60s, too slow for his 10-15s ask, so a
background bash watcher is the right tool). It polls ~12s and EXITS on:
waiting (permission/question box) | idle (main turn ended) | error/stuck keyword
| heartbeat (~90s). Its exit wakes me; I read the de-noised tail, judge worth,
push an update or silently re-arm. Discriminator for "thinking" = a token-counter
line `· ↓/↑` is present (the v3-finetune background monitor's spinner shows
`· NN shells`, never `· ↓`, so it is filtered out as noise).

## Gotcha: preserve Cameron's draft before C-u (2026-06-14)
Before clearing a target's input with C-u to send a message, CAPTURE the current
draft line first (tmux capture-pane | grep '^❯'). If it's a non-empty draft
Cameron typed directly (e.g. "all of them" answering a pending question),
clearing it silently destroys his in-progress input. Surface it ("set aside your
draft X") and offer to relay it. Wiped his "all of them" places-scope answer once
this way -- don't repeat. Prefer capturing draft in the same step as C-u.

## Default routing rule (Cameron, 2026-06-14)
DEFAULT for any random/personal remark Cameron makes (about his life, not the
fleet): relay it to the **life_manager** agent and summarize its reply back.
Relay prefix must identify me as voice_interface relaying an audio note AND ask
life_manager to answer CONCISELY (glasses-sized), e.g.:
  -- voice_interface relaying an audio note from Cameron (interpret typos
  charitably); reply CONCISELY, a few lines for a glasses HUD: "<message>"
Exceptions I handle myself (do NOT route to life_manager):
- "what is the <X> agent doing/up to" -> read that agent's pane directly.
- anything about another named agent -> that agent.
- meta-instructions/config/tests aimed at me -> me.
Mnemonic: life -> life_manager; agents -> the agent / me.

## Send-keys operational playbook (learned 2026-06-13..15)
Reusable steps for verb B (dispatch). Tooling: `watch_agent.sh` (on disk in
agents/voice_interface/) + per-message temp file.
1. CHECK STATE FIRST: idle iff NO real token-counter line. Detect "thinking"
   with `grep -qE '· [↓↑] [0-9]'` (arrow+space+DIGIT). Do NOT use bare
   `· [↓↑]` -- it false-matches the status-bar hint "· ↓ to manage" shown when
   the agent has background tasks/monitors, making an idle agent look busy.
2. PRESERVE DRAFTS: capture `^❯` line BEFORE `C-u`. If Cameron typed a draft,
   surface it ("set aside your draft X") -- never silently wipe. See
   [[#Gotcha: preserve Cameron's draft before C-u (2026-06-14)]].
3. APOSTROPHES: compose the message to a temp file (Write tool), send via
   `tmux send-keys -t agents:X -l -- "$(cat /tmp/vi_*.txt)"`. Single-quote bash
   wrapping breaks on I'll/I'm/don't.
4. LEADING `--`: messages start with the audio-note prefix `-- ...`. tmux parses
   a leading `--` as a flag -> use the `--` option terminator:
   `send-keys -t agents:X -l -- "<msg>"`. Without it you get a usage error.
5. SUBMIT + NUDGE: send-keys text, sleep 1, Enter, sleep 1, Enter. Moderate/long
   messages land as a COLLAPSED PASTE ("paste again to expand") and do NOT
   submit on their own -> if not "thinking" a few seconds after, send ONE more
   Enter (the nudge). Pattern:
   `if ! grep -qE '· [↓↑] [0-9]'; then send-keys Enter; fi`
6. MONITOR: `watch_agent.sh <window> 12 <heartbeat>` in background. It exits
   (wakes me) on waiting/idle/changed/heartbeat; I read the de-noised tail,
   judge worth, push a `voice:` line or silently re-arm. Re-arm after each wake
   until the task reaches a clean stop (done / waiting-for-input).
7. WATCHER NOISE: filter is tuned to drop yams' v4-monitor spam (Step N vol,
   cook-timer spinners), "Running… (Ns)" tool timers, compaction progress
   (`Compacting conversation` + `▰▱` bars). Add patterns to NOISE if a new
   recurring line causes false "changed" wakes.
8. yams hits 100% context and AUTO-COMPACTS mid-task (pauses ~1-3 min). Launches
   queued in background bash wait-loops survive it; yams just can't confirm until
   it resumes. Flag the compaction to Cameron, keep watching.

## Wiring (2026-06-13)
- serve.py ALREADY has `/api/agents/<name>/send` (serve.py:937) using the exact
  send-keys+double-Enter pattern, and `_find_tmux_window`. The future
  `/api/voice/ask` should reuse both rather than reinvent.
- Whisper STT already live at `/api/transcribe` (faster_whisper 'small',
  multipart field "audio", returns {text}). glasses already wired its `record`
  action to it. The missing piece is only ask->me->answer routing.
- Proposed answer delivery: single-line `agents/voice_interface/answer.txt`
  (overwrite per turn) instead of `tail -1 outbox.md` (multi-line, fragile).

## Relay VERBATIM, not paraphrased (Cameron, 2026-07-02)
New standing rule from Cameron: when relaying his voice notes to another agent,
send the **exact transcript**, not a paraphrase/restructure. Prefix it as his
voice note, then the raw words. Strip only the trailing harness bracket
(`[glasses chat from Cameron -- ...]`). Prefix pattern:
`-- Voice note from Cameron, relayed verbatim by the voice_interface agent
(interpret Whisper typos charitably)[; reply CONCISELY, glasses-sized]:
"<EXACT transcript>"`
Verbatim applies to what I send OUT to agents; the `voice:` summary I send back
to his HUD can still be distilled. Supersedes the old tidy-numbered-brief habit.
See [[feedback_voice_dispatch_style]] in auto-memory.