---
date: 2026-06-10
phase: 3
headline: Multi-view volumetric (per-voxel grip/rot) — new canonical arch
---

## What we did
- **Architecture pivot**: multi-view volumetric model with **per-voxel grip/rot heads** is now canonical, replacing the single-view KV-factored production arch (as of today)
- **Two-view input**: scene cam (fixed ZED, aruco PnP locked) + right-wrist (FK + hand-eye). Shared DINOv3 ViT-S/16+ backbone, one forward per view.
- **World-space voxel grid**: P×P=128×128 pixel grid × Z=64 height bins per view, N=2 slabs stacked → V ≈ 1M voxels. View identity + height live in the slab dim. No time axis on the voxels.
- **Per-voxel features (cross-view fusion happens here)**: for each voxel, project its world XYZ into every view, bilinear-sample each view's feat_head (32ch) maps, concat with 16-d learned height embedding, → 3-layer MLP → voxel_feat (32-d).
- **Volume scoring**: T=32 query heads off global state ([eef | cls_scene | cls_wrist]) → einsum bilinear scores → per-timestep softmax over voxels. CE on GT-nearest-3D voxel (no +1 slot trick).
- **Grip + rot**: per-step gather at GT voxel (teacher) / argmax voxel (infer) → 3-layer 1D CNN over T → per-step linear heads to 32 grip / 64 rot bins.
- **Deploy**: action chunking depth=4 + closed-form **reverse-projection volume pool** (~5 ms/chunk, replaces the 3s cKDTree path) — older chunks' logits remapped onto the newest chunk's voxel grid.
- **Optional flags (off by default)**: `--cross_view_layers N` (DA3-style alternating within/cross self-attn over DINO patches, ~7M params + 60ms/forward for N=4); `--border_mask_px N` (default 5, zeros logits in N-pixel frame at every view's pred-grid edge).

## Reflection
- **Why per-voxel won locally over global-CLS**: rot correlates strongly with EE position, so a per-position prediction has the right spatial inductive bias. Earlier global-CLS variant beat it (rot loss 0.73 vs 2.60 at step 2400 on the 35-ep wrist-only set) but that was a capacity issue — 32-d voxel features can't carry what the 800-d [eef|cls|cls] global input did. Today's bet: bigger / fresh datasets buy back the gap. Fallback if rot still lags: global residual `grip_logits_v = local(voxel_feat_v) + global(CLS_concat)`.
- **Binding framing locked** (canonical pitch as of today): *"Today's policies must learn the EEF↔pixel association + depth-along-ray + scene geometry + camera-to-robot extrinsics, all from image-to-action labels. PARA binds the action space to its projecting image features and reformulates policy learning as feature selection over a discretized candidate volume. The geometry-learning burden disappears."* This subsumes both old PARA (factorized single-view binding) and new PARA (joint multi-view binding) — view count is now an implementation detail, not a story. The Jun-3 single-view headline lock is retired. Wired into `pitch.md` + `vault/para/goals.md` + `vault/fleet/agents/project_highlevel/memory.md`, old framings preserved under OLD markers.

## What we did (today, concrete)
- **First deploy** of the new multi-view volumetric model
- Collected **more + better** data with the deployed model in the loop
- Making figures while training continues in background
- Re-running the same eval suite as before so cross-arch comparison is apples-to-apples

## Next steps
- 3 trainings queued:
  1. **cup_pluswrist_moredata** (50 ep, GPU 0) — running
  2. **cup_pluswrist_25ep** (25 ep symlink, GPU 1) — running, **data-efficiency probe**
  3. **towel_fold_basic_wrist** (30 ep) — auto-launches when (1) or (2) finishes
- Watch: does the global residual fallback become necessary on rot? If yes, that's an honest scaling story; if no, the per-voxel-only arch holds.
- Family trip in 10 days — these three runs are the cool-result-for-family candidates.