# YAM — overview

> **Fresh / post-compaction agent?** Read
> [`yam_agent_on_startup.md`](yam_agent_on_startup.md) first — it's
> the minimum context-restoration checklist. Then
> [`FAQ.md`](FAQ.md) for copy-paste commands. Then come back here for
> state + tree layout.

YAM bimanual rig at TRI (russet workstation). Two i2rt YAM arms with linear_4310
grippers, a ZED 2i scene camera (HD1080) mounted on the exo-base aruco
calibration board, and one ZED Mini wrist camera per arm (HD720). Data
collection, training, and inference for the PARA "place mug" task live here.

## Canonical code home (rehaul landed 2026-06-02)

All YAM code lives at **`/data/cameron/para/robot/yam/`** on the lab
server. Puget and russet sshfs-mount it as `~/lab/para/robot/yam/`.
**Single source of truth** — never edit or scp to puget/russet.

```
/data/cameron/para/robot/yam/   ← mounted as ~/lab/para/robot/yam/
├── lib/
│   ├── robot.py             ← Robot (raiden-direct, implemented)
│   ├── smooth_move.py       ← smooth_move(robot, target, vel)  (implemented)
│   ├── policy_client.py     ← chiral remote variant (TODO)
│   ├── calibration.py       ← exo aruco solve + T_lfr  (implemented)
│   ├── render.py            ← mujoco exo+arm render + warp  (implemented)
│   ├── viz/
│   │   ├── overlay.py       ← mask-aware blend  (implemented)
│   │   └── (feature_pca, heatmap, keypoints, trajectory).py  ← TODO
│   └── data_io.py           ← PKL conventions  (TODO)
├── unit_tests/
│   ├── control_speed_test.py  ← 2-pose .npz, parallel smooth_move, timing
│   └── calibration_test.py    ← offline aruco + render + composite PNG
├── bin/
│   ├── yam-mount             ← sshfs /data/cameron → ~/lab
│   └── yam-rd-serve          ← rd serve in a respawn loop
├── raiden_fork/              ← TRI's robot tooling (source + meshes only)
├── code/                     ← legacy / pre-refactor (deploy_yam_2view.py etc)
├── yam_control/              ← legacy (record_skip_wrist.py etc)
├── calibrate.py              ← launcher (implemented, live + rerun)
├── record.py / train.py / deploy.py / convert.py  ← TODO launchers
└── rot_kmeans_K64*.npz
```

**Large files stay on puget** (per GUIDELINES "Lab is for scripts"):
`~/yam_para/data/`, `~/yam_para/checkpoints/`, `~/yam_para/dinov3/weights/`,
`~/yam_para/eval_videos/`, `raiden_fork/weights/`. Reference them via env
vars (`DINO_WEIGHTS_PATH=$HOME/yam_para/dinov3/weights/...`).

## Current state (2026-06-02)

- **Task**: right-arm-only "place mug" (single-arm mode on bimanual rig).
- **Production architecture**: DinoVolumeQueryBimanual with scene + wrist
  fusion. Cross-attention between scene patch grid and wrist patch grid
  (`n_xattn_layers=2`), scene-CLS ⊕ wrist-CLS at the input projection,
  factorized YX × Z × T volume head. Wrist abstain via aug-token
  (`F_oof_token`).
- **Current ckpt**: `yam_wrist_xattn_v1/latest.pth` (12h training run, epoch
  138/200, ~66k steps when last checked). Trained on
  `~/yam_para/data/place_mug_wrist_cam/` (puget).
- **Deploy script**: `deploy_yam_2view.py` (2200 lines, being refactored).
  Fusion modes: `sum` / `max` / `scene_only_decode`.

## What's running right now

- `agents:5` (puget) — `deploy_yam_2view.py` at the user prompt. Uses
  `--scene_only_decode 1 --max_joint_vel_rad_s 0.3 --first_move_vel_rad_s 0.2`.
- `agents:6` (russet via `robot-lab`) — `rd serve --action-type joint
  --resize-images 1080x1920`. Must be restarted between deploys (client
  disconnect triggers estop → shutdown).
- Background training has finished its 12h budget.

## See also

- [`yam_agent_on_startup.md`](yam_agent_on_startup.md) — **read this first** on a fresh / post-compaction session
- [`FAQ.md`](FAQ.md) — copy-paste commands + error/fix table
- [`tasks.md`](tasks.md) — active workstreams
- [`memory.md`](memory.md) — conventions and code pointers
- [`INDEX.md`](INDEX.md) — canonical scripts + lab paths
- [`control.md`](control.md) — chiral protocol, smooth_move_to, teleop
- [`calibration.md`](calibration.md) — exo aruco, wrist verify, HD1080 patch
- [`recording.md`](recording.md) — data collection procedure
- [`conversion.md`](conversion.md) — SVO2 → PNG + PKL via `rd convert`
- [`rendering.md`](rendering.md) — exo+arm mujoco render + warp
- [`training.md`](training.md) — architecture, wandb, bg launch
- [`inference.md`](inference.md) — deploy, fusion modes
- [`rerun-panels.md`](rerun-panels.md) — what we log + how to view
- [`known-bugs.md`](known-bugs.md) — chiral lock, estop, ZED lockup
- [`raiden.md`](raiden.md) — TRI's robot tooling