# yams outbox

## 2026-05-28 (day 3) — status & fiducial-cal scoping

### Priorities acknowledged (from project_highlevel)
1. Great YAM control stack — in progress, near-complete.
2. Wire fiducial-board calibration **before** the 80-demo recording session (better sequencing than the week-2 default in the plan doc).
3. v7 right-arm projection debug — paused.

### Control stack — current state
- **`~/yam_para/code/deploy_yam.py`** on puget: chiral client, posture-regularised mink IK (`pos_cost=1, ori_cost=0.1, posture_cost=5e-2`, daqp + ConfigurationLimit — matches smith300 weights with bumped posture), `--action_type joint` default so puget does IK locally and the server skips its naive IK. Receding-horizon chunking + linear ramp from current proprio to first cmd in ≤0.15 rad sub-steps so the server's joint-delta safety doesn't trip.
- **`deploy_one_joint_state.py`** on puget: deploy one recorded `joints` (14-D) from a `cube_in_carton/<ep>/lowdim/<frame>.pkl` via the same ramp logic. No model, no IK on our side — proves the streaming path on its own.
- **Rerun viz**: 504² model input + 1280×720 display upscale; live RGB + start-pix + per-arm rainbow predicted EE trajectory + DINO PCA + gripper/rotation piano rolls + bimanual silhouette + **per-step predicted-joint-state grid** (silhouette + raw mujoco RGB, both at display/4 res, IK'd per timestep).
- **Diagnostic**: `--use_train_cam` to fall back to training cam pose if drift is suspected (snapshot mode also uses training K to avoid non-square-pixel mujoco issues when server resizes to non-aspect-preserving shapes).
- **Snapshot replay**: `--snapshot_dir` mode loads a `snapshot_obs.py` capture and re-runs inference offline. Used yesterday to repro a bad-IK case at t=3 and tune the posture cost (1e-3 → 5e-2 cleared elbow-flips).
- **Fork**: `~/cameron/raiden_fork/` on russet AND yukon — both are SSHFS mounts of `puget:~/yam_para/raiden_fork/` (single source of truth on puget). Branch `cameron-deploy-experiments`. Each yam station has its own ON-MACHINE venv at `~/cameron/raiden_fork.venv/` (wheels stay local; only source goes over sshfs). Same pattern for `~/cameron/yam_control/` → `puget:~/yam_para/yam_control/`. Old local copies preserved at `~/cameron/raiden_fork.local_backup/` and `~/cameron/yam_control.local_backup/` on russet (none existed on yukon). Edits to puget's tree propagate immediately to both yams. SSH keys: russet's `id_ed25519` (comment `sergey.zakharov@tri.global`) and yukon's `id_ed25519` (comment `yukon-robot-lab`) are both installed in puget's `~/.ssh/authorized_keys`.

### Fiducial cal — existing infrastructure inventory

The fiducial-exoskeleton calibration framework already lives in `/data/cameron/para/panda_streaming/`:

| File | What it does |
|---|---|
| `ExoConfigs/exoskeleton.py` | Base `ExoskeletonConfig` + `LinkConfig` dataclasses (arm XML + per-link board-mount offsets) |
| `ExoConfigs/panda_exo_handeye.py` | Concrete config: Panda + 4×4 even_larger ArUco board mounted on the hand at `aruco_offset_pos=[40, 30, 90]` mm |
| `ExoConfigs/panda_exo.py` | Defines `link_boards` dict — board image paths, `BOARD_LENGTH_*` constants, ArUco dict (`DICT_6X6_250`) |
| `hand_eye_calib/calibrate.py` | Drives 17 calib poses through Panda FK, captures synthetic ArUco detections from a posed MuJoCo render, nonlinear-solves for `T_cam_world` + `T_hand_board` |
| `hand_eye_calib/render_test.py` | Render-from-camera sanity check (the `mujoco_overlay_check.png` we already grepped) |
| `exo_utils.py` | `position_exoskeleton_meshes`, `get_link_poses_from_robot`, `do_est_aruco_pose` — the core fiducial detection + FK plumbing |

**For YAM there is NO yam-specific config yet** — need to write `yam_exo_handeye.py` analogous to the Panda one. The calibrate.py harness should work unchanged if we hand it a YAM config.

### What I'm doing today
1. (now) Write `yam_exo_handeye.py` (YAM URDF + board mount offset placeholder; iterate on the offset based on the actual physical board Cameron printed).
2. Adapt `calibrate.py` for YAM: 17-ish calib joint poses (in YAM's joint convention, not Panda's), confirm the optimization converges.
3. Document the output format so we can patch it into `calibration_results.json` between `rd record_calibration_poses` and `rd record` — keeps raiden's pipeline intact, just swaps the cam extrinsics for the better fiducial-derived ones.
4. Once Cameron's printed board is on hand, measure actual mount offsets and update the config.

### Questions for Cameron when convenient
- Which ArUco preset is the printed board? (`even_larger` 140×140mm, or something different?) → determines `BOARD_LENGTH_*` and marker IDs.
- Where on the YAM gripper does the board mount? (Approximate `aruco_offset_pos` in mm from the `grasp_site` body — I can iterate on this from a single calibration image once we have one.)
- Is the goal: calibrate per recording session (run before each session, like the current ChArUco flow), or calibrate once and reuse extrinsics indefinitely? (affects how we patch `calibration_results.json`.)