# Always visualize

This is the single most important rule in this repo. Every time you
finish a step — calibration, dataset parsing, IK, training, eval —
**you save an image or a video** and **look at it** before reporting the
step as done.

## Why this matters here specifically

Real-robot work fails in ways that **look fine in the numbers** and
**look obviously broken on the screen**:

- Calibration converges to "low error" but the camera pose is off by
  90° in some axis convention you forgot to convert. → MuJoCo overlay
  ends up sideways. Numbers don't catch it; eyes do in two seconds.
- IK reports "succeeded" but ends in a mirrored configuration. → robot
  flips through itself in render. The position error is fine; the
  trajectory isn't.
- Training loss goes down but the heatmap argmax sits in a corner of
  the image because the start-keypoint embedding is unmasked or the
  pixel coordinates are flipped. → the wandb pixel-error metric is
  small because most points happen to fall there in the val set.
- Pre-cached 448×448 images get resized with a different intrinsics
  scaling than the dataloader assumes. → projected EEF pixel is shifted
  by a few px everywhere. PARA still trains; deployment still misses.

In all of these, a single PNG with the relevant overlay would have made
the bug obvious.

## What "visualize" looks like at each stage

| Stage | What to render |
|---|---|
| Hand-eye calibration | ArUco reprojected on the calibration image; MuJoCo render from the calibrated camera vs the real frame. Save both. |
| Camera intrinsics check | Project the robot link origins into the image; they should sit on the corresponding link. |
| Dataset parsing | Per-episode strip: every Nth frame as a tile, with EEF keypoint + ground projection + height. |
| Mask overlay | RGB with the MuJoCo robot silhouette alpha-blended on top, across multiple poses. |
| IK reproduction | Side-by-side: recorded joints vs IK-recovered joints, both rendered + projected. Diff map. |
| Training | wandb `vis/train_strip` and `vis/val_strip` per-timestep tiles with heatmap + pred + GT. |
| Deployment | Record the rollout from the camera. Save the MP4. |

## Concrete patterns

**Cameron's go-to overlay** (already used in `train_panda_para.py`):

```python
# White circle on EEF pixel
cv2.circle(vis, (u, v), 6, (255, 255, 255), -1)
# Cyan ring on ground projection (z=0)
cv2.circle(vis, (ug, vg), 6, (0, 255, 255), 2)
# Yellow line for the height drop
cv2.line(vis, (u, v), (ug, vg), (255, 255, 0), 2)
# Height label
cv2.putText(vis, f"h={z:.3f}", (ug + 8, vg + 12), ...)
```

**Heatmap overlay** (red channel only so the underlying image stays
visible):

```python
heat_rgb = np.zeros_like(frame); heat_rgb[..., 0] = heat_normalized
vis = np.clip(frame * 0.55 + heat_rgb * 0.45, 0, 1)
```

**MuJoCo silhouette** (cheap mask: render with all colors → threshold by
non-background pixels, or use `mujoco.mjv_renderer` segmentation mode):

```python
renderer.update_scene(data, camera=cam_id, scene_option=opt)
seg = renderer.render(...)  # use mjr_setBuffer / segmentation if available
mask = seg.sum(axis=-1) > 0
overlay = rgb.copy()
overlay[mask] = (0.6 * overlay[mask] + 0.4 * np.array([255, 100, 100])).astype(np.uint8)
```

## Where to save

- One-off debug images: `panda_streaming/checkpoints/stageN_*.png`.
- wandb-ready vis: log via `wandb.Image` like `train_panda_para.build_wandb_strip`.
- Mid-run: drop a `media/` folder under your active dataset and put the
  MP4 there with a clear name.

## And before you report

If the agent harness says "I've done X" but there's no image link in the
report, the step is **not** done. Re-render, re-attach, re-report.