---
name: Dataset upload always includes overlay bake + viewer registration
description: When uploading a new smith300/UMI dataset from the Mac to the server, always also bake the rgb_overlay/ subdir with future-keypoints + gripper sparkline viz and register both raw + _overlay entries in the data viewer's datasets.json under mac_teleop.
type: feedback
originSessionId: 531aff41-3330-4729-8ba1-09499450783f
---
When the user asks to "upload the latest dataset" (or otherwise pulls a new `dataset_<TS>/` from `/home/cameronsmith/mnt/mac/smith300_para_stuff/scratch/datasets/` to `/data/cameron/mac_robot_datasets/`), do **all three** of these steps in the same turn unless the user says otherwise:

1. **Copy the raw dataset** to `/data/cameron/mac_robot_datasets/<name>/` (timestamp dir name is fine; the user will rename via custom id if they want).

2. **Bake overlays** by running:
   ```
   cd /home/cameronsmith/mnt/mac/smith300_para_stuff && \
     MUJOCO_GL=egl python3 scratch/render_dataset_overlay.py \
       /data/cameron/mac_robot_datasets/<name>
   ```
   The baker (`scratch/render_dataset_overlay.py`) auto-detects smith300 vs UMI from `meta.json` and by default draws: alpha-blended robot/UMI mesh, EEF crosshair + axis triad, top-right `gripper value: X` HUD with a yellow line plot of the next 30 gripper values, and a yellow fading polyline through the next 20 EEF positions ("future-keypoints"). Output writes to `<name>/rgb_overlay/`. Defaults are `--future-keypoints 20 --future-gripper-plot 30` — keep them on; that combination is the one Cameron explicitly liked. Runs server-side via `MUJOCO_GL=egl` (~6 fps for smith300, ~7 fps for UMI; ~2-3 min for an 800-frame session).

3. **Register both entries** in `/data/cameron/para/.agents/reports/data_viewer/datasets.json`:
   - Raw entry: id `<name>`, type `frames_flat`, path `mac_robot_datasets/<name>`, frame_ext `jpg`, fps 2.5.
   - Overlay sister entry: id `<name>_overlay`, same type/fps, path `mac_robot_datasets/<name>/rgb_overlay`. Subtitle should mention "mesh + EEF axes + future-keypoints + gripper sparkline" so it's distinguishable in the viewer tab list.
   - Append BOTH ids to the `mac_teleop` experiment's `"datasets": [...]` list (raw first, then `_overlay`).

**Why:** Cameron annotates episodes/keyframes from the viewer; without the overlay variant he can't see whether the robot mesh + EEF + future trajectory line up correctly, so a raw-only registration makes the viewer ~useless for the actual workflow. Skipping the bake forces him to ask for it explicitly every time. He's said the previous overlays with the future-keypoints polyline + gripper sparkline "were great" — that's the standard now, not optional.

**How to apply:** Trigger phrases include "upload the latest dataset", "pull this dataset to the server", "I have a new dataset", or "can you copy <dataset> over". After the cp completes, kick off the bake (use `run_in_background: true` for any 800+ frame dataset since it takes minutes), then update `datasets.json` for both entries while the bake runs. Notify the data_visualizer agent only if the agent owns datasets.json edits in your current setup — Cameron's pattern is to let me edit `datasets.json` directly and only loop in data_visualizer for novel viewer features (annotation tools etc.), not for routine dataset registration.

If the user gave the dataset a custom id (e.g. `robot_pick_cup`), use that as both the directory name on the server AND the entry id; otherwise keep the timestamp form.