# PARA — active tasks

Strategic-level workstreams, 3–6 items. Not a TaskList dump.

## Active

1. **TRI Phase 1 OOD experiment (cup → mug pick-place, left/right hemisphere split)**
   - Owner: Cameron + yams (data collection) + me (experiment design)
   - State: 2026-06-02 — recollecting demos today with paper-ready setup, fixed container at center, left-half-only training. Eval on right half later this week.
   - Soft gate: PARA ≥ 30 points better than ACT at held-out positions.
   - Doc: `/data/cameron/para/notes/tri_internship_plan.md` Phase 1.

2. **Backbone ablation table** (DINOv2 vs DINOv3 sizes vs non-DINO; same head)
   - Owner: backbones (runs remotely on lab GPUs)
   - State: queued — scheduled for Wk 3 (Jun 9–15) per the internship plan
   - Purpose: strengthens "head > backbone" claim already won at smith300; produces fig 5 / paragraph for paper abstract.

3. **Phase 4 lead-task BAKE-OFF (revised plan 2026-06-14)** — try multiple, converge by Tue night
   - Owner: Cameron + me
   - **Strategy shift**: instead of locking one task tonight, run a bake-off across the shortlist Mon–Wed. Training is the cheap part (~few hours per task); the expensive question is "which task gives feasible teleop + good data" — answer that empirically rather than guess.
   - **Primary bake-off duo — LOCKED 2026-06-14 evening, 25 demos each, models training**:
     - **Teapot** — take off lid → put tea bag in teapot → close lid. 3 substeps.
     - **Egg-find-and-replace** — (operator removes a random egg and sets it on the table) → robot finds the displaced egg → picks it → places it in the empty slot in the carton. **Context-conditioned spatial reasoning task** — model has to read carton state from features to know which slot is empty. Much stronger than naive "place in carton" — the empty-slot reasoning is itself a visual-reasoning claim.
   - **OOD eval axes for egg task** (clean breakdown): (a) novel egg position on table, (b) novel empty-slot configuration in carton, (c) both. Per-axis breakdown writes a strong fig.
   - **Secondary candidates if time / for teasers**:
     - drawer-place-close, sort-by-category, water-wiping
   - **Post-vacation ambition (NOT this week)**:
     - **Keurig long-horizon** (6 steps: pick cup → place under spout → pick pod → open lever → place pod → close lever → press start). The headline money-shot. Needs 3+ weeks of iteration. Start Jun 23 after vacation. Pre-vacation prep: ask current Keurig owner for borrow permission Mon-Wed; take photos of setup for vacation viz brainstorming.
   - Towel folding parked (YAM teleop can't do cloth precisely enough). set-the-table parked (lower priority than the duo above).
   - **Day plan**:
     - Sun Jun 14: prop-scoping at office (reminder set 9:47am). Identify which props are physically available.
     - Mon Jun 15: v2.1 eval first. Then teleop-test 3-4 tasks for feasibility. Collect data for the 2-3 smoothest. Kick off training in parallel (cheap).
     - Tue Jun 16: collection + training continues. **Converge to ONE headline task + 1-2 secondaries by end of day.** Don't let bake-off become "5 mediocre tasks."
     - Wed Jun 17: finalize collection, queue background runs for vacation, pre-trip checklist.
   - **Narrative bonus**: a bake-off where 2-3 tasks work on the *same model* with few demos each = Phase 5's "teaser tasks on the same model" story landing early. If 3 tasks work at 8-15 demos each, the data-efficiency sticky number gets even stronger ("3 different tasks, ~10 demos each").
   - **Risk to manage**: cognitive overhead of 3+ parallel experiments. Counter: kill any task that's clearly not working by Tue noon, don't drag dead branches into vacation.

4. **Vacation block — Thu Jun 18 → Mon Jun 22 (~5 days at home, GPU-only remote)**
   - Owner: Cameron (solo, remote via SSH chain school → mac → robot-lab)
   - Three workstreams, all non-physical-robot:
     - **YCB object simulation grasp training in MuJoCo** — sim2real story setup. Train a PARA pick-place policy on YCB objects in sim; prep for zero-shot real deploy when back at TRI. The "vibecode in half a day" path Cameron sketched 2026-06-11. Pair with a real2sim eval if time allows (2×2 table is paper-stronger than diagonal-only).
     - **Website setup for the paper** — V2 with new YAM evidence + binding-framing narrative. Lives under `para/.agents/reports/project_site/`.
     - **Visualizations for the paper** — overview figure tightenings (binding-arrow emphasis, step-2 rename to "Bind...", grip/rot acknowledgment, label cleanup), plus any new figures the canonical arch needs.
   - Reminder: push notification scheduled for Thu Jun 18 morning.
   - Hard constraint: no physical robot access. Don't plan data collection or live deploy.
   - Soft constraint: it's a vacation. 2–4 productive hours/day is realistic, not 8.
   - Pre-trip checklist: SSH chain verified working from home WiFi BEFORE leaving Wed Jun 17.

4. **Money-shot video production**
   - Owner: yams (recording) + figure_maker (cuts)
   - State: structure locked 2026-05-12 — 45-sec, live-camera-move with heatmap overlay is the signature visual
   - Material: starting to accumulate this week from OOD evals

5. **Fig1 right-panel YAM data swap**
   - Owner: figure_maker
   - State: pending — needs cup-task OOD numbers + stills with heatmap overlay
   - Will trigger when Phase 1 OOD results land (~end of TRI Wk 1)

6. **FidExoskeleton paper V2** (Week 11 — Aug 4–10)
   - Owner: Cameron + me (not yams)
   - Scope-capped at 1 day
   - State: scheduled. Capture Raiden-vs-FidExo calibration residuals NOW while infra is hot.

## Recently completed

- 2026-05-30: Wrist cam calibration on YAM (residual a few px, acceptable)
- 2026-05-29: First 15-demo mug pick-place on YAM, 10/10 in-dist
- 2026-05-19: PARA architecture locked (dino_kv + sin/sin) — backbones leaderboard finalised
- 2026-05-12: fig1 v3 right-panel shipped (placeholders), lab presentation deck (22 slides) locked

## Blocked

(none currently)

## Watch list — under evaluation, not yet committed

(empty — scope cut 2026-06-16 moved everything here to Parked.)

## Vacation weekend task — detailed method diagram (logged 2026-06-17 PM)

**Deliverable**: a more detailed architecture-flow figure than the current `para_overview_comparison`. The current diagram is the high-level comparison; this one is the technical architecture detail for the paper's method section.

**Must include**:
- **Rotation + gripper argmax prediction heads** (explicitly shown — current figure ends at EEF XYZ)
- **Volume pre-fusion AND post-fusion with concatenation** (the architecture flow showing how features get combined across views, before and after MLP fusion)
- **Feature shapes at each stage** — DINO patch resolution (likely 14×14 or 16×16 for ViT-S/16+), the upsampling CNN that changes resolution to the working voxel-grid resolution (P=128), per-stage tensor shapes

**Reuse starting point**: an OLD INKSCAPE FILE in scratch_files that already has detailed pre-fusion / post-fusion + concatenation diagrams. Probable path: `/data/cameron/scratch_files/para_overview_inkscape_draft.svg` (28MB, dated 2026-05-14). Find this file and reuse as the starting point — don't redraw from scratch.

**Timing**: weekend work (during vacation Thu Jun 18 → Mon Jun 22). NOT before vacation. Pre-vacation today stays focused on collector instructions + space-mouse check + pre-trip checklist.

## Parked / deferred

- **MAJOR SCOPE CUT 2026-06-16 (Cameron's call)**: pulled all "nice-to-haves" out of active scope. The paper is core-experiments-only. Sergey validated the paper as in good shape; further scope was becoming drag, not value. Discussion: the nice-to-haves were exciting to advisors but not to Cameron; the bottleneck isn't more extensions, it's nailing core excellently. Also: half the nice-to-haves required hacky workarounds on hardware Cameron doesn't control (extrinsics recovery on Sergey's 1k-hours, sim infrastructure on YAM gripper Cameron didn't design) — the right answer is building own-robot for hardware/data control post-paper, not hacking around the gap pre-paper.
- **Video model as inverse dynamics** — parked. Strong prior result (SVD 90 vs 0 on same features) preserved as a paper future-work paragraph; no active work.
- **Sim2real (incl. real-to-sim re-render)** — parked. The MuJoCo + YCB re-render approach is sound but not the bottleneck. Vacation slot freed for figures + website V2 only. **Reasons Cameron enumerated 2026-06-18** for not pulling sim work back in despite vacation-idle temptation: (a) Arsh should be picking it up soon — delegation candidate, not Cameron's track; (b) gripper coordinate transforms are blocked on the person who designed the YAM gripper (shared-hardware "no stack control" pattern again); (c) the sim2real result wouldn't be that informative — sim training is limited to pick-and-place anyway, doesn't stress the dexterous claims that matter for the paper. **If urge resurfaces during vacation**: re-read this entry, return to paper drafts / method diagram / Onshape instead.
- **1k-hours YAM dataset pretraining** — parked. Calibration recovery is hacky on hardware Cameron doesn't control. If pretraining matters, do it on own robot post-paper.
- **Panda calibration → DROID pretraining (or Panda → YAM transfer)** — surfaced 2026-06-17 PM, parked. Idea: calibrate the Panda cameras at TRI, then either (a) train on DROID large-scale directly (the data already exists, well-calibrated), or (b) use Panda as a cross-embodiment transfer / pretraining source for YAMs. NOT priority per Cameron — capture as a post-paper idea. Could become live if (a) someone else picks it up as a delegated extension, or (b) the core paper ships early and there's bandwidth.
- **Skilled-teleop recruitment for 500 demos** — STAYS ACTIVE (this is more core data, not an extension). Cameron messaged teleop people 2026-06-16.
- **UMI-style data collection — properly parked again 2026-06-14**: empirically validated as the right *future* direction (leader-follower teleop has a precision floor that wiping/sealing/threading tasks live below; Cameron noticed this hands-on at the carton-lid step). Out of scope for *this* paper — but should appear as a *limitations + future work* paragraph in the paper acknowledging the precision floor. UMI cross-embodiment as a *paper contribution* remains dropped (didn't generalize on May 11). The distinction: UMI for *better data collection* is the long-term direction; UMI for *cross-embodiment claim* is dead.
- Right-arm `T_left_from_right` convention debug (yams paused — not load-bearing)
- Custom arm v2 rebuild + Onshape CAD work (post-paper, see `arm_v2_plan.md`)

## Internship back-half plan (locked 2026-06-17, refined evening of 2026-06-17)

**Sequenced phases, single-arm → bimanual → scale-up**:

| Window | Focus | Owner |
|---|---|---|
| **Now (Jun 17) → ~Jul 1** | Core experiments single-arm. Baselines, visualizations, paper-compelling. | Cameron |
| **Pre-vacation (Wed Jun 17) + Tue Jun 23 session** | Pipeline derisk with skilled-teleop collectors. **Key constraint surfaced 2026-06-17 PM**: collectors are NOT programmers. Onboarding requires in-person Tuesday session, not async Loom-and-go. Pre-vacation prep: (a) write copy-paste-friendly non-programmer instructions (exact dirs, exact commands, no dev jargon), (b) verify space-mouse teleop still works (collectors prefer it). Tue Jun 23: in-person onboarding + first real collection session with Cameron present. First task: egg-find-and-replace ~25 demos (extends Cameron's 25 → 50). | Cameron (prep + present Tue) + collectors (collect under guidance) |
| **Tue Jun 23 → Fri Jun 26 — HARD EVAL WEEK** | Cameron's lock 2026-06-20: paper-quality evals only. Baselines already trained, just need to run. Goal by Fri: results that go into Fig 4 (in-dist + OOD obj pos + OOD viewpoint, with/without finetune). Discipline: no new training, no new tasks, no new ideas this week. Just evals + compile + visualize. | Cameron |
| **Mon Jun 29 onwards — ANNIE COLLABORATION HARD MODE** | Cameron's lock 2026-06-20: go hard with Annie on (a) rebuild robot arm in CAD, (b) UMI interface for the YAMs (easier, first), (c) UMI interface for the rebuilt arm (harder, second). "Have fun with it." | Cameron + Annie |
| **~Jul 1 → mid-Jul** | Add bimanual arm, integrate into canonical arch, small architectural tweaks if needed. Derisk only — not much there per Cameron's read. **Bandwidth note (2026-06-20)**: with Annie hard mode starting Jun 29, bimanual integration may shift to mid-July primary focus (sequenced after the first Annie sprint) rather than Jul 1. Don't try to go hard on both Annie AND bimanual simultaneously. | Cameron |
| **~mid-Jul → Aug 17** | Bimanual scale-up with skilled-teleop team + final presentation polish. | Cameron + Sergey's teleop team |
| **In parallel throughout** | Nice-to-haves (video IDM, sim2real, pretraining) — delegated to other interns / team members if interested. **NOT Cameron's priority.** | Others (if they want them) |
| **In parallel daily, starts now** | 1 hr/day Onshape → own-robot REDO ramps in alongside back-half work. | Cameron, discipline track |

**Key constraints**:
- Single-arm core is non-negotiable as the first ~2 weeks
- Scale-up only happens on the FINAL config (bimanual) — never scale a fragile foundation, especially with someone else's time
- Nice-to-haves only get done if SOMEONE ELSE is interested — Cameron does not commit time to them
- Own-robot REDO progresses via the 1hr/day track without competing with the core work

**Updated framing for Sergey**: scale-up is mid-July on bimanual, not this week. Cameron will send the sequenced batch plan closer to mid-July. In the meantime, Cameron can draft a "nice-to-haves" list if Sergey's team has interns who want to pick up extensions.

**Onshape 1hr/day discipline (starts now) — now upgraded to pair work with Annie starting next week** — 1 hour of every 10-hour workday. Pre-vacation: solo Onshape skill-building. Starting week of Jun 22 (Annie arrives): 1hr/day pair work with Annie (hardware intern, "free to help on projects" per Cameron 2026-06-18). Annie fabricates + integrates while Cameron designs. Goal: minimum-viable own-robot + UMI-style data collection device, "as easily and simply as possible." First-demo milestone: 10 UMI demos recorded within 3 weeks of Annie's start, else scope needs to come down further. Discipline guard: this is the *bounded* version of the prior Onshape antipattern (1hr/day pair, minimum-viable framing, ≠ 2-month side project).

**Minimum-viable spec for the first 2-3 weeks with Annie**:
- Arm: v1 (already calibrated Apr 29) + minor cleanup, NOT full v2 redesign
- Wrist cam mount: 3D-printed bracket on v1's EE
- UMI device: 3D-printed handheld shell + phone or GoPro camera
- Pose recording: ArUco fiducials + existing scene cam (no SLAM/IMU)
- First milestone: 10 usable UMI demos recorded

*Note*: this is structurally different from the prior "Onshape during paper crunch" antipattern (see [[project_highlevel_memory]]). Antipattern was bundling a 2-month rebuild into a half-day task slot. This is a *bounded skill-building discipline* (1hr/day) in the post-Sergey-validation window, with the actual rebuild scoped for the back-half. Allowed.

**Own-robot REDO plan** (~Jul 1 onwards, in parallel with extras + paper polish):
- Print in parallel (no full pause for fab)
- Design custom UMI gripper
- Buy parts
- Wrist camera + custom UMI camera integrated
- Goal: own robot in the paper's experiments (or at minimum, in the final TRI presentation)

**Extras / scaling-up that come back in the back half** (the previously-parked items):
- Sim2real (now meaningful because own-robot has known sim)
- Video model as IDM
- Skilled-teleop scaling (continues active throughout)
- Possibly: 1k-hours pretraining, but only if extrinsics recovery is feasible

**Why this works**: in the back half, own-robot eliminates the "hacky workaround" problem with the parked nice-to-haves. With own gripper + own sim + own calibration, sim2real becomes principled, pretraining becomes clean, etc. Cameron's instinct from 2026-06-16: *"every shared-hardware nice-to-have turns into a hacky workaround because I don't control the stack."* Building own-robot fixes the root cause.

## Post-paper north star (2026-06-16)

**Build own robot.** The pattern from this internship: every shared-hardware "nice-to-have" turns into a hacky workaround because Cameron doesn't control gripper / sim / extrinsics. The right answer isn't better hacks — it's owning the stack. Post-paper move: build own robot to get clean control over hardware + data + sim. THEN scale all the parked nice-to-haves cleanly. (As of 2026-06-17 own-robot rebuild moves INTO the internship back-half via the 1hr/day Onshape track + Jul 1 sprint.)
- Diffusion Policy as 2nd baseline (nice-to-have, open)