# Inbox ## 2026-05-14 — from figure_maker: figure reordering + replace `fig:method` Cameron uploaded two new method-figure PDFs and wants the figure layout reshuffled. ### Files staged at `paper/figs/` | File | Size (pt) | Aspect | |---|---|---| | `paper/figs/method_perpixel.pdf` | 595 × 108 | wide strip, page-width thin | | `paper/figs/method_height.pdf` | 289 × 295 | ~square, fits half-column wrapfig | Both originals are at `/data/cameron/scratch_files/para_method_manual_perpixel.pdf` and `/data/cameron/scratch_files/para_method_height.pdf` if you need to re-stage. ### What Cameron asked for 1. **Old Fig 3 → new Fig 2, top of page 2.** The "Real robot results" figure (currently `\begin{figure}[t!] \label{fig:real_results}`, line ~198 of `main.tex`) gets promoted to second position. Place it with `[t!]` near the top of section text that lands on page 2 (probably the Real Robot Results section). 2. **New per-pixel method figure: full width, top of page 3.** Use `method_perpixel.pdf` at `\linewidth` in a `\begin{figure}[t!]` placed so it floats to the top of page 3. The PDF is already designed as a single-column-width strip (595pt wide × 108pt tall). 3. **New height-vs-depth figure: half width, page 3, text-wrapped.** Use `method_height.pdf` inside a `\begin{wrapfigure}{r}{0.5\linewidth}` (or `{l}` if it reads better on the right page). The PDF is roughly square (~289×295pt), so half column-width is the natural fit. Place it inside the page-3 text body so the paragraph wraps around it. ### Implied numbering / labeling Current order (after swap, expected): 1. `fig:overview` — page 1 hero (unchanged) 2. `fig:real_results` — was Fig 3, now Fig 2, top of page 2 3. `fig:method_perpixel` — NEW, full-width, top of page 3 *(suggest splitting the old `fig:method` label into two)* 4. `fig:method_height` — NEW, wrapfig on page 3 5. `fig:ood` — was Fig 4, now Fig 5 6. `fig:video` — was Fig 5, now Fig 6 7. `fig:pretrain` — was Fig 6, now Fig 7 The old combined `\label{fig:method}` figure (lines ~146-151, "PARA method" with sub-panels a/b) is **replaced** by the two new figures above. Any `\ref{fig:method}` in body text needs to be re-pointed — probably to `fig:method_perpixel` (since that's the dominant method panel) or split per context. Grep `main.tex` for `fig:method` to find all callsites. ### Captions (suggested, edit as you see fit) - **Fig 3 — per-pixel method (full width):** *"\textbf{PARA method.} A DINOv2 backbone produces spatial features; a 1×1 conv head predicts a 2D heatmap stacked with per-pixel height-bin logits to form a full 3D volume. The 3D argmax yields the end-effector target in camera coordinates, transformed to the robot frame via known camera extrinsics."* - **Fig 4 — height vs depth (wrapfig):** *"\textbf{Height is view-invariant.} For the same physical target, depth changes with camera position (0.38\,m vs.\ 0.50\,m), but height above the table remains constant (0.10\,m). PARA predicts height, making the lifting step camera-invariant."* These are recycled from the old `fig:method` caption (line 150) so they should fit the body text without re-narration. ### Notes / caveats - The wrapfigure on page 3 will interact with whatever paragraph happens to be there. Once you place it, check the build — wrapfigures are notorious for nudging text around. You may need to tweak the `\begin{wrapfigure}{r}{0.5\linewidth}` value (try 0.45 or 0.48 if it crowds the body) or insert a small `\vspace{-...em}` to align with the paragraph top. - I have NOT verified the visual contents of the two PDFs — I'm working off filenames + dimensions only. If you open them and they don't match the captions, let me know and I'll inspect. - The old `fig2_method.svg` / `fig2a_method.svg` / `fig2b_invariance.svg` builders in `/data/cameron/penpot/` can be considered superseded for paper purposes (still useful for the website / talk). --- ## Task (normal) — 2026-06-23 05:03:39 Two figure swaps for the paper: 1. REPLACE Figure 1 (currently `eef_bind_comparison.pdf` at main.tex:46, the binding-pitch overview) with the results_v2 figure. The new Figure 1 should be: - Image: `figs/generated/results_v2.png` (already in the repo, 1988x3000) - Caption: keep the existing real-robot caption pattern (in-distribution + OOD pick-and-place) — see the current Figure 2 caption at main.tex:62 for inspiration. Reword to fit it being the lead figure now. 2. INSERT a new figure AFTER the method-details figure (currently Figure 3 at main.tex:106-119). This becomes the new Figure 4. Image to use: - Source path: `/data/cameron/para/robot/yam/figure_dumps/three_task_4panel_grid.png` - Please copy into the paper figs dir, e.g. `figs/generated/three_task_4panel_grid.png`, so the source-of-truth is committed alongside the .tex. - It is a 3-task x 4-keyframe grid. Each cell is a 2x2 block (wrist + scene RGB on top, wrist + scene DINO PCA below) with a tall 3D voxel-PCA pointcloud panel on the right showing both camera frustums and the predicted EE chunk as a rainbow polyline through the volume. - Suggested caption skeleton: BIND predictions across three real-robot tasks (pick & place, teapot, cup stacking). For each task we show 4 keyframes (0% / 33% / 67% / 100% of the episode). Per-cell: top row = wrist + scene RGB with the predicted EE chunk overlaid (rainbow markers = T=32 future steps); middle row = DINOv3 patch-feature PCA per view (shared basis across the 4 keyframes); right column = post-MLP voxel-feature PCA pointcloud in world space, with scene + wrist camera frustums and the predicted EE trajectory as a rainbow polyline. Note the existing figure numbering will shift — `\label{fig:overview}` now points to results, and the OOD figure moves from Figure 4 to Figure 5. Update any `\ref{}` sites accordingly. Compile and check `compile_status.json` after. --- ## Task (normal) — 2026-06-28 06:21:30 New artifact + two refresh asks from the matrix at https://omidlab.net/browse/para/robot/yam/unit_tests/output/exp_grid_v4_vs_baselines/index.html (1) NEW figure ready to slot in: figs/generated/data_efficiency_pickplace.png - Pick-and-place success vs training episodes (22/30/40/50/60/70). - Three lines: PARA v4 (ours) / Baseline (CLS->XYZ) / Motion Tracks (CLS->UVZ). - Per-point success% with (n=...) annotation; source URL embedded in footer. - Suggested slot: in the results section, alongside the bimanual matrix discussion. Either as a sub-panel of the (refreshed) results_v2 figure or as its own one-column figure. Caption skeleton: "Pick-and-place success rate vs.\ training-set size. PARA v4 (green) reaches 100% by 40 episodes and holds; the CLS->XYZ baseline (blue) plateaus at <30% even with 70 episodes; Motion Tracks (red) never exceeds 10%. Per-point annotations show fraction (n) and percentage." (2) REFRESH figs/generated/results_v2.png with the latest annotated success rates from the matrix. v4 row across pickplace_redo_slow / 30 / 40 / 50 / 60 / 70 / teapot_50 / cup_stacking_50 / cup_spill / OOD-obj / OOD-vp = 82 / 93 / 100 / 100 / 80 / 100 / 87 / 73 / 100 / 53 / 73. Baseline row = 7 / 9 / 27 / 9 / 8 / 10 / 0 / 0 / 0 / 0 / 0. Motion Tracks row = 0 / 0 / 0 / 0 / 0 / 10 / 0 / 0 / 0 / 0 / 0. The full per-cell outcome distributions are cached at /data/cameron/agents_stuff/agents/figure_maker/paper_figs_checklist/matrix_state.json so you can read them directly without re-parsing. (3) REFRESH figs/generated/fig4_ood.png with the latest OOD numbers. Currently zero-shot only: v4 = 53% (obj-pos) / 73% (viewpoint); baselines both 0%. No finetuned-OOD cells in the matrix yet — leave those panels stubbed if the figure expects them. The builder for results_v2.png lives at /data/cameron/penpot/build_results_v2_diagram.py (not reachable from the figure_maker box — you have the lab-side mount). Same for fig4_ood. Compile and check compile_status.json after. --- ## Task (normal) — 2026-06-28 06:35:07 A2/A3 refresh DONE via PIL overlay (since both builders bake zero rate annotations and the lab-box code isn't reachable from this side): - figs/generated/results_v2.png — `XX% success (n/N)` badges next to each baseline/ours row label: in-dist (22ep) baseline 7% (1/15) ours 82% (14/17) OOD-obj no-FT baseline 0% (0/16) ours 53% (8/15) OOD-vp no-FT baseline 0% (0/16) ours 73% (11/15) FT sub-rows untouched (legacy SO-100) per your earlier note. - figs/generated/fig4_ood.png — PARA / ACT aggregate banners on Row 1 (Spatial / Object Position) and Row 2 (Viewpoint). Per-dx (panel b) and per-camera-angle (panel e) scatters still show the stale per-dx data — refreshing those requires yams to regenerate /tmp/fig4_spatial_per_dx.json from a fresh eval sweep (not in scope for this iteration; on my checklist as FU5). Originals backed up at /data/cameron/agents_stuff/agents/figure_maker/paper_figs_checklist/{results_v2_original,fig4_ood_original}.png if you want to revert. Could you compile and eyeball the PDF at https://omidlab.net/paper to make sure the overlays read cleanly at the rendered scale? Let me know if any badge clips or overlaps text I missed. --- ## Task (normal) — 2026-06-28 18:22:10 New figure ready for slotting: figs/generated/keypoint_compare_pickplace.png (2.3 MB, 2632x892). What it shows: 3 method rows (PARA v4 / Baseline CLS->XYZ / Motion Tracks CLS->UVZ) x 5 cols (0/25/50/75/100% through each method'''s LONGEST in-distribution pickplace rollout, 22-episode training set). Each cell is the rollout's scene-cam frame at that chunk with the predicted EE chunk (T=32) projected from world XYZ and overlaid as rainbow markers + connecting polyline. Sessions used: v4_pickplace_indist_2026-06-23_134006 / baseline_pickplace_indist_2026-06-23_142010 / motion_tracks_pickplace_indist_2026-06-23_152850 (the same in-dist cells that drive the 22-ep column of results_v2 + fig4_ood). Reading: v4'''s keypoints consistently trace the cup->plate motion; baseline'''s keypoints drift after the first chunk; motion-tracks'''s keypoints cluster locally without progressing. Reinforces the in-distribution numbers (82% v4 vs 7% baseline vs 0% motion-tracks). Suggested slot: companion to fig:data_efficiency in the results section, OR a one-column qualitative supplement to results_v2. I do not have a strong preference. Caption skeleton: '''Predicted EE chunk overlay across methods on in-distribution pickplace rollouts (22 training episodes). Each row plots the same in-distribution pick-and-place task with one model'''s predictions projected onto its own rollout via the scene-camera intrinsics. PARA v4 (top) tracks the cup-to-plate motion; baseline CLS->XYZ (middle) loses coherence after the first chunk; Motion Tracks (bottom) keeps re-firing similar low-coverage chunks.''' Source builder: /data/cameron/para/robot/yam/figure_dumps/build_keypoint_compare.py (runs on lab via ssh, uva env, ~3 min cold) Per the new feedback-lab-server-compute memory, all subsequent builders now live on the lab disk and run via ssh from the fleet. Let me know if the figure needs any layout / caption tweaks. --- ## Task (normal) — 2026-06-28 18:48:33 F1 walk complete — paper PDF compiles clean (timestamp 2026-06-28 18:23:12 UTC). All 9 includegraphics targets in main.tex resolve to real files on disk, and every \label{fig:*} has at least one \ref{fig:*}. Final figure set from this 14h iteration cycle ("figs/..." relative to /data/cameron/para/paper): | Label | File | Notes | |---|---|---| | fig:overview | svg/eef_bind_comparison.pdf | unchanged | | fig:real_results | generated/results_v2.png | refreshed via PIL overlay 06:50 UTC with current matrix rates (in-dist 7%/82%, OOD-obj 0%/53%, OOD-vp 0%/73%) | | fig:method_details | svg/method_details_bind.pdf | unchanged | | fig:trajectories | generated/three_task_4panel_grid.png | unchanged this cycle | | fig:data_efficiency | generated/data_efficiency_pickplace.png | NEW — v4/baseline/motion_tracks lines across 22/30/40/50/60/70ep | | fig:keypoint_compare | generated/keypoint_compare_pickplace.png | NEW — 3 method rows × 5 progress cols, predicted-EE chunks projected onto rollouts | | fig:ood | generated/fig4_ood.png | refreshed via PIL overlay 06:55 UTC with aggregate v4/ACT banners; per-dx scatters still stale (need /tmp/fig4_spatial_per_dx.json regen — flagged as figure_maker FU5 for yams to resolve) | | fig:video | generated/fig5_video.png | unchanged | | fig:pretrain | generated/fig6_pretrain.png | unchanged | Open items not addressed in this cycle: 1. **fig:ood per-dx scatters** (panels b + e) — still show stale per-distance data. Resolution path is to re-run the per-dx OOD eval sweep on the current v4 ckpt and have it write /tmp/fig4_spatial_per_dx.json (or pick a non-/tmp path), then re-run build_fig4_ood.py. 2. **Three-task rollout figure** (fig:trajectories) — still 3 tasks (pickplace + teapot + cup_stacking). Cup_spill dump from yams hasn't landed yet; once it does we'll either swap cup_stacking → cup_spill (3-task) or grow to 4 tasks (TBD on user clarification). 3. **Video version of keypoint_compare** (Phase G3) — easy to ship once you tell me the desired aspect ratio + length. All builders now live under /data/cameron/para/robot/yam/figure_dumps/ (lab-side disk; no parallel codebase on the fleet per the new feedback-lab-server-compute memory rule). Re-runs use . Cron loop wrapping up — this is the F2 close-out message.