# our_wandb — file-system wandb viewer

## What this agent does

Builds and maintains a lightweight web viewer that reads wandb run directories on disk and renders a wandb-style dashboard. No external services, no DBs, no code changes in training scripts.

## Why it exists

On 2026-06-09 we tried self-hosting real wandb (docker `wandb/local`, then K8s operator + minio). Both paths burned ~4 hours and hit upstream bugs / chart-config rabbit holes. Cameron's instinct: just read the files wandb already writes locally and present them. Most of how Cameron uses wandb is image gallery + final summary + loss curve, all of which are recoverable from disk without parsing the proprietary `.wandb` binary format (or only lightly).

See [[wandb-selfhost-attempted-2026-06-09]] for the prior attempt notes.

## Current state

- **MVP LIVE (2026-06-10)** at `omidlab.net/runs/` (local :8090). Verified end-to-end in a real browser on the live yukon mount: grouped run list, loss chart + smoothing slider, config/summary tables, image gallery + per-panel step slider, 10s auto-refresh, no console errors.
- Code: `reports/wandb_runs.py` (scan/parse) + `reports/runs/index.html` (UI) + a `/runs/` + `/api/runs/*` route block in `serve.py` (after the data_viewer block). website_builder owns serve.py restarts.
- Next: confirm grouping granularity with Cameron (currently per exact `--name`; some groups size-1). v2: `.wandb` protobuf parser for older runs' full loss history.

## Data sources

⚠️ ROLE.md's original `yam_remote/...` defaults are **stale** — `/data/cameron/yam_remote`
symlinks to the abandoned russet mount (`mnt/robot-lab`) and the path doesn't exist.
Live roots are on the **yukon** mount (configurable via `WANDB_ROOTS`):
- `/home/cameronsmith/mnt/yukon/cameron/puget/wandb/` (most recent / today's runs)
- `/home/cameronsmith/mnt/yukon/cameron/puget/code/wandb/` (older, complete runs)
- `/home/cameronsmith/mnt/yukon/cameron/puget/data/wandb/`

(yukon is the live YAM rig — see [[project_russet_abandoned]].)

## Parse facts (confirmed from real dirs)

- **Grouping key**: `--name` value in `files/wandb-metadata.json` → `args` (always present). e.g. `mv_scene_wrist_clsconcat_wristpp`. Fallback: `program` basename → run id.
- **Loss curve**: parse `files/output.log` lines like `ckpt @ 200  total=.. vol=.. grip=.. rot=..` (newer) / per-epoch lines (older). No `.wandb` protobuf parsing needed for MVP.
- **config.yaml + wandb-summary.json**: only on *completed* runs. In-progress runs have only metadata + output.log + media — viewer must not crash when they're absent.
- **Media**: `files/media/images/<panel>/<key>_<idx>_<step>_<hash>.png`.

## File layout the viewer parses

```
run-<YYYYMMDD_HHMMSS>-<id>/
├── files/
│   ├── media/images/*.png          ← logged images
│   ├── config.yaml                 ← hyperparams
│   ├── output.log                  ← stdout
│   ├── wandb-metadata.json         ← run metadata (cmd line, host, etc.)
│   └── wandb-summary.json          ← final metric values
├── logs/debug.log
└── run-<id>.wandb                  ← binary history (skipped in MVP)
```

## Where it's hosted

TBD by agent. Recommended: extend `serve.py` (see [[fleet-infra-overview]]).

## Cameron's one explicit new feature

Group runs in the top-level list by **model type / model name**, not just chronologically. Look up the model identifier in `wandb-metadata.json` (cmd line / `--name` flag) or `config.yaml`. Confirm the right key with Cameron after a first pass.