+ UjRt^RIHt^RIt^RIt^RIt^RIt^RIHt^RI t ^RI t ^RI t ^RI t ^RIHt]P P#^]!]!]4P)4P*44^RIHt^RIHtHtHt^RIHt^RIHtH t ^R I!H"t"^R I#H$t$^R I%H&t&R R lt'RRlt(RRlt)RRRllt*Rt+],R8Xd ]+!4R#R#)u(Scene-only DinoVolumeScene training loop — single arm, no wrist. Per-step losses: volume CE (pixel + height) + gripper CE + K-means rotation CE. Per-vis: PCA of refined features, GT-vs-pred keypoints overlay, per-T heatmap grid, and per-(T, bin) heatmap grids for rotation / gripper / height. Checkpoint payload (everything inference needs to unpack predictions): - ``model`` state_dict - ``opt`` state_dict - ``step`` int - ``args`` dict (CLI args) - ``dataset_cfg`` includes rot_centroids (K, 4) + bin-range configs Run on puget (GPU):: ssh dev source ~/yam_para/.venv/bin/activate cd ~/lab/para/robot/yam CUDA_VISIBLE_DEVICES=0 nohup python train.py --name \ --data_root ~/yam_para/data/ > /tmp/train.log 2>&1 & ) annotationsN)Path) DataLoader) ArViewScene)DinoVolumeSceneDinoVolumeScenePerVoxel all_losses)bin_grid_panel)feature_pca_panelshared_feature_pca_panels)marginal_heatmap_grid)height_curve_panel)keypoints_overlayc(V^8dQhRRRRRRRR/#)xyz_wz np.ndarrayKT_w2cimg_sizeint)formats"(/data/cameron/repos/smithbot_v4/train.py __annotate__r.s)    : &) cz\P!VP^,^3VPR7p\P!W.RR7pW%P ,P R,p\P !VR,RR4pWP ,P pVR,V, #) uProject ``xyz_w`` (T, 3) through one cam → image-size pixel coords (T, 2). Caller should clip + treat as drawing coords; in-frustum mask is implicit in clipped values. Assumes K is already scaled to img_size×img_size.dtypeaxisgMbP?N)NNN:NN)r!:rr"N)r!:NrN)nponesshaper concatenateTclip) rrrrr$xyz_hcamzuvws &&&& r_project_world_to_pixr-.s 77EKKNA&ekk :D NNE=r 2E 77?  e $C F T4(A uu9--C u:>rc V^8dQhRRRR/#rrrvis_nr)rs"rrr;saasa3arc /p\VR,4pVR,P4P4P4pVR,P4P4P4pVR,P4P4P4pVR,P4P4P4p VR,p V U u.uF.qP4P4P4NK0 p p VR,p VP R4pVR ,P4P4pVR ,P4P4pVR ,P4P4pVR ,P4P4pVR ,P4P4pRV9d&VR,P4P4MRpRV9d&VR,P4P4MRpVP wppppp\ \VV44EFp.p.p.pVEFp V R8XdqVP\P!V V,W"3\PR74VPVV,4VPVV,4K{V R8XgKVe VV,MRpVeVfKVP\P!VW"3\PR74VPVV,4VPVV,4EK V'gEK VV,PVR#4p V P^R7p!V!VV,,p"V!VV,,p#V#V,p$V#V,p%VVV"V$V%R3,p&VV,p'.p(.p)\VR\V44EFBwp*p VV*,p+\!V'VV*,VV*,V4p,\!V&VV*,VV*,V4p-\#V+P%4V,RRR$^ R7p.\#V.V-R^R7p.\&P(!^V.P ^,^3^\&P*R7p/\P,!V/V R%\P.RR&^\P04V(P\&P2!V/V..44V)PV V*,V,P5^^^44EKE \7V)W"3R7p0.p1\\9VR\V4V044Fwp*wp p2\&P(!^V2P ^,^3^\&P*R7p3\P,!V3V R2R%\P.RR&^\P04V1P\&P2!V3V2.44K \&P:!V(4p4\&P:!V14p5\<P>!V4RRRRR#13,4VRV 2&\<P>!V5RRRRR#13,4VRV 2&\<P>!\AVV,VV,R 7RRRRR#13,4VR!V 2&\<P>!\AV V,VV,R 7RRRRR#13,4VR"V 2&EK V#uup i)'zMulti-view panel set: pred vs GT keypoints overlaid on EACH view's image side-by-side, plus per-view PCA, plus the existing grip/rot grids. Voxel-position lookup gives us the predicted world XYZ DIRECTLY (no ray-cast); we then project that to each camera.views volume_logitsvoxel_positions grip_logits rot_logitsview_feat_maps_bgr_src_wrist_bgr_src target_xyz target_grip target_rot_K_in_T_w2c_K_wristN _T_w2c_wristscene interpolationwristrr!Ffilled draw_linecolor_override radius_pxTrGrIr皙?out_wh PCA (joint)viz/kp_gt_vs_pred_viz/pca_dino_mlp_targetviz/grip_grid_ viz/rot_grid_r rVrVr[r[)!listdetachcpunumpygetr%rangeminappendcv2resize INTER_AREAreshapeargmax enumeratelenr-rcopyr#fulluint8putTextFONT_HERSHEY_SIMPLEXLINE_AAvstack transposer ziphstackwandbImager )6batchoutrr0panelsr2vol voxel_posgriprotr7v view_feats_np bgrs_scene bgrs_wristr:r;r<K_scene T_w2c_scene K_wrist_b T_w2c_wrist_bBr'NZP_b view_bgrsview_Ksview_Tswbgrflat idx_jointnz_idxhwy_idxx_idxpred_xyz_world gt_xyz_world kp_stripsfeats_per_viewkbggt_pixpred_pixkpbarpca_imgs pca_stripspcabar2 kp_concat pca_concats6&&&& r_viz_batch_multiviewr;s F W E o  % % ' + + - 3 3 5C%&--/335;;=I }  $ $ & * * , 2 2 4D l  " " $ ( ( * 0 0 2C)*N7EF~!XXZ^^%++-~MFz"J+,J|$((*002J &**,224K|$((*002JGn  "((*G/%%'--/K3=3Fj!%%'--/DI;IU;RE.)--/557X\MYYNAq"a 3ua= ! AG|  JqMH;Oz!}D<9#4  D82F!!}  eOS^45DAq1B*<WQZQYZF,^WQZQRU]^H"2779f,1U4CrSB#2x41MB''2rxx{A."((CC KKsWc.F.F3;; @   RYYRy1 2  ! !-"21"5"?"?1a"H I6$- H#79 $SI)?%JKKAx377B ! a0"BHHED KK<0'3;S;S3;; @   biis 4 5 L IIi( YYz* +0;;yAtt7L+M#A3'(*/++jAtt6L*M"1#&'(-{{ 47;q> :1a2: F(Hs#$&+kk 3q6*Q- 8Att D'Fqc"#K"N MkGs.4]c V^8dQhRRRR/#r/r)rs"rrrscccCcrc ^RIHpHpHp/p\ VR,4pVR,P 4P 4P4p VR,P 4P 4P4p VR,P 4P 4P4p VR,U u.uF.qP 4P 4P4NK0 p p VR,pVPR4pVR ,P 4P4pVR ,P 4P4pVR ,P 4P4pVR ,P 4P4pVR ,P 4P4pRV9d&VR,P 4P4MRpRV9d&VR,P 4P4MRpV P^,pV P^,pV!V^R7Uu.uF/pVP 4P 4P4NK1 ppV!V4P 4P 4P4p\\VV44EFp.p.p.pVEFp V R8XdqVP\P!VV,W"3\PR74VPVV,4VPVV,4K{V R8XgKVe VV,MRp V eVfKVP\P!V W"3\PR74VPVV,4VPVV,4EK V'gEK V V,p!VV,p".p#.p$.p%\!VR\#V44EFwp&p VV&,p'\%V"VV&,VV&,V4p(\%V!VV&,VV&,V4p)\'V'P)4V(RRR/^ R7p*\'V*V)R^R7p*\*P,!^V*P^,^3^\*P.R7p+\P0!V+V R0\P2RR1^\P44V#P\*P6!V+V*.44V$PV V&,V,P9^^^44VV&,V,p\*P:!VVP=4, 4p,V,\=V,P=4R4,p,V,R,P?\*P.4p-\P@!V-\PB4p.\P!V.W"3\PDR7p.\PF!V'RV.R^4p/\*P,!^V/P^,^3^\*P.R7p0\P0!V0V R2R0\P2RR1^\P44V%P\*P6!V0V/.44EK \IV$W"3R 7p1.p2\!\KVR\#V4V144Fwp&wp p3\*P,!^V3P^,^3^\*P.R7p4\P0!V4V R!2R0\P2RR1^\P44V2P\*P6!V4V3.44K \LPN!\*PP!V#4R"R"RRR213,4VR#V 2&\LPN!\*PP!V24R"R"RRR213,4VR$V 2&\LPN!\*PP!V%4R"R"RRR213,4VR%V 2&VV,p5V5V5P=^RR&7,p5\*P:!V54p6V6\*PR!V6PU^RR&7R4,p6^p7^p8V7V,p9\*P,!V8V9^3^\*P.R7p:.R3Op;\V4FpT8p?\V6P^,4Fp&\W\YV6V&V<3,V8^, ,44p@V@^8dK7V;V&\#V;4,,pA\PZ!V:V=V?X@, 3V>V?3VAR2R'7V?V@,p?K K \*P,!^V9^3^\*P.R7pB^pC\!VRV6P^,4Fwp&p V;V&\#V;4,,pA\PZ!XBXC^3VC^,^3VAR2R'7\P0!VBV VC^,^3\P2R(R1^\P44VC^^\#V 4,,, pCK \LPN!\*P6!XBV:.4R"R"RRR213,4VR)V 2&\LPN!\]V V,VV,R*7R"R"RRR213,4VR+V 2&\LPN!\]V V,VV,R*7R"R"RRR213,4VR,V 2&EK 0R4mpDVR-,P 4P4pE\\VV44EF@p/pFVP_4Fwp&p V&XD9dV XFV&&K\`Pb!V 4'dV VV^,XFV&&KA\eV \4'dMV 'dE\`Pb!V ^,4'd"V UCu.uFpCVCVV^,NK upCXFV&&KV XFV&&K V!XF4^,P 4P 4P4pG\LPN!\]VGPfXEV,R*7R"R"RRR213,4VR.V 2&EKC V#uup iuupiuupCi)5zv3 panel set: per-view marginal heatmap (t=0), per-view PCA, pred vs GT keypoints projected to each view, grip/rot bin grids, and a per-timestep view-dominance bar. )v3_per_view_heatmapsv3_view_dominance v3_z_marginalr2rr5r6r7r8r9r:r;r<r=r>r?Nr@)t_indexrArBrDFrETrJrrKg& .>go@皙?g333333?z heatmap (t=0)rLrNr!rOrPzviz/heatmap_per_view_t0_rkeepdims) thicknessg?zviz/view_dominance_rQrSrTtarget_z viz/z_grid_rUrWrZr ))<)rPr)rrx)rrr>n_zpe_tpe_zr2n_views pred_size)4lib.model_volumetric_v3rrrr\r]r^r_r`r%rarbrcrdrerfrirjr-rrkr#rlrmrnrorprqrrexpmaxastype applyColorMapCOLORMAP_INFERNO INTER_LINEAR addWeightedr rsrurvrtmaximumsumrround rectangler itemstorch is_tensor isinstancer')Hrwrxrr0rrrryr2 chosen_xyzr|r}r~rrrr:r;r<rrrrrT_hplog_p_per_view_t0domrrrrrrrrrheatmap_stripsrrrrrrhp_linhp_imghp_colorblendrrrrbar3dom_tdom_pcol_wHbar_wbar_imgpalettetx0x1y_curh_kcollegx _no_batchrout_bz_logsH&&&& r _viz_batch_v3rs ;;F W E%&--/335;;=J }  $ $ & * * , 2 2 4D l  " " $ ( ( * 0 0 2C7:;K7LM7L!XXZ^^%++-7LMMz"J+,J|$((*002J &**,224K|$((*002JGn  "((*G/%%'--/K3=3Fj!%%'--/DI;IU;RE.)--/557X\MA   ! B%9a$HJ$Hb*002$HJ C ' ' ) - - / 5 5 7C 3ua= ! AG|  JqMH;O,Jg(2(>z!}D<9#4  D82FE77B A2BbhhGD KK>2G114(!S[[ :  ! !"))T5M": ;76:- H#79 $SI)?%JKKAx377B ! a0"BHHED KK<0'114(!S[[ :   biis 4 5 L,1;;ryy7KAqRVTVRVJ7W+X#A3'(*/++bii 6KAqRVTVRVJ6W*X"1#&'16 IIn %aDbDj 123)!-. A D11u  EII1tI*%ad q1u 5677a#g,./ gECK'82u+"b2 +ggr5!nb9 eOU[[^45DAq!c'l*+C MM#1vB|SB G KKQR c.F.F3;; @ 1s1v:% %A 6 -2KK IIsGn %aDbDj 1-3$QC()(-{{ 47;q> :1a2: F(Hs#$&+kk 3q6*Q- 8Att D'Fqc"#U"`IIZ $$&,,.H 3ua= !IIKDAqI~a##Qq1u:aAt$$uqt/D/D0121AaAJ2aa e$Q'..0446<<>$)KK 5778A; 71dd C%EQC !" MkN J~3s4u5u uc ,V^8dQhRRRRRRRRRR/#) rrrr0 per_voxelboolz_lofloatz_hir)rs"rrrEs9NNSNN N05NCHNrc J RV9dRV9d \WW#4#\VPR4\\34'd$\ VR,4^8d \ WW#4#/pVR,P4P4P4pVR,P4P4P4p VR,P4P4P4p VR,P4P4P4p VR,p VR,P4P4p VR ,P4P4pVR ,P4P4pVR ,P4P4pVPwpppppVV, p\\VV44EFp\P!V V,W"3\PR 7p\!V V,P#^^^44p\$P&!VR R RRR!13,4VRV 2&V V,R RRR!13,P)\*P,4V,pVV,P/VR!4pVP1^R7pVVV,,pVVV,,p\*P2!VV,P)\*P,4V,VV,P)\*P,4V,.^R7p\5VP74VRRR"^ R7p \5V VR^R7p \$P&!V R R RRR!13,4VRV 2&\9VVV,VRR7p!\$P&!V!R R RRR!13,4VRV 2&V'dB\;VVV,WVVR#R7p"\$P&!V"R R RRR!13,4VRV 2&MVVP=^RR7, p#\*P>!V#4p$V$V$PA^RR7,p$V$P/VVVV4PAR$R7p%\$P&!\C\*PD!V%R,4VV,R7R R RRR!13,4VRV 2&\$P&!\CV V,VV,R7R R RRR!13,4VRV 2&\$P&!\CV V,VV,R7R R RRR!13,4VR V 2&EK V#)%aPer-batch wandb panels. ``per_voxel=True`` swaps the marginal-Z bin grid for a pred-vs-GT height-over-time line plot (the per-voxel model doesn't have a separate height head). Multi-view models (``out["views"]`` set) take a different path that renders pred-vs-GT keypoints + PCA on EACH view's image, concatenated horizontally. Required batch fields for multi-view: ``_bgr_src``, ``_wrist_bgr_src``, ``_K_in``, ``_T_w2c``, ``_K_wrist``, ``_T_w2c_wrist``, ``target_xyz``. view_Fzr3r2r5r6 F_refinedr8 target_yxrr;r<rBr!NrPrFrETrJrO?) cell_sizealphazviz/heatmap_grid_) pred_z_bins gt_z_binsrrrrMzviz/height_curve_rg-q=rQrrSrTr rU))rr")#rrr`r\tuplerjrr]r^r_r%rarbrdrerfr rrrurvrr#float32rgrhstackrrkr r rrrr log)&rwrxrr0rrrryrzr|r} feats_refbgrsrrr;r<rr'Zrrscalerrrrrr pred_z_binrrrgridhc flat_normeprobs_zs&&&&&&&& r _viz_batchr EsCO36U99#'''"T5M22s3w<7HA7M#E@@ F o  % % ' + + - 3 3 5C }  $ $ & * * , 2 2 4D l  " " $ ( ( * 0 0 2CK '')--/557I  Dk"&&(..0IZ $$&,,.H &**,224K|$((*002JIIMAq!Q qLE 3ua= ! ZZQ(!5S^^ T ! 6 6q!Q ?@*/++c!Q"*o*F"1#&'A,q$B$w'..rzz:eC1v~~a$KKQK' 1q5) !a% 88b1f__RZZ85@!Qw..rzz:UBDJKM rwwy&',/>"Nr8tq I+0;;r!Q"*~+F#A3'(%RQ1CH*/++d1a2:6F*G"1#&' $ /7{*.q,68B/4kk"Q4R4Z..IF&qc* +txxQx>>Iy!A11t(D#D1ii1a+//V/ :1a2: F(Hs#$',kk 3q6*Q- 8Att D'Fqc"#["` MrcEaa+aOaPaQ\P!4pVPRRR7VPRRR7VPR\\P !4R, 4R7VPRRR7VPR \ ^(R 7VPR \ ^R R 7VPR\RRR 7VPR\RRR 7VPR\RRR 7VPR\ ^RR 7VPRRRR.RR7VPR\ ^R 7VPR\ ^R 7VPR \ ^ R 7VPR!\ ^ R 7VPR"\ R#R 7VPR$\ ^R 7VPR%\ R&R 7VPR'\ ^0R 7VPR(\ R#R 7VPR)\ ^R 7VPR*\ ^R 7VPR+\RR,R 7VPR-\RR.R 7VPR/\ERPR 7VPR0\R1R 7VPR2\ R3R4R 7VPR5\ ^dR 7VPR6\ ^2R 7VPR7R8R7VPR9RR:R;7VPRR?7VPR@R=RAR?7VPRBR=RCR?7VPRDR=RER?7VPRFR=RGR?7VPRHR=RIR?7VPRJRKRKRL.RMR7VPRN\ ^ROR 7VPRP\ ^R 7VPRQR=RRR?7VPRSR=RTR?7VPRUR=RVR?7VPRWR=RXR?7VPRYR=RZR?7VPR[R=R\R?7VPR]RR^R;7VPR_R`RaR;7VPRbRcRdR;7VPRe\PRfRgRh7VPRi\RjRkR 7VPRlR=RmR?7VPRn\ ^R 7VPRo\ ^R 7VPRp\ ^ R 7VPRq\PRRrRh7VPRsRtRtRu.RvR7VPRw\ ^RxR 7VPRy\ ^R 7VPRz\ R{R|R 7VPR}\ ^R~R 7VPR\ ^@RR 7VPRRRR;7VPR\ ^RR 7VPR\ ^@RR 7VPRR=RR?7VPRR=RR?7VPRR=RR?7VPR\ ^RR 7VPRR=RR?7VPRR=RR?7VPRR=RR?7VPRR=RR?7VPRR=RR?7VPR\ ^RR 7VPRR=RR?7VPR\RRR 7VPRR=RR?7VP4oP\ SPP4SPP, pVPRRR7RpSPP'd\P!SPPRRfR7p\!RSPP RVR, R24SPP"'gAVR,R,p\V^,4\V^,4uSPnSPnM \!R4SPP$e'SPP&eSPP$SPP&3MRpVe#SPP('gVR,R,MRpSPP('dVe \!R4\+\ SPP,4SPP.SPP0SPP2SPP4SPP6SPP8SPP:TSPP<SPP>3TSPP@'d SPPBMSPPD'd SPPFM^R7 pVP$VP&uSPnSPnVPI4pERQoOVO3Rlp\KVSPPLRSPPNRRVR7p \QSPPRSPPTSPPVSPPXSPP@SPPZSPP\\_SPPD4.4p V ^8d \aR4hSPPD'dSPPDPcR4MRoQSPPR'd ^RI2H3p T p EMSPPV'd ^RI4H5p T p EMSPPX'd ^RI4H6pTp EMSPP@'d ^RI4H7pTp EMSPPZ'd ^RI4H8pTp EMtSPP\'d ^RI9H:pTp EMXSPPv'dSQ'gQR4h^RIE\6EP8!4p?V>SPEP:8EdV EF p@V@ER,PSPPRER7pAV@ER,PSPPRER7pBV@ER#,PSPPRER7pCV@ER$,PSPPRER7pDV@ER%,PSPPRER7pEV@ER&,PSPPRER7pFV3EP<F!pGV4!V>VGER ,VGER,4VGER &K# SQ'djV8!VX@4pHV@ER',PSPPRER7P4pISPP'd V9!XHXIXEXF4pJM|V9!XHXCXDXEXFXI4pJMnV!XAXB3/V5!X@4BV6!V@4BpHV7eCX@ER',PSPPRER7P4pIV7!XHXCXDXEXFVI4pJME\?XHXCXDXEXF4pJV3EPARER(7XJER),EPC4V3EPE4V>^, p>E\6EP8!4pKRVKV?, , pLTKp?ER*V>ER+VL/pMV3EP<F'pGVGER ,XMER,VGEPGER ER-4 2&K) V3EP<^,ER ,XMER &XJP4Fwop V EPI4XMS&K V>SPEPJ,^8XdsXMPE\5X@XHSPP4SPPT;'g!SPPX;'g SPP\SPP$SPP&ER 74E\EP2!XMV>ER!7V>SPEPL,^8XEd>VER., pN\EPN!RVP4ER/V3P4RT>ER0E\!SP4RTER1SPPv'dER2EM)SPP|'dER3EMSPP'dER4MSPP'dER5MSPP'dER6MSPP'dER7MSPP'dER8MSPPR'dER9MSPP@'dER:MSPPZ'dER;MkSPPV'dERM,SPPD'dER?MSPP\'dER@MERA/XN4VNEPQVERB, 4\!ERCV> ERDXJER),EPI4ERE ERFVJERG,EPI4ERE ERHVJERI,EPI4ERE ERJVJERK,EPI4ERE ERLXLERM ERN2 RERO7V>SPEP:8gEK EK" EK'R#uupiuup piuupiuup piuupp+iuupiuupi +'giELy;i(Uz--nameT)requiredz --data_rootz --ckpt_rootzyam_para/checkpoints)defaultz--wandb_projectyam_train_simplez --batch_size)typer z --num_workersuLDefault bumped 4→8 since training is CPU-bound on JPEG decode at batch 40.)rr helpz--lrga2U0*3?z3Head LR (and backbone LR if --backbone_lr_mult==1).z--lr_minNzvTerminal head LR for cosine schedule. Default: same as --lr (constant LR). Set to e.g. 1e-5 with --lr_schedule cosine.z--backbone_lr_mult?uBackbone LR = --lr * this. <1.0 protects pretrained DINO weights from being damaged by the fresh-head LR. The ablation experiment showed lr=2e-5 was the magic value for from-scratch — i.e. 0.1× of head-friendly 2e-4. Backbone min LR scales by the same factor.z --lr_warmupzkLinear-warmup iters for both groups (0 = no warmup). Recommended ~5%% of --max_iters for from-scratch runs.z --lr_scheduleconstantcosineu<LR schedule after warmup. cosine = decay from lr → lr_min.)r choicesrz --n_windowz--frame_stridez--n_height_binsz--n_gripper_binsz--n_rot_clustersrz --pred_sizez --img_sizeiz --feat_dimz --d_modelz --n_blocksz --mlp_hiddenz--z_louLower height bin edge (m). Default = auto-derive from dataset (min EE z − pad). Finetunes override with the source ckpt's range automatically.z--z_hizOUpper height bin edge (m). Default = auto-derive from dataset (max EE z + pad).z --grip_loz --grip_hig?z --max_itersi@zbDefault sized so a fresh run finishes in ~70 min at 2 it/s (avoids overfitting on small datasets).z--ckpt_every_itersz--vis_every_itersz--devicecudaz --init_fromzPath to a checkpoint to initialize model weights from (finetune mode). Also reuses the source ckpt's K-means rot centroids so the rot_head stays calibrated.)r rz --per_voxel store_truezUse DinoVolumeScenePerVoxel instead of DinoVolumeScene (per-voxel features, no factorized scoring, CLS-only input). Saves model_kind='dino_volume_per_voxel' in ckpt.)actionrz--da3u&Use DA3VolumeScene (Depth-Anything-3 backbone + camera token + parallel feat DPT branch deep-copied from depth). Same factored YX×Z×T scoring as DinoVolumeScene. Defaults swap to n_height_bins=128, feat_dim=32. Saves model_kind='da3_volume_scene' in ckpt. Mutually exclusive with --per_voxel.z--hires_factorizeduUse DinoVolumeSceneHires — DINOv3 backbone + bilinear up + 5-layer CNN at pred_size=256. Factorized scoring. Defaults: n_height_bins=128, feat_dim=48. Saves model_kind='dino_volume_scene_hires'.z--hires_per_voxeluUse DinoVolumeScenePerVoxelHires — DINOv3 + 5-layer CNN at pred_size=256, then 1×1 conv expansion to feat_dim*n_z and reshape to per-voxel volume. Defaults: n_height_bins=64, feat_dim=16. Saves model_kind='dino_volume_per_voxel_hires'.z --hires_flatuUse DinoVolumeSceneHiresFlat — same backbone + CNN as DinoVolumeSceneHires, but the per-T queries come from a single flat MLP output (output dim = T × per_T_dim) instead of T independent AdaLN-Zero forwards. Better temporal coherence by construction.z --volumetricuZUse DinoVolumeSceneVolumetric — unproject image features into a P×P×Z world-space voxel grid, concat sin/cos height encoding, 3-layer per-voxel MLP, then dot-product against per-T queries from a flat global MLP. Adds a +1 GT-voxel slot at training for precise (unquantized) supervision. Defaults: pred_size=128, n_height_bins=64, feat_dim=32.z --cls_fusionconcatprimaryzMulti-view CLS fusion mode for the grip/rot head. 'concat' (new default) concatenates every view's CLS before the global MLP; 'primary' uses only views[0]'s CLS (matches the original 2026-06-09 ckpts).z--cross_view_layerszDA3-style alternating within/cross-view self-attention blocks over the native DINO patch grid (~31x31). 0 disables. 4-5 is the sweet spot per Depth-Anything-3 (arXiv:2511.10647).z--cross_view_headsz--v4zv4 = v2 multi-view MLP fusion + sin/cos height PE + per-view adaptive z range. Uses DinoVolumeSceneV4. Keeps the same out dict shape as the v2 multiview model so multiview_losses + viz pipeline reuse unchanged. Requires --multiview.z--v4_da3zxv4 head + DA3 backbone (multi-view ViT with built-in cross-attention + camera token; NO DPT head). Requires --multiview.z--v4_vlmz{v4 head + InternVL2.5-1B vision tower (LLM frozen, vision trainable, text-conditioned via task name). Requires --multiview.z --v4_dynaflipzv4 head + DynaFLIP-base vision backbone (jlee-larr/dynaflip-base, ~0.2 B params). Per-view loop, no backbone-level cross-view fusion. Requires --multiview.z--v4_paligemmazv4 head + PaliGemma vision tower + Gemma LLM (google/paligemma-3b-pt-448 by default). Mirrors v4_vlm but with the PaliGemma stack instead of InternVL. Requires --multiview.z--v4_pi0zv4 head + pi0's PaliGemma backbone (LeRobot port lerobot/pi0_base, ~14 GB safetensors). PaliGemma weights replaced by pi0's robot-data pretrain; the flow-matching action expert is discarded. Requires --multiview.z--pi0_safetensors_pathzROptional override for the pi0_base safetensors file. Default = pull from HF cache.z--paligemma_model_namezgoogle/paligemma-3b-pt-448zHF model id for the PaliGemma variant. The 448 variant fits puget 24 GB at B=2 with grad-ckpt + frozen LLM; for unfrozen-LLM training run on yukon (96 GB).z--vlm_task_textzpick and placez,Task prompt for the VLM (e.g. dataset name).z--vlm_freeze_llmFzYDefault: train the LLM (gradient checkpointing on). --no-vlm_freeze_llm keeps LLM frozen.)rr rz--cam_margin_mg{Gz?uv4: each view's z_hi is capped at cam_z - this margin (metres). Wrist cam → adaptive small z range; scene cam → unchanged (sanity-checked to stay above global z_hi).z--v3uv3 factorized dot-product model (DinoVolumeSceneV3). Per-pixel feature splits into [F_z|F_t|F_s] which dot-product against [PE_z|PE_t|spatial_q]. No per-voxel MLP. Factored loss never materialises the (B, T, V) volume — supports P=Z=256 cheaply.z--v3_F_zz--v3_F_tz--v3_F_sz--per_voxel_grip_rotuKPredict grip + rot from the GT-nearest voxel feature (teacher-forced at train, argmax at infer) via a small 1D-CNN over T, instead of from the global MLP query. Same idea as production PARA's pixel-indexed heads. Default ON (2026-06-10 — Cameron made this canonical). Pass --no-per_voxel_grip_rot to fall back to the global head.z--temporal_headflat_mlp temporal_cnnu%Per-voxel grip/rot temporal head. 'flat_mlp' (new default 2026-06-10): 4-layer dense MLP over the T·F flattened gathered features → full receptive field, implicit absolute positional info. 'temporal_cnn': 3-layer 1D conv over T — the old default; receptive field ~7 timesteps, no time PE.z--temporal_cnn_layersz61D CNN depth (only with --temporal_head temporal_cnn).z--temporal_cnn_kernelz--temporal_mlp_hiddenrz)Hidden dim of the flat MLP temporal head.z --mv_past_nu$(multi-view only) Past N frames of (grip, z) fed as proprio conditioning to the global query MLP. 0 disables. 8 was Cameron's 2026-06-10 default — matches frame_stride * 8 = 32 raw frames of recent EE state, enough to disambiguate 'just closed' vs 'closed for a while → ready to release'.z--mv_past_enc_dimz/Encoder output dim for the past-trajectory MLP.z --multiviewufComma-separated views for the multi-view volumetric model — e.g. 'scene', 'wrist', or 'scene,wrist'. Builds one P×P×Z sub-volume per view; each voxel gets features bilinear-sampled from ALL listed views' feature maps, concatenated, then fused by a 3-layer MLP. Loss is nearest-3D-point across the full N×P×P×Z combined volume. Mutex with --volumetric.z--height_enc_dimzHVolumetric model only: sin/cos PE dim for the per-voxel height encoding.z--voxel_mlp_hiddenz2Volumetric model only: per-voxel MLP hidden width.z--refit_z_rangeuWhen finetuning, ignore the source ckpt's z_range and let YamScene auto-fit a fresh range on the new dataset. Also drops ``height_emb.weight`` from the source state_dict (the bin-index → physical-Z mapping changes, so the embedding is stale). Required when the new task's EE-Z extent differs materially from the source's (e.g. table-level manipulation finetuned from a pickplace cup).z--refit_rot_centroidsa4Used with --init_from: refit K-means rot centroids on the NEW dataset instead of inheriting from the source ckpt, AND skip loading rot_head.* weights (since their output indices map to the OLD centroid IDs). Use when the new dataset has a different rotation distribution from the source ckpt's training data.z--hires_ablatezUse DinoVolumeSceneHiresAblate (factorized hires with configurable past-encoding / MLP / backbone-fusion ablation knobs). Mutex with the other model flags.z--past_nz'Past timesteps to encode (0 = no past).z--past_z_deltauiEncode past_z as (past_z − current_z) instead of absolute height. Pairs naturally with --past_uv_delta.z --past_use_uvu3Concat past pixel coords (img_size px → [-1, 1]).z--past_uv_deltaz/With --past_use_uv: subtract current start_pix.z--past_use_xyzz)Concat past world XYZ (m, centered ~0.3).z--past_xyz_deltaz8With --past_use_xyz: also concat past_xyz - current_xyz.z --mlp_layersu3Depth of the global MLP (1=plain Linear, ≥2=MLP).z--mlp_residualu9Residual MLP (skip every block; needs --mlp_layers ≥3).z--mlp_hidden_multg@u3MLP hidden = d_model × this (mlp_layers > 1 only).z--backbone_layer_fusionu]Concat DINOv3 patch tokens from layers [3,6,9,11] before feat_head (4× embed_dim per patch).)parentsexist_okr^) map_location weights_onlyz finetune from z (source step=step) dataset_cfgz_rangez[ --refit_z_range: ignoring source ckpt's z_range; auto-fitting fresh range on new dataset. rot_centroidsz` --refit_rot_centroids: ignoring source ckpt's rot_centroids; refitting K-means on new dataset.) rootn_window frame_striderrrn_gripper_binsn_rot_clustersr" grip_ranger#past_nc<a/pRF1p\P!SUu.uF q3V,NK up4W&K3 SUu.uF q3R,NK upVR&RS^,9dSUu.uF q3R,NK upVR&RFApVS^,9gK\P!SUu.uF q3V,NK up4W&KC \;QJdV3RlS4F 'dK RM RM!V3RlS44'd8SF1p\P!SUu.uF q3V,NK up4W&K3 V#uupiuupiuupiuupiuupi)rgbr8r9c3:<"TFqS^,9xK R#5i)Nr).0rrs& r )main.._collate..s1jE!H}jsFT)r, start_pixrrr;r<)r:r=r> wrist_rgbr?r@)rrall)rrxrr _past_keyssf r_collatemain.._collates&/A[[!61A$$!67CF/388%QZ==%8J uQx 'BG$H%Q'7%8%8%$HC !;AE!H}E%:EqddE%:;; 31j13331j1 1 1E%:EqddE%:;  "78$H&;&;sE E)E )E E ) batch_sizeshuffle num_workers pin_memory drop_last collate_fnz--da3 / --per_voxel / --hires_factorized / --hires_per_voxel / --hires_ablate / --hires_flat / --volumetric / --multiview are mutually exclusive,)DA3VolumeScene)DinoVolumeSceneHires)DinoVolumeScenePerVoxelHires)DinoVolumeSceneHiresAblate)DinoVolumeSceneHiresFlat)DinoVolumeSceneVolumetricz(--v4_da3 requires --multiview to be set.)DinoVolumeSceneV4DA3z(--v4_vlm requires --multiview to be set.)DinoVolumeSceneV4VLMz---v4_dynaflip requires --multiview to be set.)DinoVolumeSceneV4DynaFLIPz.--v4_paligemma requires --multiview to be set.)DinoVolumeSceneV4PaliGemmaz(--v4_pi0 requires --multiview to be set.)DinoVolumeSceneV4Pi0z$--v4 requires --multiview to be set.)DinoVolumeSceneV4z$--v3 requires --multiview to be set.)DinoVolumeSceneV3)"DinoVolumeSceneVolumetricMultiView)r% n_height_binsrfeat_dimheight_enc_dimvoxel_mlp_hiddend_model mlp_layersmlp_hidden_multr'r(rrrr2r%rMrrNrOrPrQrRrSr'r(rrr cls_fusioncross_view_layerscross_view_headsper_voxel_grip_rot temporal_headtemporal_cnn_layerstemporal_cnn_kerneltemporal_mlp_hiddenr* past_enc_dim cam_margin_mtask_text_default freeze_llmpaligemma_model_namepi0_safetensors_pathF_zF_tF_s) r%rMrrNrQn_blocks mlp_hiddenr'r() rr* past_z_delta past_use_uv past_uv_delta past_use_xyzpast_xyz_deltarR mlp_residualrSbackbone_layer_fusionz model = z (feat_dim=z, n_height_bins=modelz rot_head.z! --refit_rot_centroids: dropped z< rot_head keys from source ckpt (keeping fresh random init).c3`<"TF#pSV8H;'gSPV4xK% R#5iN startswith)r/prs& rr0main..s( RMqa!:!:1<<?!:Ms..z --refit_z_range: dropped z z-axis keys from source ckpt (u); bin-index → physical-Z mapping changed, keeping fresh random init for height_emb and the model's freshly-computed bin_centers_z buffer.z shape-skip z3 keys (n_window or n_height_bins probably changed)::NNz z: src u → dst )strictz load_state_dict missing keys: z# load_state_dict unexpected keys: z loaded model weights from c3F<"TFpSPV4xK R#5irprq)r/prens& rr0rts"M 1<<#4#4 s!z param groups: head=z (z.1eu→z ) backbone=z ) warmup=z schedule=paramslrnameheadlr_maxlr_minbackbonec(V^8dQhRRRRRRRR/#)ritrr~rrreturnr)rs"rrmain..__annotate__s(3urc<SP^8d2VSP8d!W^,,SP, #SPR8Xd\^SPSP, 4p\ R\RVSP, V, 44pVRW, ,R\ P !\ PV,4,,,#V#)r.rrr) lr_warmup lr_scheduler max_itersrbmathcospi)rr~rdenomprogressargss&&& r_lr_atmain.._lr_ats >>A "t~~"5!V$t~~5 5   x '4>>DNN:;E3C"t~~*=)F GHHC6?3sTXXdggPXFX=Y7YZZ Z r)projectr|configzwandb: z (disabled)u ckpts → c<SP'dRV9d/#RVR,PSPRR7RVR,PSPRR7RVR,PSPRR7RVR,PSPRR7RVR,PSPRR7/#)uzBuild past-feature kwargs for model.forward() — empty dict if --hires_ablate isn't set or no past keys are in the batch.past_zT non_blocking past_grippast_uvpast_xyz current_xyz) hires_ablatetodevicerwrs&r_past_kwmain.._past_kw's   HE$9I eHo((4(H {+..t{{.N uY'**4;;T*J j),,T[[t,L 5/224;;T2R   rc V^8dQhRRRR/#)rrwdictrr)rs"rrr4s  d t rc<SP'g/#RVR,PSPRR7P4RVR,PSPRR7P4/#)z9K_in, T_w2c kwargs the volumetric model needs at forward.K_inr=Trrr>) volumetricrrrrs&r_volumetric_kwmain.._volumetric_kw4sgI E'N%%dkk%EKKM U8_'' $'GMMO  r)volumetric_lossesc <.p.p.pS EFpVR8XdVPVR,PS PRR7P44VPVR,PS PRR7P44VPVR,PS PRR7P44KVR8XdVPVR,PS PRR7P44VPVR ,PS PRR7P44VPVR ,PS PRR7P44EK\ R V: 24h VR ,PS PRR7P4pS P ;'g S P pV'd3VR ,PS PRR7P4MRpR;rS P^8deVR,PS PRR7P4p VR,PS PRR7P4p S P 'd V!W#WHWR7#V!W&W4VWR7#)zyForward call for the multi-view volumetric model. Builds the (rgb, K, T_w2c) lists in `multi_views` order from the batch.rAr,Trr=r>rDr3r?r@zunknown view: r2r:Nrr)rrr:rr)rcrrr ValueErrorv3rW mv_past_n) rnrwrgb_listK_listT_listr~spwant_tgttgtrrr multi_viewss && r_multiview_callmain.._multiview_callBsXAG|e  $ O U U WX eGn// $/OUUWX eHo0040PVVXYgk 2 5 5dkkPT 5 U [ [ ]^ eJ/224;;T2RXXZ[ eN366t{{QU6V\\^_ >!!788;  " "4;;T " B H H J7755d55\"%%dkk%EKKM!% "! >>A k*--dkk-MSSUI8_'' $'GMMOF 777f$-> >X ): :r) v3_losses)multiview_lossesr,rr2)rrrr)rz+ step 0 viz logged (pre-training baseline)rrr;r<r:) set_to_nonez loss/totaliterit_per_szlr/groupzlatest.pth.tmpoptr model_kinddino_volume_scene_v4_da3dino_volume_scene_v4_vlmdino_volume_scene_v4_dynaflipdino_volume_scene_v4_paligemmadino_volume_scene_v4_pi0dino_volume_scene_v4dino_volume_scene_v3da3_volume_scenedino_volume_scene_hires_ablatedino_volume_scene_hires_flatdino_volume_scene_hiresdino_volume_per_voxel_hiresdino_volume_per_voxel&dino_volume_scene_volumetric_multiviewdino_volume_scene_volumetricdino_volume_scenez latest.pthz ckpt @ z total=z.4fz vol=z loss/volumez grip=z loss/gripz rot=zloss/rotz z.2fz it/s)flushgɿ)rrrrrr)z height_emb. bin_centers_z)z backbone.z dynaflip.zvlm.)argparseArgumentParser add_argumentstrrhomerrBooleanOptionalAction parse_args ckpt_rootr|mkdir init_fromrloadprint refit_z_rangerrrefit_rot_centroidsr data_rootr%r&rrrMr'r(grip_logrip_hirr* multiviewrrrr8r:rda3rhires_factorizedhires_per_voxel hires_flatrrrsplit lib.model_da3r?lib.model_hiresr@rArBrClib.model_volumetricrDv4_da3lib.model_volumetric_v4_da3rEv4_vlmlib.model_volumetric_v4_vlmrF v4_dynaflip lib.model_volumetric_v4_dynafliprG v4_paligemma!lib.model_volumetric_v4_paligemmarHv4_pi0lib.model_volumetric_v4_pi0rIv4lib.model_volumetric_v4rJrrrKlib.model_volumetric_multiviewrLrrrNv3_F_zv3_F_tv3_F_srrOrPrQrRrSrTrUrVrWrXrYrZr[mv_past_enc_dimr] vlm_task_textvlm_freeze_llmr`rarerfupdatergrhrirjrkrlrmrr__name__rrrrjany state_dictr%rcrload_state_dictnamed_parameters requires_gradid parametersr{rbackbone_lr_multrroptimAdamWruinit wandb_projectvarsrunurlrrrno_gradnextrrr time perf_counterr param_groupsr zero_gradbackwardrr`itemvis_every_itersckpt_every_iterssavereplace)Rrsckpt_dirinit_cksrc_z z_range_argrot_c_from_srcdsr!r6loader mutex_flagsr? model_clsr@rArBrCrDrErFrGrHrIrJrKrLmodel_feat_dim model_kwargsrnsrc_sdrdroppedr~ drop_prefixes dropped_zmodel_sd shape_skipped filtered_sd src_shape dst_shapemissing unexpected _BB_PREFIXESrybackbone_paramsbb_ids head_params head_lr_max head_lr_min bb_lr_max bb_lr_minrrrr _vol_lossesr _mv_lossespeekrgb0sp0out0rt_looprwr,rtgt_yxtgt_ztgt_gtgt_rpgrxtgt_xyzlossesnowitsrtmpr5rrsR ` ` @@@rmainrJs!ANN8dN+NN=4N0NN=#diikRN8NN?a34NN6tMONN:E4UVNN'eSEF NN=sAQRNN?J&1VXNNSN9NN8%ABNN8%45NN;UDN9NN;UCN8NN=sDLMNN'c3N?NN&S"N=NN:vN.NN=$JKNN=TUNN7<?@NN' FG NN&|JK NN>,OP NN>,HINN>8h =RGH NN(sAYZNN'c1N=NN6,NO NN:l78NN:l01NN?<01 NN#L:; NN:l01 NN+T>?NN+7#$ NN$.>FHNN%h.L.L @ANN#%34 NN6,45NN:CN4NN:CN4NN:CN4NN)(2P2PXYNN$j&7MNNN*aPRNN*aN@NN*cCENN=sAEFNN&S"IKNN=$OPNN%C56NN'c2LNNN$\TUNN*<@ANN#LMNNN:CACNN#LQRNN?<MONN$\IKNN#LCENN%lRTNN>QMONN#LSUNN&UCMONN,\FG <<>DDNN#dii/H NN4$N/ G ~~~**T^^%eT   01%fo.a1 2!!!M*95E#(q?E%(O DIty = >  -$))2GIItyy)  (@(@(@  /  G$7 2 3  $.. !T->->..4==   t/B/B**t||(D$#000 '+~~~T^^ B77BGGDIty))+KNJ(t%)%5%5$#'H>Ftxx,,d.B.B(($//T^^(<>?KQMN N154>>'',DK xxx0"   8(   @0   >. <, B- FFF{D( FFF{D(   KKK{N-   LLL{P. FFF{D( BBB{=% BBB{=% U6 + #  xxxDMMR'   $--2"5 DMMR/ T]]b0 T]]b(t{{2T[[@ ","DM ]],,nn#..!22LL 00....]]   t'7'7'74;L;L;L  ]] ,, nn  $   ..  "22 LL  !00  ..  .. ]]  "&  #44! ""22# $ $66% &,,' (!% 8 8) *!% 8 8+ ,!% 8 8- .>>/ 0--1 2**3 6 ;;;040B0BL, -)-)<)>/ 0--1 2**3 6   ]] ,, nn  LL    !00  ..  .. ]]  "&       >> --! $   ]] ,, nn  $   ..  "22 LL  !00  ..  .. ]]  "&  #44! ""22# $ $66% &,,' (!% 8 8) *!% 8 8+ ,!% 8 8- .>>/ 0--1 6]]$2D2Dnn~LL4==....   ]];;**((,,**..** 00"&"<"<    % % ( ( 5E Jy))*+n5EF--.a 12!  # # #"(F&QALL,Eqq&GF'-||~W~tq!Q\\+=Vdad~FW 5c'l^DGG H     =M$*TFq S RM RSSS RM RRFIT'-||~L~tq!)9Kdad~FL /I/?@''0k2KK L##%  LLNDAqH}!!2!2agg!=$$aqwwx{?P?P9Q%RS!" A #  M#m"4!5655 6+8+<+<'9iQCvi[ )EF,=#33K3N 4333 4WR[MB C  7 27GH I ,T^^,<=>6L,,..da ?? "s"M "Msss"M "MM .- -_be_F -#..0?0oo*,Q%v*=10K?''K!%!8$++dggK$///Id333I !#k"2!32k#5Fc+VYIZ[/*+2i_C #ONN#:d.>.>-? AB ++   ;D+vv ;+ 7 ?D)vz 9) 5 C JJt)) $t*M GUYYYEIIMMLA BC Jxj !"     I  :D wwwC Q  DL!E{~~dkk~=;""4;;T"B "5$/DsMhtnMt8LMD *T4$--'+~~'`'`9M9M'`'`QUQ`Q`hlhqhqx|yByBCJK L ;<  D    F  E,//$++D/AC{#&&t{{&FB;'**4;;T*JF*%((4(HE-(++DKKd+KE,'**4;;T*JE&&!$8 blC4'%eU3 -0040PVVX777'WeUCF'VUE5'RFCPxP.:OP*#L144T[[t4TZZ\G(feUE7SF'VUE5IF MMdM + < ) ) + HHJ AID##%Cv&CF4S1C&&79$xc"&&1234'((+D1CI 1A'd***a/ :eS4==26..2k2kDDXDX2k2k\`\k\k,0IIDIIGH IIc %d+++q0!11 U--/3>>+DDJ!; 6:kkk2;?;;;7@D@P@P@P:O:O:O6>B>R>R>R:8<4IME?C;0-./0 H|34 $x|0D0I0I0KC/PQ#M2779#>?$[1668=>#J/446s;2c#YeM! " t~~%] SGWTLP.?D sV$BJWBJWBJ!W>BJ!X:BJ'Y BJ'Y!BJ'ZBJ'Z BJ,Z-BJ,aBJ2a, BJ2a7BJ2bBJ2b0BJ2b=BJ8c'BJ=dBJ=dBJ=l=CBKpBKpr^s*#   '3tH~--/6678*JJ,L13/ aHcLNbs l zFr