o  ja@s,dZddlZddlZddlZddlZdD] ZeeZeeje<qddejd_ ddejd_ d dejd_ ddl Z ddl mZdd lmZd Zd Zd ZdZdZdZdZeeeksaJdGdddejZedkre e jrxdndZeeZe dde!DZ"e#de"de $ddeeeZ%e & ee%Z'Wdn1swYe'(D]>\Z)Z*e+e*dre#de)de,e*j-qe.e*e/re#de)d e0e*d!e*re+e*ddre#d"e,e*dj-qej1dkre#d#e j2d$d%d&dSdSdS)'uzDA3-based volume model with factored KV-attention. Architecture (Cameron 2026-05-18 spec): - DA3 backbone + DPT head; aux head's final 1×1 conv replaced to output KEY_DIM (+1 conf). - Per-pixel feature F ∈ R^(B × KEY_DIM × H × W) — "value/query" stream. - Learnable height embeddings h_emb ∈ R^(N_HEIGHT_BINS × H_DIM). - Learnable time embeddings t_emb ∈ R^(N_WINDOW × T_DIM). - Key per (t, z): key(t, z) = concat(t_emb[t], h_emb[z]) ∈ R^(T_DIM + H_DIM = KEY_DIM). - Volume logits via bilinear scoring: l(b, t, z, u, v) = F(b, :, u, v) · key(t, z) / sqrt(KEY_DIM). We replace the original libero (B, N_WINDOW * N_HEIGHT_BINS, H, W) dense head with the factored KV decomposition: parameter-efficient, structural inductive bias that height/time are categorical attributes that share representation across spatial locations. forward(rgb) returns: volume_logits: (B, N_WINDOW, N_HEIGHT_BINS, h_out, w_out) pred_depth: (B, H, W) — for distillation against frozen DA3 depth dino_feats: list of intermediate DINO features (for PCA viz) pixel_feats: (B, KEY_DIM, h_out, w_out) — for debugging / viz N)depth_anything_3.utils.export!depth_anything_3.utils.pose_aligncOdSNakrr-/data/cameron/para/libero/model_da3_volume.pyr rcOrrrrrrr r r rcOrrrrrrr r r )DepthAnything3z/data/cameron/da3_large_weightsi 0z(TIME_DIM + HEIGHT_DIM must equal KEY_DIMc sVeZdZeeeeeedfde de de de de de f fdd Z d d Z d d Z ZS)DA3VolumeModelN weights_pathn_window n_height_binskey_dimtime_dim height_dimc s8t|||ks J||_||_||_||_||_t|}|j j |_ |j j |_ ~|dur=t t |j dgd|_nt ||_|j jjd} | d} tj| j|ddddd} tj| jtjj| jdd| | d<t|||_t|||_tjj|jjd d d d tjj|jjd d d d t||_dS) NZ out_layers) r) kernel_sizestridepaddingg?)stdg?gɿg?)r"rb)super__init__rrrrrr from_pretrainedmodelbackboneheadlistgetattrdino_feat_layersscratchZoutput_conv2_auxnnConv2d in_channelsinitzeros_biasnormal_weight Embeddingh_embt_emb trunc_normal_ LayerNorm pixel_norm) selfrrrrrrr,fullZ last_aux_seqZold_convZnew_conv __class__rr r%,s2     zDA3VolumeModel.__init__cCsR|j}|j}|jjd|||j}|jjd|||j}t j ||gddS)zjReturns keys: (n_window, n_height_bins, key_dim). key(t, z) = concat(t_emb[t], h_emb[z]). rrr)dim) rrr8r5 unsqueezeexpandrr7rtorchcat)r<TZZt_eZh_errr _build_keysUs zDA3VolumeModel._build_keyscCs"|d}tjr tjntj}tj|jj|d|j |d|j dd\}}Wdn1s0wY|j d|j d}}tj|jjdd |j |||d d }Wdn1s]wY|d } | d krv| ddd d f} n| dkr| ddd f} n tdt| j |d} | d kr| ddd fd ddd} n| dkr| d ddd} n tdt| j | j \} }}}|| d dddd ddd}|}dt|j}td|||}g}|D]}t|ttfr||d q||q|| | |dS)z rgb: (B, 3, 504, 504) in [0, 1].r) device_typedtypeNZsaddle_balanced)Z cam_tokenZexport_feat_layersZref_view_strategyrF)rHenabledr)Zpatch_start_idxdepthrzunexpected depth shape rayzunexpected ray shape g?zbchw, tzc -> btzhw) volume_logits pred_depth pixel_feats dino_feats)rArCcudais_bf16_supportedbfloat16float16autocastdevicetyper(r,shaper)r@ RuntimeErrortuplepermuter;rGmathsqrtreinsum isinstancer*append)r<rgbxZautocast_dtypeZfeatsZ_auxHWZhead_outrLrRrNrSBZCfZHfZWff_normkeysscalerQrTZ layer_featsrrr forward_sP      " zDA3VolumeModel.forward)__name__ __module__ __qualname__DA3_WEIGHTS_DEFAULTN_WINDOW N_HEIGHT_BINSKEY_DIMTIME_DIM HEIGHT_DIMstrintr%rGrm __classcell__rrr>r r+s$) r__main__rUcpuccs|] }|jr|VqdSr) requires_gradnumel).0prrr srz Trainable: ,rPrOr\z z: z: list()z first: zpeak: geAz.2fz GB)3__doc__systypesosr`n ModuleTypemmodulesexportZalign_poses_umeyamaZbatch_align_poses_umeyamarCtorch.nnr.Zdepth_anything_3.apir rq DA3_INPUTrrrsrtrurvModulerrnrZrU is_availabletoevalsum parametersZn_tprintrandreno_gradoutitemsr vhasattrr^r\rcr*lenr[max_memory_allocatedrrrr sN    l    *