o c j+&@sdZddlZddlZddlZddlZdD] ZeeZeeje<qddejd_ ddejd_ d dejd_ ddl Z ddl mZdd lmZd Zd Zd ZdZdZdZdZGdddejZedkre e jrndndZeeZe dde!DZ"e#de"de $ddeeeZ%e & ee%Z'Wdn1swYe'(D]>\Z)Z*e+e*dre#de)de,e*j-qe.e*e/re#de)de0e*de*re+e*ddre#d e,e*dj-qej1dkr e#d!e j2d"d#d$dSdSdS)%uzDA3-based volume model with factored KV-attention. Architecture (Cameron 2026-05-18 spec): - DA3 backbone + DPT head; aux head's final 1×1 conv replaced to output KEY_DIM (+1 conf). - Per-pixel feature F ∈ R^(B × KEY_DIM × H × W) — "value/query" stream. - Learnable height embeddings h_emb ∈ R^(N_HEIGHT_BINS × H_DIM). - Learnable time embeddings t_emb ∈ R^(N_WINDOW × T_DIM). - Key per (t, z): key(t, z) = concat(t_emb[t], h_emb[z]) ∈ R^(T_DIM + H_DIM = KEY_DIM). - Volume logits via bilinear scoring: l(b, t, z, u, v) = F(b, :, u, v) · key(t, z) / sqrt(KEY_DIM). We replace the original libero (B, N_WINDOW * N_HEIGHT_BINS, H, W) dense head with the factored KV decomposition: parameter-efficient, structural inductive bias that height/time are categorical attributes that share representation across spatial locations. forward(rgb) returns: volume_logits: (B, N_WINDOW, N_HEIGHT_BINS, h_out, w_out) pred_depth: (B, H, W) — for distillation against frozen DA3 depth dino_feats: list of intermediate DINO features (for PCA viz) pixel_feats: (B, KEY_DIM, h_out, w_out) — for debugging / viz N)depth_anything_3.utils.export!depth_anything_3.utils.pose_aligncOdSNakrr0/data/cameron/para/libero/model_da3_volume_v3.pyr rcOrrrrrrr r r rcOrrrrrrr r r )DepthAnything3z/data/cameron/da3_large_weightsi 0c sbeZdZeeeeeedfde de de de de de f fdd Z e d d Z d d ZddZZS)DA3VolumeModelN weights_pathn_window n_height_binskey_dimtime_dim height_dimc s~t||kr ||ksJ||_||_||_||_||_t|}|j j |_ |j j |_ ~|dur?t t |j dgd|_nt ||_|j jjd} | d} tj| j|ddddd} tj| jtjj| jdd| | d<t|||_t|||_tjj|jjd d dd tjj|jjd d dd |jd |||d d|jd|||d dt||_ t!t"#d|_$dS)NZ out_layers) r) kernel_sizestridepaddingg?)stdg{Gz?g)r!rbt_sinF) persistenth_singHzG@)%super__init__rrrrrr from_pretrainedmodelbackboneheadlistgetattrdino_feat_layersscratchZoutput_conv2_auxnnConv2d in_channelsinitzeros_biasnormal_weight Embeddingh_embt_emb trunc_normal_register_buffer_sinusoidal_features LayerNorm pixel_norm Parametertorchtensor logit_scale) selfrrrrrrr.fullZ last_aux_seqZold_convZnew_conv __class__rr r'3s@      zDA3VolumeModel.__init__cCs|ddksJ|d}tj|tjdt|dd}dtj|tjd}|d|dtj}tjt|t |gddS)zNeRF/transformer-style sinusoidal positional encoding. Returns (n_values, dim) where each row is sin/cos at log-spaced freqs. Normalize position to [0, 1] so all n_values fit one cycle base. r)dtyperg@r)dim) rAarangefloat32max unsqueezemathpicatsincos)n_valuesrJLposZfreqsanglesrrr r=fs z#DA3VolumeModel._sinusoidal_featurescCs0|j|jjd}|j|jjd}||S)uv3: sinusoidal (fixed) + small learned embedding, SUMMED across t and z. key(t, z) = (t_sin[t] + t_emb[t]) + (h_sin[z] + h_emb[z]) ∈ R^key_dim. rr)r#r:r7rNr%r9)rDZt_totalZh_totalrrr _build_keystszDA3VolumeModel._build_keyscCsZ|d}tjr tjntj}tj|jj|d|j |d|j dd\}}Wdn1s0wY|j d|j d}}tj|jjdd |j |||d d }Wdn1s]wY|d } | d krv| ddd d f} n| dkr| ddd f} n tdt| j |d} | d kr| ddd fd ddd} n| dkr| d ddd} n tdt| j | j \} }}}|| d dddd ddd}||jdddd}|}||jdddd}|jjtdd}td|||}g}|D]}t|ttfr||d q ||q || | |dS)z rgb: (B, 3, 504, 504) in [0, 1].r) device_typerINZsaddle_balanced)Z cam_tokenZexport_feat_layersZref_view_strategyrF)rYenabledr)Zpatch_start_idxdepthrzunexpected depth shape rayrHzunexpected ray shape T)rJkeepdimgư>gY@)rMzbchw, tzc -> btzhw) volume_logits pred_depth pixel_feats dino_feats)rNrAcudais_bf16_supportedbfloat16float16autocastdevicetyper*r.shaper+rJ RuntimeErrortuplepermuter?normrXrCclamprOlogexpeinsum isinstancer,append)rDrgbxZautocast_dtypeZfeatsZ_auxHWZhead_outr\rbr^rcBZCfZHfZWfZf_lnZf_unitkeysZ keys_unitscalerardZ layer_featsrrr forward|sT      "zDA3VolumeModel.forward)__name__ __module__ __qualname__DA3_WEIGHTS_DEFAULTN_WINDOW N_HEIGHT_BINSKEY_DIMTIME_DIM HEIGHT_DIMstrintr' staticmethodr=rXr~ __classcell__rrrFr r2s(3  r__main__recpuccs|] }|jr|VqdSr) requires_gradnumel).0prrr srz Trainable: ,rHr_rlz z: z: list()z first: zpeak: geAz.2fz GB)3__doc__systypesosrOn ModuleTypemmodulesexportZalign_poses_umeyamaZbatch_align_poses_umeyamarAtorch.nnr0Zdepth_anything_3.apir rZ DA3_INPUTrrrrrModulerrrjre is_availabletoevalsum parametersZn_tprintrandrwno_gradoutitemsr vhasattrrnrlrur,lenrkmax_memory_allocatedrrrr sN         *