o = j@sdZddlZddlZddlZdD] ZeeZeeje<qddejd_ddejd_ ddl Z ddl m Z dd l mZd Zd Zd ZGd dde jZGddde jZedkre e jrbdndZeeZeddeDZedede ddeeeZ e ! ee Z"Wdn1swYe"#D]@\Z$Z%e&e%drede$de'e%j(qe)e%e*e'frede$de+e%de%re&e%ddrede'e%dj(qej,dkred e j-d!d"d#dSdSdS)$uDA3-based pixel-aligned heatmap predictor. Wraps Depth-Anything-3 (DA3-SMALL). Repurposes the DualDPT head: - main head: 1 ch depth + 1 ch conf — kept, supervised by frozen-DA3 depth predictions. - aux head: original 7 + 1 ch — REPLACED with (N_WINDOW + 1) channels so the DualDPT split (`aux_pred=[..., :-1]`, `aux_conf=[..., -1]`) yields N_WINDOW heatmap channels and 1 unused conf channel. We bypass DA3's `_process_camera_estimation` (which DELETES `ray`/`ray_conf` from the output) and call `backbone` + `head` directly. Output of `forward(rgb)`: pred_heatmap: (B, N_WINDOW, H_out, W_out) raw logits pred_depth: (B, H_out, W_out) dino_feats: list of (B, n_patches, C) intermediate DINO features (for PCA viz) N)depth_anything_3.utils.export!depth_anything_3.utils.pose_aligncOdSNakrr,/data/cameron/para/libero/model_da3_pixel.pyr rcOrrrrrrr r r r)DepthAnything3z/data/cameron/da3_weightsics8eZdZdZdefdedeffdd ZddZZS) FiLMHeaduJPer-timestep FiLM modulation + per-timestep 1×1 conv → 1 logit per (pixel, t). Input: (B, in_dim, H, W) shared per-pixel features (from DA3 DPT aux pre-final layers) Output: (B, N_WINDOW + 1, H, W) where the last channel is a dummy conf (DualDPT splits it off). Heatmaps live in channels [:N_WINDOW]. in_dimn_windowcsxt||_||_tt|||_tt |||_ tj |ddd|_ tj |j jtj j|j jdddS)N) kernel_size{Gz?std)super__init__rrnn ParametertorchonesscalezerosshiftConv2dconvinitzeros_biasnormal_weight)selfrr __class__rr r)s zFiLMHead.__init__c Cs|j\}}}}|d|jdddddddf|jdddddddf}|||j|||}||}|||j||}tj|d|||j |j d}tj ||gddS)Nr)devicedtype)dim) shape unsqueezerr reshaperr"rrr+r,cat) r(xBCHWZx_toutZ dummy_confrrr forward7s zFiLMHead.forward) __name__ __module__ __qualname____doc__N_WINDOWintrr8 __classcell__rrr)r r!srcs6eZdZeedfdedeffdd ZddZZ S) DA3PixelModelNr weights_pathcst||_||_t|}|jj|_|jj|_~|jdur-t t |jdgd|_ nt |j|_ |jj j d}|d}tj|j|ddddd}tj|jtjj|jdd||d<dS) NZ out_layers) rr)rstridepaddingrr)rrrZ_user_feat_layersr from_pretrainedmodelbackboneheadlistgetattrdino_feat_layersscratchZoutput_conv2_auxrr! in_channelsr#r$r%r&r')r(rrArOfullZ last_aux_seqZold_convZnew_convr)rr rIs       zDA3PixelModel.__init__cCs|d}tjr tjntj}tj|jj|d|j |d|j dd\}}Wdn1s0wY|j d|j d}}tj|jjdd |j |||d d }Wdn1s]wY|d } | d krv| ddd d f} n| dkr| ddd f} n tdt| j |d} | d kr| ddd fd ddd} n| dkr| d ddd} n tdt| j g} |D]}t|ttfr| |d q| |q| | | dS)z rgb: (B, 3, 504, 504) in [0, 1].r) device_typer,NZsaddle_balanced)Z cam_tokenZexport_feat_layersZref_view_strategyrFF)rSenabledr)Zpatch_start_idxdepthrBzunexpected depth shape rayzunexpected ray shape ) pred_heatmap pred_depth dino_feats)r/rcudais_bf16_supportedbfloat16float16autocastr+typerKrOr.rLr- RuntimeErrortuplepermute isinstancerMappend)r(rgbr2Zautocast_dtypeZfeatsZ_auxr5r6Zhead_outrVr\rXr[r]Z layer_featsrrr r8jsD       zDA3PixelModel.forward) r9r:r;r= DA3_WEIGHTSr>strrr8r?rrr)r r@Hs !r@__main__r^cpuccs|] }|jr|VqdSr) requires_gradnumel).0prrr srrz Trainable: ,rZrYr.z z: z: list()z first: zpeak: geAz.2fz GB).r<systypesosn ModuleTypemmodulesexportZalign_poses_umeyamartorch.nnrZdepth_anything_3.apir rjZ DA3_INPUTr=Modulerr@r9r+r^ is_availabletoevalsum parametersZn_tprintrandrino_gradr7itemsr vhasattrrer.rgrMlenrcmax_memory_allocatedrrrr sD  'S   *