o iD @sddlmZddlmZmZmZddlZddlmZddl mZddl m Z ddl m Z mZmZmZGdddejZ   dd ed eeefd edededejf ddZ ddeededededejf ddZGdddejZGdddejZdS))Dict)ListSequenceTupleN) rearrange)Permutecreate_uv_gridcustom_interpolateposition_grid_to_embedc#sXeZdZdZddddddddd d d d dd dddededededededeededededededededededd f"fd!d"Z #d9d$e e j d%ed&ed'ed(ede f d)d*Z d$e e j d%ed&ed'edeee j ff d+d,Zd$e e j de j fd-d.Z /d:d0e j dede j fd1d2Zd0e j de j fd3d4Zd;d0e j d&ed%ed6ede j f d7d8ZZS)<DPTaA DPT for dense prediction (main head + optional sky head, sky always 1 channel). Returns: - Main head: * If output_dim>1: { head_name, f"{head_name}_conf" } * If output_dim==1: { head_name } - Sky head (if use_sky_head=True): { sky_name } # [B, S, 1, H/down_ratio, W/down_ratio] expexpp1)rirFdepthTskyreluidt) patch_size output_dim activationconf_activationfeatures out_channels pos_embed down_ratio head_name use_sky_headsky_namesky_activationuse_ln_for_heads norm_typefusion_block_inplacedim_inrrrrrrrrrrr r!r"r#r$returnNc sHt||_||_||_||_| |_| |_| |_||_ |dk|_ | |_ | |_ d|_ |dkr6t|_n|dkr@t|_ntd|dtfdd|D|_ttj|d |d d d d d tj|d|dd d d d ttj|d |d d d dd g|_tt||dd|_t||d|j_t||d|j_t||d|j_t|d|d|j_|}d}tj||d d ddd |j_ |rt!dt|t!dgng}tj"tj|d |d ddd g|tj#ddtj||ddd d R|j_$|j r"tj"tj|d |d ddd g|tj#ddtj|dddd d R|j_%dSdS)Nr )rr layerrzUnknown norm_type z, should be 'layer' or 'idt'.c s g|] }tj|ddddqS)r r kernel_sizestridepadding)nnConv2d).0ocr%8/data/cameron/da3_repo/src/depth_anything_3/model/dpt.py _s z DPT.__init__..rr*r'r(F)expandinplace) has_residualr9 )rr'r(r )rr(r r'T)&super__init__rrrrr head_mainr out_dimhas_confrr!intermediate_layer_idxr. LayerNormnormIdentity Exception ModuleListprojectsConvTranspose2dr/ resize_layers _make_scratchlistscratch_make_fusion_block refinenet1 refinenet2 refinenet3 refinenet4 output_conv1r SequentialReLU output_conv2sky_output_conv2)selfr%rrrrrrrrrrr r!r"r#r$head_features_1head_features_2ln_seq __class__r2r4r=*s        z DPT.__init__featsHWpatch_start_idx chunk_sizec  sL|ddj\fdd|D}i}d|vr)|dt|ddi|dus1|krM|j||||fi|}fdd|D}t|Sgtd|D]3t|i} d|vrq| d|di|jfd d|D|||fi| qUfd dd D}fd d|D}t|S) a Args: feats: List of 4 entries, each entry is a tensor like [B, S, T, C] (or the 0th element of tuple/list is that tensor). H, W: Original image dimensions patch_start_idx: Starting index of patch tokens in sequence (for cropping non-patch tokens) chunk_size: Chunk size along time dimension S Returns: Dict[str, Tensor] rcs"g|] }|dqS)r)reshape)r0feat)BCNSr3r4r5s"zDPT.forward..imageszB S ... -> (B S) ...Nc0i|]\}}||jg|jddRqSr Nviewshaper0kvrerhr3r4 0zDPT.forward..csg|]}|qSr3r3)r0f)s0s1r3r4r5scs*i|]tjfddDddqS)csg|]}|qSr3r3)r0odrpr3r4r5sz*DPT.forward...r)dim)torchcat)r0) out_dictsryr4rss*crjrkrlrorrr3r4rsrt) rnupdater _forward_implitemsrrangeminappendkeys) rWr^r_r`rarbkwargs extra_kwargsout_dictkwr3)rerfrgrhr}rvrwr4forwards*&z DPT.forwardcCs|dj\}}}||j||j}} g} t|jD]A\} } || dd|df} || } | ddd|||| } |j| | } |j rO| | ||} |j | | } | | q| | }t||j|j}t| |j|j}|j|}t|||fddd}|j r| |||}|}|j|}i}|jr|dddd}||ddd f|j}||d |j}|d||j<|d||jd <n |||jd||j<|jr|j|}||d||j<|S) Nrr'r bilinearTmode align_cornersr(.).r_conf)rnr enumeraterArCpermute contiguousrcrGr_add_pos_embedrIr_fuseintrrLrRr rUr@_apply_activation_singlerrsqueezer>rrV_apply_sky_activationr )rWr^r_r`rare_rfphpw resized_feats stage_idxtake_idxxfusedh_outw_outrd main_logitsoutsfmappredconfZ sky_logitsr3r3r4rsJ      zDPT._forward_implc Cs|\}}}}|j|}|j|}|j|}|j|} |jj| |jddd} |jj| ||jddd} |jj| ||jddd} |j | |} | S)zf 4-layer top-down fusion, returns finest scale features (after fusion, before neck1). r'Nsize) rL layer1_rn layer2_rn layer3_rn layer4_rnrQrnrPrOrN) rWr^l1l2l3l4l1_rnl2_rnl3_rnl4_rnoutr3r3r4r s     z DPT._fuselinearrcCst|tr |n|}|dkrt|S|dkrt|dS|dkr(t|S|dkr1t|S|dkr:t|S|dkrEtjj |S|dkrNt |S|S) z Apply activation to single channel output, maintaining semantic consistency with value branch in multi-channel case. Supports: exp / relu / sigmoid / softplus / tanh / linear / expp1 rrr expm1rsigmoidsoftplustanh) isinstancestrlowerr{rrrrr. functionalrr)rWrractr3r3r4rs      zDPT._apply_activation_singlecCsDt|jtr |jn|j}|dkrt|S|dkr t|S|S)z Sky head activation (fixed 1 channel): * 'sigmoid' -> Sigmoid probability map * 'relu' -> ReLU positive domain output * 'linear' -> Original value (logits) rr)rr!rrr{rr)rWrrr3r3r4r7s   zDPT._apply_sky_activation皙?ratiocCsp|jd|jd}}t|||||j|jd}t||jd|}|dddd|jdddd}||S)z:Simple UV position encoding directly added to feature map.r) aspect_ratiodtypedevicer r'rN)rnrrrr rr7)rWrr`r_rrrper3r3r4rJs $zDPT._add_pos_embed)r])r)r)__name__ __module__ __qualname____doc__rrrboolr=rr{TensorrrTyDictrrrrfloatr __classcell__r3r3r[r4r s    { 0  = ,r Tr Frrr:groupsr9r&c Cs"t|tj|ddddd|||d S)Nr8FT) rrdeconvbnr7rrr:r)FeatureFusionBlockr.rT)rrr:rr9r3r3r4rMVs rMin_shape out_shaper7c Cst}|}||r dnd}||rdnd}||rdnd}tj|d|dddd|d|_tj|d|dddd|d|_tj|d|dddd|d|_tj|d|dddd|d|_|S) Nr'r r6r]rr(Fbiasr)r.Moduler/rrrr) rrrr7rLc1c2c3c4r3r3r4rJjsrJc sPeZdZdZddedejdededdf fd d Zd e j de j fd d Z Z S)ResidualConvUnitz1Lightweight residual convolution block for fusionr rrrrr&Nc slt||_||_tj||dddd|d|_tj||dddd|d|_d|_d|_ ||_ tj |_ dS)Nr(r Tr)r<r=rrr.r/conv1conv2norm1norm2r quantizedFloatFunctionalskip_add)rWrrrrr[r3r4r=~s zResidualConvUnit.__init__rcCs^||}||}|jdur||}||}||}|jdur(||}|j||S)N)rrrrrradd)rWrrr3r3r4rs        zResidualConvUnit.forward)r ) rrrrrr.rrr=r{rrrr3r3r[r4r{s& rcseZdZdZ       ddedejded ed ed ed eeefd ededdffdd Z ddde j d eeefde j fddZ Z S)rzOTop-down fusion block: (optional) residual merge + upsampling + 1x1 contractionFTNr rrrrr7rrr:rr&c st||_||_||_|rt|||| dnd|_t|||| d|_|r*|dn|} tj || dddd| d|_ tj |_ dS)N)rr'r rTr)r<r=rrr:r resConfUnit1 resConfUnit2r.r/out_convrrr) rWrrrrr7rrr:r out_featuresr[r3r4r=s zFeatureFusionBlock.__init__rxscGs|d}|jrt|dkr|jdur|j|||d}||}|dur1|jdur1ddi}n|dur;d|ji}nd|i}t|fi|d|jd}| |}|S) z xs: - xs[0]: Top branch input - xs[1]: Lateral input (can do residual addition with top branch) rr N scale_factorr'rrr) r:lenrrrrrr rr)rWrryZ up_kwargsr3r3r4rs    zFeatureFusionBlock.forward)FFFTNTr )rrrrrr.rrrr=r{rrrr3r3r[r4rs>    0r)NTr F)r F)typingrrrrrr{torch.nnr.addicteinopsr'depth_anything_3.model.utils.head_utilsrrr r rr rrrMrJrrr3r3r3r4sR    ;