h 8ddlZddlmZddlmcmZddlmZmZddl m Z m Z m Z m Z mZmZdZddZGddej$ZGdd ej$Zdd ZGd d ej$ZGd dej$Zy)N) rearrangerepeat)UnionTupleIterableListOptionalDictc.t|tr|S||fS)N) isinstancetuple)ts D/home/cameronsmith/repos/controll3r/dust3r/croco/models/dpt_block.pypairrs1e$101a&0c tj}|}|}|}|}|dk(r|}|dz}|dz}|dz}tj|d|dddd| |_tj|d|dddd| |_tj|d|dddd| |_tj|d|dddd| |_tj|j|j|j |j g|_|S) NTrF kernel_sizestridepaddingbiasgroups) nnModuleConv2d layer1_rn layer2_rn layer3_rn layer4_rn ModuleListlayer_rn) in_shape out_shaperexpandscratch out_shape1 out_shape2 out_shape3 out_shape4s r make_scratchr/s9iikGJJJJ ~ ] ] ]    G   G   G   G}} &G Nrc(eZdZdZfdZdZxZS)ResidualConvUnit_customzResidual convolution module.c t|||_d|_t j ||ddd|j |j|_t j ||ddd|j |j|_|jdk(r4t j||_ t j||_ ||_ tjj|_y)KInit. Args: features (int): number of features rrrTN)super__init__bnrrr conv1conv2 BatchNorm2dbn1bn2 activation quantizedFloatFunctionalskip_add)selffeaturesr<r6 __class__s rr5z ResidualConvUnit_custom.__init__Rs  YY  WW;;  YY  WW;;   77d?~~h/DH~~h/DH$ 446 rc|j|}|j|}|jdk(r|j|}|j|}|j |}|jdk(r|j |}|j dkDr|j|}|jj||S)znForward pass. Args: x (tensor): input Returns: tensor: output Tr) r<r7r6r:r8r;r conv_merger?add)r@xouts rforwardzResidualConvUnit_custom.forwardysooa jjo 77d?((3-Cooc"jjo 77d?((3-C ;;?//#&C}}  a((r__name__ __module__ __qualname____doc__r5rH __classcell__rBs@rr1r1Os&%7N)rr1c4eZdZdZ dfd ZdZxZS)FeatureFusionBlock_customzFeature fusion block.c ltt| ||_||_||_d|_||_|}|jdk(r|dz}tj||ddddd|_ t||||_ t||||_ tjj|_y)r3rTrrrN)r4rQr5 width_ratiodeconv align_cornersrr)rr out_convr1 resConfUnit1 resConfUnit2r=r>r?) r@rAr<rTr6r)rUrS out_featuresrBs rr5z"FeatureFusionBlock_custom.__init__s '79& *   ;;$ #q=L    4Hj"M3Hj"M 446 rc|d}t|dk(rs|j|d}|jdk7r4tj||j d|j dfd}|j j||}|j|}|jdk7r|j d|j dz d|jzkrd|j dz}n(t|jdz|j dz}tj|d|j dz|fd}n-tjj |dd|j}|j|}|S) zBForward pass. Returns: tensor: output rrrrbilinear)sizemodegUUUUUU? scale_factorr]rU)lenrWrSF interpolateshaper?rErXintr functionalrUrV)r@xsoutputresrcs rrHz!FeatureFusionBlock_custom.forwardsW A r7a<##BqE*C1$mmCv||A Q.PWab]]&&vs3F""6*   q  Q&,,q/1et?O?O5OOFLLO+D,,q06<<?BCmmF!V\\!_2De1LS]^F]]..vA#43E3E/GFv& r)FFFTrrIrOs@rrQrQs"  (7TrrQc Lt|tjdd|dd|S)NFT)rTr6r)rUrS)rQrReLU)rAuse_bnrSs rmake_fusion_blockrls, $   rc*eZdZdZdfd ZdZxZS) InterpolatezInterpolation module.ctt| tjj |_||_||_||_ y)zqInit. Args: scale_factor (float): scaling mode (str): interpolation mode N) r4rnr5rrerbinterpr_r]rU)r@r_r]rUrBs rr5zInterpolate.__init__s: k4)+mm// ( *rcl|j||j|j|j}|S)zyForward pass. Args: x (tensor): input Returns: tensor: interpolated data r^)rpr_r]rU)r@rFs rrHzInterpolate.forwards: KK **,,  r)FrIrOs@rrnrns +rrnceZdZdZddddgdgdddd d d df d ed edeeeeeffdeede ede ededede de edeffd Z ddZ dZde ej fdZxZS)DPTOutputAdapteraDPT output adapter. :param num_cahnnels: Number of output channels :param stride_level: tride level compared to the full-sized image. E.g. 4 for 1/4th the size of the image. :param patch_size_full: Int or tuple of the patch size over the full image size. Patch size for smaller inputs will be computed accordingly. :param hooks: Index of intermediate layers :param layer_dims: Dimension of intermediate layers :param feature_dim: Feature dimension :param last_dim: out_channels/in_channels for the last two Conv2d when head_type == regression :param use_bn: If set to True, activates batch norm :param dim_tokens_enc: Dimension of tokens coming from encoder r)rgb)rr )`i FN regression num_channels stride_level patch_size main_taskshooks layer_dims feature_dimlast_dimrkdim_tokens_enc head_typec t|||_||_t ||_||_||_||_||_ | | t|j znd|_ | |_ td|j d|z|_td|j d|z|_t!||dd|_t%|| | |j"_t%|| | |j"_t%|| | |j"_t%|| | |j"_|jdk(rt/j0t/j2||dzdddt5dd d t/j2|dz|dddt/j6d t/j2||jddd|_n|jd k(rt/j0t/j2||ddd | rt/j:|nt/j<t/j6d t/j>ddt/j2||jdt5dd d |_n tAd|j|jC| yy)NrrF)rr)r}rr)rrrr[Tr^semseg)rrrg?)rz/DPT head_type must be "regression" or "semseg".)r)"r4r5r~rrrrrrrr`rrmaxP_HP_Wr/r*rl refinenet1 refinenet2 refinenet3 refinenet4r Sequentialr rnrjheadr9IdentityDropout ValueErrorinit)r@r~rrrrrrrrkrroutput_width_ratiokwargsrBs rr5zDPTOutputAdapter.__init__sE ((z*$ $&GUGans4??/CCgk"q$//!, <=q$//!, <=#J AeT "3KI["\ "3KI["\ "3KI["\ "3KI["\  >>\ )  +{a'7QqZ[\4P +*H!AWXY  (D$5$51QXYZ DI^^x '  +{1SXY/5{+2;;=  3& +t'8'8aH4P DINO O    * II^I 4 +rc t|trd|gz}|Dcgc]}|t|jzc}|_t j t j|jd|jddddt j|jd|jddddddd|_ t j t j|jd|jddddt j|jd|jddddddd|_ t j t j|jd|jdddd|_ t j t j|jd|jddddt j|jd|jdddd|_ t j|j|j|j|jg|_y cc}w) z Initialize parts of decoder that are dependent on dimension of encoder tokens. Should be called when setting up MultiMAE. :param dim_tokens_enc: Dimension of tokens coming from encoder rrr) in_channels out_channelsrrrT)rrrrrrdilationrrrN)r rdr`rrrrr rConvTranspose2dact_1_postprocessact_2_postprocessact_3_postprocessact_4_postprocessr%act_postprocess)r@rdts rrzDPTOutputAdapter.initUs nc *.!11NCQRRrC$88R!# II //2!__Q/a     OOA.!__Q/aAa   " "$ II //2!__Q/a     OOA.!__Q/aAa   " "$ II //2!__Q/a " "$ II //2!__Q/a  II OOA.!__Q/a  "  "}}  " "  " "  " "  " " .  gSsIclg}|j|ddddftj|d}|S)N)dim)appendtorchcat)r@encoder_tokensrFs r adapt_tokenszDPTOutputAdapter.adapt_tokenss1  1%& IIaR rrc t|jJd|\}}||j|jzz}||j|jzz}|jDcgc]}|| }}|D cgc]} |j | }} |D cgc]} t | d||}} t|D cgc]\} } |j| | }} } t|D cgc]$\} } |jj| | &}} } |jj|d} |jj| |d} |jj| |d} |jj| |d}|j|}|Scc}wcc} wcc} wcc} } wcc} } w)Nz0Need to call init(dim_tokens_enc) function firstzb (nh nw) c -> b c nh nw)nhnwrrrr)rrrrrrr enumeraterr*r&rrrrr)r@r image_sizeHWN_HN_Whooklayerslidxpath_4path_3path_2path_1rGs rrHzDPTOutputAdapter.forwards"".b0bb1D%%01D%%0148::>4.&>>1771$##A&77U[[q)A9ccJ[[=Fv=NO63+$&&s+A.OO>G>OPFC,$,,'',Q/PP((3((;((;((;ii +?8\OPs F1F$F)5F.%)F4)rz)rJrKrLrMrdrrrstrrboolr r5rrrTensorrHrNrOs@rrsrss  &'%&;=-5$1)<$'!# %15".$%;5";5";5#3c3h#78;5&c] ;5 S ;5 "#Y ;5";5;5;5"*#;5 ;5zE Nd5<<&8rrs)rF)r)rtorch.nnrtorch.nn.functionalreraeinopsrrtypingrrrrr r rr/rr1rQrlrnrsrrrss $??19v?)bii?)BJ JX "))Bzryyzr