o ?߱ip@sBddlmZddlZGdddZGdddZGdddZdS) )CallableNc@s.eZdZ d defddZdejfddZdS) TrainTimeWeightuniformweightcCs.|dkrd}||_||_|jdksJddS)NZ reweightingrz+Only uniform loss weight is supported in RF)rnoise_scheduler)selfrrra/data/cameron/vidgen/cosmos-predict2.5/cosmos_predict2/_src/predict2/schedulers/rectified_flow.py__init__s zTrainTimeWeight.__init__returncCs*|jdkr t|}|Std|jd)Nrz Time weight '' is not implemented.)rtorch ones_likeNotImplementedError)rt tensor_kwargswtsrrr __call__$s  zTrainTimeWeight.__call__Nr)__name__ __module__ __qualname__strr r Tensorrrrrr rs  rc @sTeZdZ d defddZeedejfde dejdej d ej fd d Z d S)TrainTimeSamplerr distributioncCs ||_dS)N)r)rrrrr r .s zTrainTimeSampler.__init__cpu batch_sizedevicedtyper cCs|jdkrt|fj||d}|S|jdkr'tt|fj||d}|S|jdrit|jdd}|ddksAJd tj|ftj d }d ||t tj d |d d|}|j||d}|St d|jd)z~ Sample time tensor for training Returns: torch.Tensor: Time tensor, shape (batch_size,) rrr logitnormalZ waver_mode__gp= ף?g-C6?z9Waver's mode distribution is only supported with s = 1.29)rg?g@zTime distribution 'r ) rr randtosigmoidrandn startswithfloatsplitfloat32cospir)rrrrrsurrr r4s   ,zTrainTimeSampler.__call__Nr) rrrrr r no_gradrr-intrrrrrrr r-s" rc@seZdZddddedejfdedeeBdede d e d ejd ej fd d Z de fddZ ddZddZdejdejdejfddZdS) RectifiedFlowrFrvelocity_fieldtrain_time_distributiontrain_time_weight_methoduse_dynamic_shiftshiftrrcCs||_t|tr |nt||_|dusJd||_d|_td||_t|tr-t |n||_ t|tr=t ||_ dS||_ dS)aInitialize the RectifiedFlow class. Args: velocity_field (`Callable`): A function that predicts the velocity given the current state and time. train_time_distribution (`TrainTimeSampler` or `str`, *optional*, defaults to `"uniform"`): Distribution for sampling training times. Can be an instance of `TrainTimeSampler` or a string specifying the distribution type. train_time_weight (`TrainTimeWeight` or `str`, *optional*, defaults to `"uniform"`): Weight applied to training times. Can be an instance of `TrainTimeWeight` or a string specifying the weight type. Fz/Dynamic shift is not supported in RectifiedFlowiN) r6 isinstancertrain_time_samplerr:num_train_timestepsrtrain_time_weightrr rr)rr6r7r8r9r:rrrrr r Rs $zRectifiedFlow.__init__rcCs|j||j|jd}|S)aThis method calls the `TrainTimeSampler` to sample training times. Returns: t (`torch.Tensor`): A tensor of sampled training times with shape `(batch_size,)`, matching the class specified `device` and `dtype`. r )r<rr)rrtimerrr sample_train_timewszRectifiedFlow.sample_train_timecCsF|}|j|d|jd|}||j}|jdkr!|dS|S)z/This method map time from 0,1 to discrete stepsr%r)squeezer:r=ndim unsqueeze)rr1r timestepsrrr get_discrete_timestamps z$RectifiedFlow.get_discrete_timestampcCs|jdi||j}|S)Nr)r'r=)rrDrsigmasrrr get_sigmasszRectifiedFlow.get_sigmasx_0x_1rcCs|j|jks Jd|jd|jdksJd|jd|jdks&Jd|j|jdgdgt|jdR}|||d|}||}||fS)a This method computes interpolation `X_t` and their time derivatives `dotX_t` at the specified time points `t`. Note that `x_0` is the noise, and `x_1` is the clean data. This is aligned with the notation in the recified flow community, but different from the notation in the diffusion community. Args: x_0 (`torch.Tensor`): noise, shape `(B, D1, D2, ..., Dn)`, where `B` is the batch size, and `D1, D2, ..., Dn` are the data dimensions. x_1 (`torch.Tensor`): clean data, with the same shape as `x_0` t (`torch.Tensor`): A tensor of time steps, with shape `(B,)`, where each value is in `[0, 1]`. Returns: (x_t, dot_x_t) (`Tuple[torch.Tensor, torch.Tensor]`): - x_t (`torch.Tensor`): The interpolated state, with shape `(B, D1, D2, ..., Dn)`. - dot_x_t (torch.Tensor): The time derivative of the interpolated state, with the same shape as `x_t`. z%x_0 and x_1 must have the same shape.rz%Batch size of x_0 and x_1 must match.zBatch size of t must match x_1.r%)shapeviewlen)rrHrIrx_tZdot_x_trrr get_interpolations(zRectifiedFlow.get_interpolationN)rrrr rr-rrrboolr3rr r@rErGrrNrrrr r4Qs@ % r4)typingrr rrr4rrrr s $