o vi@sNddlmZddlZddlmZGdddZGdddZGdd d ZdS) )CallableN)FlowMatchEulerDiscreteSchedulerc@s.eZdZ d defddZdejfddZdS) TrainTimeWeightuniformweightcCs.|dkrd}||_||_|jdksJddS)NZ reweightingrz+Only uniform loss weight is supported in RF)rnoise_scheduler)selfrrr [/data/cameron/vidgen/cosmos-policy/cosmos_policy/_src/predict2/schedulers/rectified_flow.py__init__s zTrainTimeWeight.__init__returncCs*|jdkr t|}|Std|jd)Nrz Time weight '' is not implemented.)rtorch ones_likeNotImplementedError)rt tensor_kwargsZwtsr r r __call__%s  zTrainTimeWeight.__call__Nr)__name__ __module__ __qualname__strr rTensorrr r r r rs  rc @sTeZdZ d defddZeedejfde dejdej d ej fd d Z d S)TrainTimeSamplerr distributioncCs ||_dS)N)r)rrr r r r /s zTrainTimeSampler.__init__cpu batch_sizedevicedtyper cCs`|jdkrt|fj||d}|S|jdkr'tt|fj||d}|Std|jd)z~ Sample time tensor for training Returns: torch.Tensor: Time tensor, shape (batch_size,) rrr logitnormalzTime distribution 'r )rrrandtosigmoidrandnrdist)rrrrrr r r r5s  zTrainTimeSampler.__call__Nr) rrrrr rno_gradrfloat32intrrrr r r r r.s" rc@seZdZddddedejfdedeeBdede d e d ejd ej fd d Z de fddZ ddZddZdejdejdejfddZdS) RectifiedFlowrFrvelocity_fieldtrain_time_distributiontrain_time_weight_methoduse_dynamic_shiftshiftrrcCs||_t|tr |nt||_|rt|d|_nt|d|_t|j||_t|tr/t |n||_ t|tr?t ||_ dS||_ dS)aInitialize the RectifiedFlow class. Args: velocity_field (`Callable`): A function that predicts the velocity given the current state and time. train_time_distribution (`TrainTimeSampler` or `str`, *optional*, defaults to `"uniform"`): Distribution for sampling training times. Can be an instance of `TrainTimeSampler` or a string specifying the distribution type. train_time_weight (`TrainTimeWeight` or `str`, *optional*, defaults to `"uniform"`): Weight applied to training times. Can be an instance of `TrainTimeWeight` or a string specifying the weight type. )use_dynamic_shifting)r0N) r, isinstancertrain_time_samplerrrrtrain_time_weightrrrr)rr,r-r.r/r0rrr r r r Ms $zRectifiedFlow.__init__rcCs|j||j|jd}|S)aThis method calls the `TrainTimeSampler` to sample training times. Returns: t (`torch.Tensor`): A tensor of sampled training times with shape `(batch_size,)`, matching the class specified `device` and `dtype`. r )r3rr)rrtimer r r sample_train_timesszRectifiedFlow.sample_train_timecCsF||jjj}|jjjdi||}|jdkr!|dS|S)z/This method map time from 0,1 to discrete stepsrNr ) squeezerconfignum_train_timestepslong timestepsr#ndim unsqueeze)rurindicesr;r r r get_discrete_timestamp~sz$RectifiedFlow.get_discrete_timestampcsd|jjjdi|}|jjjdi|fdd|D}t||jdks*Jd||}|S)Ncs g|] }|kqSr )nonzeror7tolist).0rschedule_timestepsr r s z,RectifiedFlow.get_sigmas..rz3Number of indices do not match the given timesteps.r )rsigmasr#r;lenshapeflatten)rr;rrG step_indicessigmar rDr get_sigmass  zRectifiedFlow.get_sigmasx_0x_1rcCs|j|jks Jd|jd|jdksJd|jd|jdks&Jd|j|jdgdgt|jdR}|||d|}||}||fS)a This method computes interpolation `X_t` and their time derivatives `dotX_t` at the specified time points `t`. Note that `x_0` is the noise, and `x_1` is the clean data. This is aligned with the notation in the recified flow community, but different from the notation in the diffusion community. Args: x_0 (`torch.Tensor`): noise, shape `(B, D1, D2, ..., Dn)`, where `B` is the batch size, and `D1, D2, ..., Dn` are the data dimensions. x_1 (`torch.Tensor`): clean data, with the same shape as `x_0` t (`torch.Tensor`): A tensor of time steps, with shape `(B,)`, where each value is in `[0, 1]`. Returns: (x_t, dot_x_t) (`Tuple[torch.Tensor, torch.Tensor]`): - x_t (`torch.Tensor`): The interpolated state, with shape `(B, D1, D2, ..., Dn)`. - dot_x_t (torch.Tensor): The time derivative of the interpolated state, with the same shape as `x_t`. z%x_0 and x_1 must have the same shape.rz%Batch size of x_0 and x_1 must match.zBatch size of t must match x_1.)rIviewrH)rrNrOrx_tZdot_x_tr r r get_interpolations(zRectifiedFlow.get_interpolationN)rrrrrr(rrrboolr)rr r6r@rMrrSr r r r r*Ls@ &  r*)typingrr diffusersrrrr*r r r r s