o vi@s|ddlZddlmZz ddlmZmZWn ey!edYnwdejdedejfdd Z dejdedejfd d Z dS) N) DeviceMesh) Replicatedistribute_tensorzHtorch.distributed.tensor is not available. DeepSeek model will not work.tensor cp_or_tp_meshreturncCs.|d}|dkrt||tg}|S)Ncuda)tosizerrto_local)rrr R/data/cameron/vidgen/cosmos-policy/cosmos_policy/_src/reason1/utils/parallelism.py broadcasts  rcCs^t|j}t|j}t||}||r(tj||j |j d}t||}|S)aVBroadcast a tensor and check if the shape is the same across CP/TP ranks. If not, create a new tensor matching rank 0 and broadcast it. Args: tensor (torch.Tensor): The tensor to broadcast. cp_or_tp_mesh (DeviceMesh): The device mesh used to broadcast. Returns: torch.Tensor: The broadcasted tensor. )dtypedevice) torchrshaperrneanyzerostolistrr)rroriginal_shape final_shaper r rbroadcast_with_shape_check!s   r) rtorch.distributed.device_meshrtorch.distributed.tensorrr ImportErrorprintTensorrrr r r rs