o ?߱iU@sZddlmZddlmZd deejdefddZd deejdefddZd d d Z dS))OptionalNgroupreturncC"d}trtrt|}|S)zgGet the rank (GPU device) of the worker. Returns: rank (int): The rank of the worker. r)dist is_availableis_initializedget_rank)rrankr X/data/cameron/vidgen/cosmos-predict2.5/cosmos_predict2/_src/reason1/utils/distributed.pyr  r cCr)zGet world size. How many GPUs are available in this job. Returns: world_size (int): The total number of GPUs available in this job. )rrrget_world_size)r world_sizer r r r!r rFcCs(t}|dkr |Stj||||ddS)N)srcrasync_op)rr broadcast)tensorrrrrr r r r-sr)N)NF) typingrtorch.distributed distributedr ProcessGroupintr rrr r r r s