o vi3@sdZddlmZddlmZddlmZmZddlm Z m Z ddl m Z m Z ddlmZddlmZmZmZ dd ed ed ed ededededefddZdS)zs Imaginaire4 Attention Subpackage: Unified implementation for all Attention implementations. cudNN backend checks )partial)Tensor)attention_param_checksattention_tensor_checks)CUDNN_DISALLOWEDCUDNN_SUPPORTED)get_bwd_dtypesget_fwd_dtypes) CausalType) get_arch_tagis_torch_compilinglog_or_raise_errorFquerykeyvalue is_causal causal_type is_varlen raise_errorreturnc Cstt|d}ts|dtddStr|dtddStr%|dtddSt|j}t|} t |} t |||| | dd|dd sG|d tddS|rQ|d tddSt |||||d |rl|t j t jfvrl|d tddSd S)a Input validation function for the cuDNN backend. Runs the common and cuDNN-specific checks. Returns False if any checks fail, otherwise True. Parameters: query (Tensor): 4-D query tensor, with the heads-last contiguous layout (`[batch, seqlen, heads, head_dim]`). key (Tensor): 4-D key tensor, with the heads-last contiguous layout (`[batch, seqlen_kv, heads_kv, head_dim]`). value (Tensor): 4-D value tensor, with heads-last contiguous layout (`[batch, seqlen_kv, heads_kv, head_dim_v]`). is_causal (bool): whether or not causal masking is enabled. causal_type (CausalType): causal masking mode. Choices: `CausalType.TopLeft`, `CausalType.BottomRight`. Required when `is_causal = True`. is_varlen (bool): whether or not a variable length (varlen) use case. Must be inferred beforehand based on arguments such as seqlens_{Q,KV} or cumulative_seqlen_{Q,KV} being passed. raise_error (bool): whether to raise an error if any checks fail or no backend is selected, instead of just returning False. Default is False. Returns: success (bool): whether use case is compatible with cuDNN backend. )rzkcuDNN is not supported in this environment. Run with debug logs to find out why, or choose another backend.) exceptionFz9cuDNN backend is disabled. Please choose another backend.z1cuDNN backend does not support torch.compile yet.zcuDNN Attention) rrrZsupported_dtypes_forwardZsupported_dtypes_backwardZ supports_mlaZsupports_gqa_mqar backend_namez(cuDNN does not support the given inputs.z1Varlen for cuDNN Attention is not integrated yet.)rrrrrz>cuDNN Attention only supports top-left causal masking for now.T)rr r RuntimeErrorrr r devicer rrrr TopLeftDontCare) rrrrrrr target_fnarch_tagZ fwd_dtypesZ bwd_dtypesrZ/data/cameron/vidgen/cosmos-policy/cosmos_policy/_src/imaginaire/attention/cudnn/checks.pycudnn_attention_check"s\ '     r N)F)__doc__ functoolsrtorchr.cosmos_policy._src.imaginaire.attention.checksrr-cosmos_policy._src.imaginaire.attention.cudnnrrZ2cosmos_policy._src.imaginaire.attention.cudnn.metarr -cosmos_policy._src.imaginaire.attention.masksr -cosmos_policy._src.imaginaire.attention.utilsr r r boolr rrrrs4