o ?߱i*@sbdZddlmZddlmZddlmZddlmZddl m Z m Z ddl m Z ddl mZe eed Zd e iZ d$d ed edededededBdededefddZdedeefddZ  d%d edededededBded edBdededBfddZ d$d ed ededededef ddZdedeefd d!Z d&d ededed edBdef d"d#ZdS)'zl Imaginaire4 Attention Subpackage: Unified implementation for all Attention implementations. Frontend APIs )Tensor)flash2_attention_check)flash3_attention_check) CausalType)natten_attention_check natten_multi_dim_attention_check) get_arch_tag)safe_log)nattenflash2flash3r Fbackendquerykeyvalue is_causal causal_typeN is_varlen raise_errorreturnc CsB|durtd|tvrtd|dt||||||||dS)a Input validation function a specified backend. Runs the common and backend-specific checks. Returns False if any checks fail, otherwise True. Parameters: backend (str): selected backend. query (Tensor): 4-D query tensor, with the heads-last contiguous layout (`[batch, seqlen, heads, head_dim]`). key (Tensor): 4-D key tensor, with the heads-last contiguous layout (`[batch, seqlen_kv, heads_kv, head_dim]`). value (Tensor): 4-D value tensor, with heads-last contiguous layout (`[batch, seqlen_kv, heads_kv, head_dim_v]`). is_causal (bool): whether or not causal masking is enabled. causal_type (CausalType): causal masking mode. Choices: `CausalType.TopLeft`, `CausalType.BottomRight`. Required when `is_causal = True`. is_varlen (bool): whether or not a variable length (varlen) use case. Must be inferred beforehand based on arguments such as seqlens_{Q,KV} or cumulative_seqlen_{Q,KV} being passed. raise_error (bool): whether to raise an error if any checks fail or no backend is selected, instead of just returning False. Default is False. Returns: success (bool): whether use case is compatible with the backend. N2Cannot pass None backend to is_backend_compatible.Unrecognized backend name .)rrrrrrr) ValueErrorBACKEND_CHECK_MAPr rrrrrrrr\/data/cameron/vidgen/cosmos-predict2.5/cosmos_predict2/_src/imaginaire/attention/backends.pyis_backend_compatible.s*rarch_tagcCsT|dkrtd|dgS|dkrgdS|dvrddgS|d kr'ddgSdgS) a Returns list of supported backends according to arch tag (attention.utils.get_arch_tag). Backends are ordered based on their known performance levels, so that the best-performing compatible backend is selected. Parameters: arch_tag (int): Arch tag for the current CUDA device. Example: 80 for A100, 90 for H100. Returns: backend_list (list[str]): a list of backend names (string). Empty if device is not supported. KzAMinimum architecture supported for Attention is 75, got arch_tag=rZ)r r r )dgr r Plogdebugrrrrget_backend_listisr)Tc Cst|durt||||||||dr|SdSt|j}t|} | D]}t|||||||ddr1|Sq|s6dStd)a Selects a compatible backend, unless one is already selected, which runs its corresponding checks. Parameters: query (Tensor): 4-D query tensor, with the heads-last contiguous layout (`[batch, seqlen, heads, head_dim]`). key (Tensor): 4-D key tensor, with the heads-last contiguous layout (`[batch, seqlen_kv, heads_kv, head_dim]`). value (Tensor): 4-D value tensor, with heads-last contiguous layout (`[batch, seqlen_kv, heads_kv, head_dim_v]`). is_causal (bool): whether or not causal masking is enabled. causal_type (CausalType): causal masking mode. Choices: `CausalType.TopLeft`, `CausalType.BottomRight`. Required when `is_causal = True`. is_varlen (bool): whether or not a variable length (varlen) use case. Must be inferred beforehand based on arguments such as seqlens_{Q,KV} or cumulative_seqlen_{Q,KV} being passed. backend (str | None): selected backend, if any. raise_error (bool): whether to raise an error if any checks fail or no backend is selected, instead of just returning False. Default is **True**. Returns: backend (str | None): selected backend, or None if no backends are compatible. NrFzvCould not find a compatible Attention backend for this use case / device. Try running with debug logs to find out why.)rrdevicer)r) rrrrrrr rr backend_listrrrchoose_backendsB*    r,cCs<|durtd|tvrtd|dt|||||dS)a Input validation function a specified multi-dimensional backend. Runs the common and backend-specific checks. Returns False if any checks fail, otherwise True. Parameters: backend (str): selected backend. query (Tensor): 4-D, 5-D, or 6-D query tensor, with the heads-last contiguous layout (`[batch, *token_layout_shape, heads, head_dim]`). key (Tensor): 4-D, 5-D, or 6-D key tensor, with the heads-last contiguous layout (`[batch, *token_layout_shape, heads_kv, head_dim]`). value (Tensor): 4-D, 5-D, or 6-D value tensor, with heads-last contiguous layout (`[batch, *token_layout_shape, heads_kv, head_dim_v]`). raise_error (bool): whether to raise an error if any checks fail or no backend is selected, instead of just returning False. Default is False. Returns: success (bool): whether use case is compatible with the backend. Nrrr)rrrr)rBACKEND_MULTI_DIM_CHECK_MAPr rrrrrrris_multi_dim_backend_compatiblesr/cCs$|dkrtd|dgSdgS)a Returns list of supported multi-dimensional backends according to arch tag (attention.utils.get_arch_tag). Backends are ordered based on their known performance levels, so that the best-performing compatible backend is selected. Parameters: arch_tag (int): Arch tag for the current CUDA device. Example: 80 for A100, 90 for H100. Returns: backend_list (list[str]): a list of backend names (string). Empty if device is not supported. r zSMinimum architecture supported for Multi-Dimensional Attention is 75, got arch_tag=rr r%r(rrrget_multi_dim_backend_list sr0cCs`|durt||||ddsJ|St|j}t|}|D]}t||||ddr+|Sqtd)a Selects a compatible multi-dimensional backend, unless one is already selected, which runs its corresponding checks. Parameters: query (Tensor): 4-D, 5-D, or 6-D query tensor, with the heads-last contiguous layout (`[batch, *token_layout_shape, heads, head_dim]`). key (Tensor): 4-D, 5-D, or 6-D key tensor, with the heads-last contiguous layout (`[batch, *token_layout_shape, heads_kv, head_dim]`). value (Tensor): 4-D, 5-D, or 6-D value tensor, with heads-last contiguous layout (`[batch, *token_layout_shape, heads_kv, head_dim_v]`). backend (str | None): selected backend, if any. Returns: backend (str): selected backend. NTr.FzCould not find a compatible Multi-Dimensional Attention backend for this use case / device. Try running with debug logs to find out why.)r/rr*r0r)rrrr rr+rrrchoose_multi_dim_backend#s0   r1)F)NT)N)__doc__torchrZ7cosmos_predict2._src.imaginaire.attention.flash2.checksrZ7cosmos_predict2._src.imaginaire.attention.flash3.checksr/cosmos_predict2._src.imaginaire.attention.masksrZ7cosmos_predict2._src.imaginaire.attention.natten.checksrr/cosmos_predict2._src.imaginaire.attention.utilsrr r&rr-strboolrintlistr)r,r/r0r1rrrrs        ;/ U ,