mrq 0d706ec6a1 added fused_attn (triton-based fused attention) and simply just query for flash_attn under rocm		2024-08-26 19:13:34 -05:00
..
attention	added fused_attn (triton-based fused attention) and simply just query for flash_attn under rocm	2024-08-26 19:13:34 -05:00
mamba_vasqu	mamba2-hf using vasqu/mamba2-torch because it lets me use mamba2 without triton ops (training with my 4xV100s are not happy with mamba2 because of triton)	2024-06-14 19:42:17 -05:00
retnet_syncdoth
__init__.py	added fused_attn (triton-based fused attention) and simply just query for flash_attn under rocm	2024-08-26 19:13:34 -05:00
bitnet.py	re-added loading multiple models because I'm now entertaining having split AR/NAR models again (and need a way to load both at once)	2024-06-06 09:48:43 -05:00
llama.py	added fused_attn (triton-based fused attention) and simply just query for flash_attn under rocm	2024-08-26 19:13:34 -05:00
mamba.py	ughghghhhh	2024-08-09 21:15:01 -05:00
mixtral.py	added flash_attn LlamaAttention (including flash_attn==1.0.9)	2024-08-18 20:51:14 -05:00
retnet.py
transformer.py