mrq 45a39fb79f very rudimentary lora support (no deepspeed support, tested training and saving but not loading yet)		2024-06-17 00:09:16 -05:00
..
arch	mamba2-hf using vasqu/mamba2-torch because it lets me use mamba2 without triton ops (training with my 4xV100s are not happy with mamba2 because of triton)	2024-06-14 19:42:17 -05:00
__init__.py
ar_nar.py	ugh	2024-06-15 12:29:03 -05:00
base.py	residual_in_fp32=False for mamba arch backends because it breaks the classifier (output projection / lm head / what-have-you) under AMP	2024-06-15 12:08:03 -05:00
experimental.py	actually going for the suggested "2x layers, no intermediate scaling" is wrong for VALL-E, directly copying the normal transformer structure fixes mamba2 performance in the test trainer	2024-06-13 20:08:22 -05:00
lora.py	very rudimentary lora support (no deepspeed support, tested training and saving but not loading yet)	2024-06-17 00:09:16 -05:00
nar.py