mrq 7047fcc6e2 actually make deepspeed work with LoRAs		2024-06-17 13:55:37 -05:00
..
arch	mamba2-hf using vasqu/mamba2-torch because it lets me use mamba2 without triton ops (training with my 4xV100s are not happy with mamba2 because of triton)	2024-06-14 19:42:17 -05:00
__init__.py	ugh	2024-06-11 23:59:28 -05:00
ar_nar.py	ugh	2024-06-15 12:29:03 -05:00
base.py	residual_in_fp32=False for mamba arch backends because it breaks the classifier (output projection / lm head / what-have-you) under AMP	2024-06-15 12:08:03 -05:00
experimental.py	actually going for the suggested "2x layers, no intermediate scaling" is wrong for VALL-E, directly copying the normal transformer structure fixes mamba2 performance in the test trainer	2024-06-13 20:08:22 -05:00
lora.py	actually make deepspeed work with LoRAs	2024-06-17 13:55:37 -05:00
nar.py	the NAR only dream is dead (it just won't work)	2024-06-12 19:49:47 -05:00