mrq 83eab4fa59 actually going for the suggested "2x layers, no intermediate scaling" is wrong for VALL-E, directly copying the normal transformer structure fixes mamba2 performance in the test trainer		2024-06-13 20:08:22 -05:00
..
retnet_syncdoth
__init__.py	actually going for the suggested "2x layers, no intermediate scaling" is wrong for VALL-E, directly copying the normal transformer structure fixes mamba2 performance in the test trainer	2024-06-13 20:08:22 -05:00
bitnet.py
llama.py
mamba.py
mixtral.py
mmfreelm.py
retnet.py
transformer.py