mrq 0db3203b21 added LLaMA/Mixtral (if experts>1) model arches, utilize XMoE's loss as well, set MoE frequency to 1 to make every layer MoE'd for RetNet, etc. (going to do tests without burning out again to see how things go)		2023-12-22 19:27:36 -06:00
..
emb
engines
models	added LLaMA/Mixtral (if experts>1) model arches, utilize XMoE's loss as well, set MoE frequency to 1 to make every layer MoE'd for RetNet, etc. (going to do tests without burning out again to see how things go)	2023-12-22 19:27:36 -06:00
utils
__init__.py
__main__.py
config.py
data.py
export.py
inference.py
plot.py
samplers.py
train.py
webui.py