mrq b482ca19ff added model config option to set KV head count for MQA/GQA instead of MHA for llama-based models (i think its very negligible both ways on such a small model size)		2024-05-31 19:32:37 -05:00
..
__init__.py	sanitizing	2024-05-11 16:31:05 -05:00
adaln.py
ar_nar.py	correctness	2024-05-30 20:50:45 -05:00
base.py	added model config option to set KV head count for MQA/GQA instead of MHA for llama-based models (i think its very negligible both ways on such a small model size)	2024-05-31 19:32:37 -05:00
retnet_hf.py
retnet_ts.py
retnet.py
transformer.py