mrq e3ef89f5aa 100x better for subtrain/eval to be by group instead		2024-05-19 16:40:14 -05:00
..
__init__.py	sanitizing	2024-05-11 16:31:05 -05:00
adaln.py
ar_nar.py	100x better for subtrain/eval to be by group instead	2024-05-19 16:40:14 -05:00
base.py	added option to split between text loss and audio loss (to-do: document this better), because it may or may not be a problem with LLaMA-backed models because my loss hovers around 3.9 / 56% accuracy despite sounding decent at the moment	2024-05-19 11:23:56 -05:00
retnet_hf.py	added FP8 support through NVIDIA/TransformerEngine, added RetNet_HF through syncdoth/RetNet (as an alternative to branch away from torchscale)	2024-04-08 20:14:51 -05:00
retnet_ts.py	backwards compat for old YAMLs with models, option to set flash attention 2 for Llama (and derivatives), included syncdoth/RetNets torchscale retnet for shits and grins, etc.	2024-04-16 10:02:31 -05:00
retnet.py	added FP8 support through NVIDIA/TransformerEngine, added RetNet_HF through syncdoth/RetNet (as an alternative to branch away from torchscale)	2024-04-08 20:14:51 -05:00
transformer.py