mrq 6b0891448c pain (some shit to try and get some flash attention for ROCm (gfx1100) through triton fused attention but no good)		2024-08-25 20:07:27 -05:00
..
arch	pain (some shit to try and get some flash attention for ROCm (gfx1100) through triton fused attention but no good)	2024-08-25 20:07:27 -05:00
__init__.py	sanity cleanup: moved experimental features under its own thing	2024-06-30 10:37:33 -05:00
ar_nar.py	ughghghhhh	2024-08-09 21:15:01 -05:00
ar.py	fix issue with sft and shared tensors...	2024-08-04 19:56:21 -05:00
base.py	pain (some shit to try and get some flash attention for ROCm (gfx1100) through triton fused attention but no good)	2024-08-25 20:07:27 -05:00
experimental.py	ughghghhhh	2024-08-09 21:15:01 -05:00
lora.py	naive model offloading support (handles automatically splitting parts of the model to requested device per memory constraints, either inferred or requested in the yaml, input tensors are automatically migrated to the right device, it SEEMS to work for training under the test trainer when split between GPU and CPU) (this was specifically only because that Flux imagegen model released so I can test it there)	2024-08-01 20:12:06 -05:00
nar.py	changed torch.Tensor().to(device, dtype) to just torch.tensor(..., device, dtype) because it's been bothering my autism that I'm creating tensors then converting rather than creating with the right device/dtype, some 'optimization' to compile the model but it doesnt seem to do anything useful	2024-08-03 22:10:21 -05:00