10 Commits

Author	SHA1	Message	Date
mrq	6676c89c0e	I sucked off the hyptothetical wizard again, just using BNB's ADAM optimizer nets HUGE savings, but I don't know the output costs, will need to test	2023-02-23 02:42:17 +00:00
mrq	4427d7fb84	initial conversion (errors out)	2023-02-22 23:07:05 +00:00
James Betker	9a3e89ec53	Force LR fix	2021-10-21 12:01:01 -06:00
James Betker	40cb25292a	Fix force_lr logic	2021-10-21 11:51:30 -06:00
James Betker	5037220ac7	Mods to support contrastive learning on audio files	2021-08-05 05:57:04 -06:00
James Betker	341f28dd82	It works!	2021-08-04 20:07:51 -06:00
James Betker	587a4f4050	resnet_unet_3 ... I'm being really lazy here - these nets are not really different from each other except at which layer they terminate. This one terminates at 2x downsampling, which is simply indicative of a direction I want to go for testing these pixpro networks.	2021-01-15 14:51:03 -07:00
James Betker	edf9c38198	Make ExtensibleTrainer set the starting step for the LR scheduler	2021-01-02 22:22:34 -07:00
James Betker	036684893e	Add LARS optimizer & support for BYOL idiosyncrasies ... - Added LARS and SGD optimizer variants that support turning off certain features for BN and bias layers - Added a variant of pytorch's resnet model that supports gradient checkpointing. - Modify the trainer infrastructure to support above - Fix bug with BYOL (should have been nonfunctional)	2020-12-23 20:33:43 -07:00
James Betker	5640e4efe4	More refactoring	2020-12-18 09:18:34 -07:00