21 Commits

Author	SHA1	Message	Date
mrq	d7152fc7b9	added pruning of old checkpoints if specified (cfg.trainer.keep_last_checkpoints)	2023-08-16 20:12:12 -05:00
mrq	44c08d828e	added sample_type that samples from speakers to truly balance an epoch by speakers rather than the entire dataset and a sampler that tries to balance by speakers	2023-08-16 19:39:21 -05:00
mrq	599e47a813	might fix user inputted saving/quitting breaking when distributed	2023-08-15 23:52:20 -05:00
mrq	1e3e1d9315	tweaks	2023-08-15 21:58:16 -05:00
mrq	277c759ab1	fixed issue with non-distributed training, oops	2023-08-14 21:42:35 -05:00
mrq	5fa86182b5	oops	2023-08-14 10:50:40 -05:00
mrq	13571380be	made exporter make more sense	2023-08-13 22:56:28 -05:00
mrq	d7deaf6def	distributed training works now (hopefully)	2023-08-13 22:07:45 -05:00
mrq	2af09d0bef	fixed that mysterious discepancy between the reported losses (I am so freaking mad, my piss is boiling, I had to interrupt halfway through an epoch)	2023-08-05 15:25:41 -05:00
mrq	d1b9770d41	set model to eval when inferencing (very important)	2023-08-05 04:29:05 +00:00
mrq	d89568a96e	some fixes for the local framework	2023-08-05 03:22:15 +00:00
mrq	5970f254e3	some fixes for the local framework	2023-08-05 02:17:30 +00:00
mrq	012f54b7f1	another classic commit so i can copy it to another machine to gut out things and use the trainer bits for a side project that I should really get around to working on sooner than later	2023-08-04 14:21:30 -05:00
mrq	0a524f1d59	reticulating splines	2023-08-03 21:39:00 -05:00
mrq	608c1970eb	ops	2023-08-03 20:36:19 -05:00
mrq	c85101403f	big cleanup	2023-08-03 20:26:36 -05:00
mrq	2e03e5ac93	Fixed an issue with having fairseq installed at all will brick logging	2023-08-02 22:57:10 -05:00
mrq	f6597e2dfe	adjustments	2023-08-02 18:36:26 -05:00
mrq	0f9b81de75	oops	2023-08-02 18:12:36 -05:00
mrq	7a06b27a9c	Tweaks	2023-08-02 22:06:39 +00:00
mrq	bf8cedc9dd	Rewrite init	2023-08-02 21:53:35 +00:00