21 Commits (ee8270bdfb0c9ba98cd6abdd1280cf057972548a)

Author	SHA1	Message	Date
mrq	84b7383428	fixes #134	2023-03-14 18:52:56 +07:00
mrq	7b16b3e88a	;)	2023-03-14 15:48:09 +07:00
mrq	008a1f5f8f	simplified spawning the training process by having it spawn the distributed training processes in the train.py script, so it should work on Windows too	2023-03-11 01:37:00 +07:00
mrq	7f2da0f5fb	rewrote how AIVC gets training metrics (need to clean up later)	2023-03-10 22:35:32 +07:00
mrq	b64948d966	while I'm breaking things, migrating dependencies to modules folder for tidiness	2023-03-09 04:03:57 +07:00
mrq	3f321fe664	big cleanup to make my life easier when i add more parameters	2023-03-09 00:26:47 +07:00
mrq	34dcb845b5	actually make using adamw_zero optimizer for multi-gpus work	2023-03-08 15:31:33 +07:00
mrq	d7e75a51cf	I forgot about the changelog and never kept up with it, so I'll just not use a changelog	2023-03-08 05:14:50 +07:00
mrq	37cab14272	use torchrun instead for multigpu	2023-03-04 20:53:00 +07:00
mrq	e859a7c01d	experimental multi-gpu training (Linux only, because I can't into batch files)	2023-03-03 04:37:18 +07:00
mrq	92553973be	Added option to disable bitsandbytesoptimizations for systems that do not support it (systems without a Turing-onward Nvidia card), saves use of float16 and bitsandbytes for training into the config json	2023-02-26 01:57:56 +07:00
mrq	d5d8821a9d	fixed some files not copying for bitsandbytes (I was wrong to assume it copied folders too), fixed stopping generating and training, some other thing that I forgot since it's been slowly worked on in my small free times	2023-02-24 23:13:13 +07:00
mrq	41fca1a101	ugh	2023-02-23 07:20:40 +07:00
mrq	941a27d2b3	removed the logic to toggle BNB capabilities, since I guess I can't do that from outside the module	2023-02-23 07:05:39 +07:00
mrq	225dee22d4	huge success	2023-02-23 06:24:54 +07:00
mrq	bacac6daea	handled paths that contain spaces because python for whatever god forsaken reason will always split on spaces even if wrapping an argument in quotes	2023-02-20 20:23:22 +07:00
mrq	d5c1433268	a bit of UI cleanup, import multiple audio files at once, actually shows progress when importing voices, hides audio metadata / latents if no generated settings are detected, preparing datasets shows its progress, saving a training YAML shows a message when done, training now works within the web UI, training output shows to web UI, provided notebook is cleaned up and uses a venv, etc.	2023-02-18 02:07:22 +07:00
mrq	bcec64af0f	cleanup, "injected" dvae.pth to download through tortoise's model loader, so I don't need to keep copying it	2023-02-17 19:06:05 +07:00
mrq	8d268bc7a3	training added, seems to work, need to test it more	2023-02-17 16:29:27 +07:00
mrq	9c0e4666d2	updated notebooks to use the new "main" setup	2023-02-17 03:30:53 +07:00
mrq	f8249aa826	tab to generate the training YAML	2023-02-17 03:05:27 +07:00