! POSSIBLE NEW ISSUES ! #100

New Issue

mrq · 2023-03-09T04:17:58Z

mrq commented

2023-03-09 04:17:58 +00:00

I've done some things that pretty much could cause problems. Below are some things that may crop up as a result from the commits of 2023.03.08:

> my finetune models disappeared!!1

I've tidied up where the finetuning data gets outputted. Instead of ./training/{voice}-finetune/, they're saved to ./training/{voice}/finetine/

I've also moved where transcribed + sliced sound files are outputted, from ./training/{voice}/ to ./training/{voice}/audio/.

I don't want to bloat up writing backwards-compat with checking the old paths, just like I don't want any more nasty icky checks for people who didn't update right.

> I can't import from an existing training configuration!!!11

I've also made the import code very simple, instead of lossily converting from iterations to epochs. It'll check for a train.json that contains the unparsed inputs when saving, to easily load them back up again, instead of the old way, where I'm reparsing the output YAML to read back in (very icky).

If you done the above to manually move the finetune output folders, and you want to automatically have it pick out the last training state, simply save first, then click import, and it'll automatically nab the training state, as it normally would.

I don't want to bloat up writing backwards-compat code with reparsing the final configuration YAML again, it smelt bad when it did that anyways.

> I've updated using git pull or ./update.sh and [something happened]!!!11

I also cleaned up moving some dependencies into the ./modules/ folder. Please run the update-force script, as it'll force re-install tortoise-tts that now lives under ./modules/tortoise-tts/. You may have to delete the old ./dlas/ and ./tortoise-tts/, as in testing, it didn't delete these folders.

I may or may not have a treat that comes from consolidating external dependencies. It just depends how well my experiment goes.

I don't expect a full clean reinstall + data migration is necessary for this, but when in doubt.

> I generated a validation dataset, but the iteration count is going crazy / progress bar is going off the screen!

Unfortunately, the progress bar for validation is the same as normal batches, and there isn't really a nice way to differentiate between the two.

I've done some things that pretty much *could* cause problems. Below are some things that may crop up as a result from the commits of 2023.03.08: >\> my finetune models disappeared!!1 I've tidied up where the finetuning data gets outputted. Instead of `./training/{voice}-finetune/`, they're saved to `./training/{voice}/finetine/` I've also moved where transcribed + sliced sound files are outputted, from `./training/{voice}/` to `./training/{voice}/audio/`. I don't want to bloat up writing backwards-compat with checking the old paths, just like I don't want any more nasty icky checks for people who didn't update right. >\> I can't import from an existing training configuration!!!11 I've also made the import code very simple, instead of lossily converting from iterations to epochs. It'll check for a `train.json` that contains the unparsed inputs when saving, to easily load them back up again, instead of the old way, where I'm reparsing the output YAML to read back in (very icky). If you done the above to manually move the finetune output folders, and you want to automatically have it pick out the last training state, simply save first, then click import, and it'll automatically nab the training state, as it normally would. I don't want to bloat up writing backwards-compat code with reparsing the final configuration YAML again, it smelt bad when it did that anyways. >\> I've updated using `git pull` or `./update.sh` and \[something happened\]!!!11 I also cleaned up moving some dependencies into the `./modules/` folder. Please run the update-force script, as it'll force re-install tortoise-tts that now lives under `./modules/tortoise-tts/`. You may have to delete the old `./dlas/` and `./tortoise-tts/`, as in testing, it didn't delete these folders. I may or may not have a treat that comes from consolidating external dependencies. It just depends how well my experiment goes. I don't expect a full clean reinstall + data migration is necessary for this, but when in doubt. >\> I generated a validation dataset, but the iteration count is going crazy / progress bar is going off the screen! Unfortunately, the progress bar for validation is the same as normal batches, and there isn't really a nice way to differentiate between the two.

👍 2

mrq added the

bug

not a bug

labels 2023-03-09 04:17:58 +00:00

ThrowawayAccount01 commented

2023-03-09 05:51:45 +00:00

Congrats on issue #100, you've come a long way!

Appreciate all the hard work you've put into this project.

Here's hoping it will get even better in the future!

Congrats on issue #100, you've come a long way! Appreciate all the hard work you've put into this project. Here's hoping it will get even better in the future!

Armored1065 commented

2023-03-09 11:07:07 +00:00

Absolute madlad, what a legend, keep up with the good work!

accountts commented

2023-03-09 23:19:58 +00:00

I don't know if it's related, but it's now 4 times "slower" for each preset
(for instance, ultra used to set 16 samples that amounted to 4 autoregressive steps, now it sets 16 samples and 16 autoregressive steps)

I don't know if it's related, but it's now 4 times "slower" for each preset (for instance, ultra used to set 16 samples that amounted to 4 autoregressive steps, now it sets 16 samples and 16 autoregressive steps)

mrq commented

2023-03-09 23:32:08 +00:00

I wonder if I botched the "cleanup" for the automatic deduction of the autoregressive sample size. I'll validate it in a moment. You can always hard-set the sample batch size under settings.

Armored1065 commented

2023-03-10 13:36:57 +00:00

Can you use code for the speed improvements from the fast-tts github? Seems like they've discontinued working on it and are sending everyone here.

mrq commented

2023-03-10 14:25:47 +00:00

Can you use code for the speed improvements from the fast-tts github?

For tortoise itself? I'll comb through it in the event I missed anything substantial, since the last I looked at it was for kv_cache's slight uplift in AR sampling.

> Can you use code for the speed improvements from the fast-tts github? For tortoise itself? I'll comb through it in the event I missed anything substantial, since the last I looked at it was for kv_cache's slight uplift in AR sampling.

Nomad commented

2023-03-10 21:16:52 +00:00

Is there any chance of getting a community page, or discord where users can share/ask/solve errors or issues, that may not actually be issues needing a post here?

Additionally, it would be useful to see a quick video tutorial voiced or not, on how to properly start the training both from Collab side as well the local PC side. This could even be community created content if there was a place for it to exist.

Is there any chance of getting a community page, or discord where users can share/ask/solve errors or issues, that may not actually be issues needing a post here? Additionally, it would be useful to see a quick video tutorial voiced or not, on how to properly start the training both from Collab side as well the local PC side. This could even be community created content if there was a place for it to exist.

mrq commented

2023-03-10 23:18:25 +00:00

Can you use code for the speed improvements from the fast-tts github?

For tortoise itself? I'll comb through it in the event I missed anything substantial, since the last I looked at it was for kv_cache's slight uplift in AR sampling.

Didn't happen to see anything I missed outside of the other diffusion sampler that seems to have caveats.

Is there any chance of getting a community page [...] This could even be community created content if there was a place for it to exist.

desu, I'm not interested in being involved in a community any more than staying in my own sphere here. But, I would prefer to keep it that way. It's already a bit astonishing seeing the reach this repo has spread from my only main involvement being a rentry on /v/ as a brief simple guide.

I've had a lengthier response, but as I mentioned before, it's rather blunt and outside of the scope here.

Someone else will do it. Not me.

>> Can you use code for the speed improvements from the fast-tts github? > For tortoise itself? I'll comb through it in the event I missed anything substantial, since the last I looked at it was for kv_cache's slight uplift in AR sampling. Didn't happen to see anything I missed outside of the other diffusion sampler that seems to have caveats. > Is there any chance of getting a community page [...] This could even be community created content if there was a place for it to exist. desu, I'm not interested in being involved in a community any more than staying in my own sphere here. But, I would prefer to keep it that way. It's already a bit astonishing seeing the reach this repo has spread from my only main involvement being a rentry on /v/ as a brief simple guide. I've had a lengthier response, but as I mentioned before, it's rather blunt and outside of the scope here. Someone else will do it. Not me.

mrq closed this issue

2023-03-13 17:39:05 +00:00

Sign in to join this conversation.