CUDA out of memory - all of a sudden? #87

New Issue

nirurin · 2023-03-07T19:34:29Z

nirurin commented

2023-03-07 19:34:29 +00:00

So today, whenever I try and generate a voice, it goes through a few minutes of the usual loading, and then throws this OOM error. Never used to happen, as I'm on a 3090. Previously I'd only be using about 16gb of memory for generations like this.

Edit - It seems to succeed if I use the 'univan' vocoder, but fails with either of the default bigvgan vocoders?

So today, whenever I try and generate a voice, it goes through a few minutes of the usual loading, and then throws this OOM error. Never used to happen, as I'm on a 3090. Previously I'd only be using about 16gb of memory for generations like this. Edit - It seems to succeed if I use the 'univan' vocoder, but fails with either of the default bigvgan vocoders?

Screenshot 2023-03-07 190929.png

160 KiB

mrq commented

2023-03-07 19:36:25 +00:00

Go under settings and set Sample Batch Size to 16. If you updated mrq/tortoise-tts recently, it added an additional automatic batch size tier, that I suppose isn't actually safe to use for lengthier sentences. I'll revert it in a moment.

mrq referenced this issue from a commit

2023-03-07 19:38:12 +00:00

reverting additional auto-suggested batch sizes, per https://git.ecker.tech/mrq/ai-voice-cloning/issues/87 proving it in fact, is not a good idea

mrq commented

2023-03-07 19:40:24 +00:00

Reverted in mrq/tortoise-tts commit 3dd5cad324.

In the future, if it does happen again, you'll need to lower the sample batch size under settings.

Reverted in mrq/tortoise-tts commit https://git.ecker.tech/mrq/tortoise-tts/commit/3dd5cad32429b8cf0e2864e07799a7626a067150. In the future, if it does happen again, you'll need to lower the sample batch size under settings.

mrq closed this issue

2023-03-07 19:40:27 +00:00

mrq referenced this issue

2023-03-07 19:41:12 +00:00

20% Inference Speed increase for Large VRAM (3090+) GPUS #84

Sign in to join this conversation.