unload_tts() doesnt unload the voice model from video memory #349

New Issue

ben_mkiv · 2023-08-26T14:46:52Z

ben_mkiv commented

2023-08-26 14:46:52 +00:00

hi,
noticed that there seems to be an issue with unload_tts() not removing the model from the video memory.
i've noticed that the main python process took 4GB video memory while training + the training thread taking another 6GB

starting the webui with --defer-tts-load parameter results in the kinda expected behavior, (the main thread still uses ~100MB video memory, but that's pretty much nothing compared to 4GB...)

edit: all of the above was only tested with tortoise

hi, noticed that there seems to be an issue with unload_tts() not removing the model from the video memory. i've noticed that the main python process took 4GB video memory while training + the training thread taking another 6GB starting the webui with `--defer-tts-load` parameter results in the kinda expected behavior, (the main thread still uses ~100MB video memory, but that's pretty much nothing compared to 4GB...) edit: all of the above was only tested with tortoise

mrq commented

2023-08-26 14:58:38 +00:00

def unload_tts():
	global tts

	if tts:
		del tts
		tts = None
		print("Unloaded TTS")
	do_gc()

I'm pretty sure there's a magical Python issue where this isn't actually working as intended. I remember that Defer TTS Load is pretty much a requirement for the non-TTS tasks (transcription, training) because of this issue.

I'm not really sure what to do about it outside of moving the tts.autoregressive_model and company to cpu with tts.autoregressive_model.cpu() before the delete and crossing my fingers.

``` def unload_tts(): global tts if tts: del tts tts = None print("Unloaded TTS") do_gc() ``` I'm pretty sure there's a magical Python issue where this isn't actually working as intended. I remember that `Defer TTS Load` is pretty much a requirement for the non-TTS tasks (transcription, training) because of this issue. I'm not really sure what to do about it outside of moving the `tts.autoregressive_model` and company to cpu with `tts.autoregressive_model.cpu()` before the delete and crossing my fingers.

ben_mkiv commented

2023-08-26 15:50:33 +00:00

i tried to find the root of the problem, but haven't succeed yet. thought you might have an idea.

~~is run_training() maybe already called on another thread/process and therefor not able to release the resources of the main thread?~~ did some tests and the process id for run_training() is the same as for the main thread, so it should be fine

i tried to find the root of the problem, but haven't succeed yet. thought you might have an idea. ~~is run_training() maybe already called on another thread/process and therefor not able to release the resources of the main thread?~~ did some tests and the process id for run_training() is the same as for the main thread, so it *should* be fine

Sign in to join this conversation.