mrq/tortoise-tts
4
6
Fork 15
Code Issues 9 Pull Requests 1 Packages Projects Releases Wiki Activity

Running out of memory despite being able to run the other gardio version? #2

New Issue
Closed
opened 2023-02-03 20:37:07 +00:00 by BugMonkey42335 · 3 comments
BugMonkey42335 commented 2023-02-03 20:37:07 +00:00
Copy Link

Attempting to run your version results in this error. Since I can run the other gardio version without issue, I shouldn't be running out of memory.

(base) C:\tortoise-tts>python app.py
C:\Users\Standard\anaconda3\lib\site-packages\paramiko\transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated
  "class": algorithms.Blowfish,
Traceback (most recent call last):
  File "C:\tortoise-tts\app.py", line 167, in <module>
    tts = TextToSpeech()
  File "C:\tortoise-tts\tortoise\api.py", line 253, in __init__
    self.clvp = self.clvp.to(self.device)
  File "C:\Users\Standard\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 989, in to
    return self._apply(convert)
  File "C:\Users\Standard\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 641, in _apply
    module._apply(fn)
  File "C:\Users\Standard\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 641, in _apply
    module._apply(fn)
  File "C:\Users\Standard\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 641, in _apply
    module._apply(fn)
  [Previous line repeated 6 more times]
  File "C:\Users\Standard\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 664, in _apply
    param_applied = fn(param)
  File "C:\Users\Standard\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 987, in convert
    return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking)
torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 4.00 GiB total capacity; 3.44 GiB already allocated; 0 bytes free; 3.52 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation.  See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF
Attempting to run your version results in this error. Since I can run the other gardio version without issue, I shouldn't be running out of memory. ``` (base) C:\tortoise-tts>python app.py C:\Users\Standard\anaconda3\lib\site-packages\paramiko\transport.py:219: CryptographyDeprecationWarning: Blowfish has been deprecated "class": algorithms.Blowfish, Traceback (most recent call last): File "C:\tortoise-tts\app.py", line 167, in <module> tts = TextToSpeech() File "C:\tortoise-tts\tortoise\api.py", line 253, in __init__ self.clvp = self.clvp.to(self.device) File "C:\Users\Standard\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 989, in to return self._apply(convert) File "C:\Users\Standard\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 641, in _apply module._apply(fn) File "C:\Users\Standard\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 641, in _apply module._apply(fn) File "C:\Users\Standard\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 641, in _apply module._apply(fn) [Previous line repeated 6 more times] File "C:\Users\Standard\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 664, in _apply param_applied = fn(param) File "C:\Users\Standard\anaconda3\lib\site-packages\torch\nn\modules\module.py", line 987, in convert return t.to(device, dtype if t.is_floating_point() or t.is_complex() else None, non_blocking) torch.cuda.OutOfMemoryError: CUDA out of memory. Tried to allocate 20.00 MiB (GPU 0; 4.00 GiB total capacity; 3.44 GiB already allocated; 0 bytes free; 3.52 GiB reserved in total by PyTorch) If reserved memory is >> allocated memory try setting max_split_size_mb to avoid fragmentation. See documentation for Memory Management and PYTORCH_CUDA_ALLOC_CONF ```
mrq commented 2023-02-03 21:35:20 +00:00
Owner
Copy Link

Rats. I suppose that's why it wasn't merged in the original repo. Some anon(s) mentioned a small performance patch that worked fine on my 2060. I suppose it's enough to make cards boasting 4GiB of VRAM to fail.

My apologies. I'll see about reverting it while maintaining the patches for Gradio's progress stuff, or at the very least adding a low VRAM flag to either forgo those optimizations, or to lower the batch size a smidge.

I should have a solution for it by the end of today, if time allows.

Rats. I suppose that's why it wasn't merged in the original repo. Some [anon(s)](https://desuarchive.org/g/search/text/4bb355b6d96a52c8dfa8ccab759003e5e3e18831) mentioned a small performance [patch](https://github.com/neonbjb/tortoise-tts/blob/4bb355b6d96a52c8dfa8ccab759003e5e3e18831/tortoise/api.py) that worked fine on my 2060. I suppose it's enough to make cards boasting 4GiB of VRAM to fail. My apologies. I'll see about reverting it while maintaining the patches for Gradio's progress stuff, or at the very least adding a low VRAM flag to either forgo those optimizations, or to lower the batch size a smidge. I should have a solution for it by the end of today, if time allows.
mrq commented 2023-02-03 21:47:03 +00:00
Owner
Copy Link

Reverted the patch in commit 8f20afc18f for now. I'll try and see about cramming in a way to toggle between using those optimizations or not. It should be easy.

Reverted the patch in commit 8f20afc18f8fe3be36daee99be591d302d54ac1f for now. I'll try and see about cramming in a way to toggle between using those optimizations or not. It should be easy.
BugMonkey42335 commented 2023-02-04 19:09:19 +00:00
Author
Copy Link

For anyone reading this in the future, they added an optinal --low-vram flag into the start.bat script.

call .\tortoise-venv\Scripts\activate.bat
py .\app.py --low-vram
deactivate
For anyone reading this in the future, they added an optinal --low-vram flag into the start.bat script. ``` call .\tortoise-venv\Scripts\activate.bat py .\app.py --low-vram deactivate ```
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
No items
No Label
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: mrq/tortoise-tts#2
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?