mrq/ai-voice-cloning
31
74
Fork 43
Code Issues 223 Pull Requests 5 Packages Projects Releases Wiki Activity

gradio.exceptions.Error: 'torch.cat(): expected a non-empty list of Tensors' #21

New Issue
Closed
opened 2023-02-21 20:55:00 +00:00 by mrchucksneed · 8 comments
mrchucksneed commented 2023-02-21 20:55:00 +00:00
Copy Link

You're gonna hate me...

image

Setting samples in Gradio to any number divides the output by 12, 16 etc?
Setting a low enough number will cause an error:

gradio.exceptions.Error: 'torch.cat(): expected a non-empty list of Tensors'
You're gonna hate me... ![image](/attachments/1550a00a-cdaf-40fd-a14d-db0730a740b8) Setting samples in Gradio to any number divides the output by 12, 16 etc? Setting a low enough number will cause an error: ``` gradio.exceptions.Error: 'torch.cat(): expected a non-empty list of Tensors' ```
image.png
114 KiB
mrq commented 2023-02-21 21:21:48 +00:00
Owner
Copy Link

Setting samples in Gradio to any number divides the output by 12, 16 etc?

That's from it automatically deducing your batch size. On DirectML, there doesn't seem to be an (easy) way to get VRAM capacity, but it's exposed with ROCm, so it's using larger batch sizes.

Setting a low enough number will cause an error:

I suppose I technically never set it to a low enough number, since 16 is the lowest I'll go, and the batch-size-deduction from original tortoise-tts I think goes up to a batch size of 16.

Can you provide a full stack trace? I'm pretty sure I won't need one, since I'll just clamp down the batch size to the sample size. Actually, I should be able to replicate it.

> Setting samples in Gradio to any number divides the output by 12, 16 etc? That's from it automatically deducing your batch size. On DirectML, there doesn't seem to be an (easy) way to get VRAM capacity, but it's exposed with ROCm, so it's using larger batch sizes. > Setting a low enough number will cause an error: I suppose I technically never set it to a low enough number, since 16 is the lowest I'll go, and the batch-size-deduction from original tortoise-tts I think goes up to a batch size of 16. ~~Can you provide a full stack trace? I'm pretty sure I won't need one, since I'll just clamp down the batch size to the sample size.~~ Actually, I should be able to replicate it.
mrchucksneed commented 2023-02-21 21:32:49 +00:00
Author
Copy Link

That's from it automatically deducing your batch size. On DirectML, there doesn't seem to be an (easy) way to get VRAM capacity, but it's exposed with ROCm, so it's using larger batch sizes.

I see, when I was on Windows using the Generate Tab, it would always show the exact number of samples i set (if I set 100 under samples it would show 1/100 steps etc).

> That's from it automatically deducing your batch size. On DirectML, there doesn't seem to be an (easy) way to get VRAM capacity, but it's exposed with ROCm, so it's using larger batch sizes. I see, when I was on Windows using the Generate Tab, it would always show the exact number of samples i set (if I set 100 under samples it would show 1/100 steps etc).
mrchucksneed commented 2023-02-21 21:41:12 +00:00
Author
Copy Link

Can you provide a full stack trace? I'm pretty sure I won't need one, since I'll just clamp down the batch size to the sample size. Actually, I should be able to replicate it.

This is with Samples set to 9:

Reading from latent: ./voices/JayDee/cond_latents.pth
[1/1] Generating line: Your bussy is no match for my rock hard cock, Potter....
Generating autoregressive samples
Computing best candidates using CLVP
Traceback (most recent call last):
  File "/media/user/Scratch/ai-voice-cloning/src/webui.py", line 49, in run_generation
    sample, outputs, stats = generate(
  File "/media/user/Scratch/ai-voice-cloning/src/utils.py", line 224, in generate
    gen, additionals = tts.tts(cut_text, **settings )
  File "/media/user/Scratch/ai-voice-cloning/tortoise-tts/tortoise/api.py", line 589, in tts
    clip_results = torch.cat(clip_results, dim=0)
RuntimeError: torch.cat(): expected a non-empty list of Tensors

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/gradio/routes.py", line 384, in run_predict
    output = await app.get_blocks().process_api(
  File "/home/user/.local/lib/python3.10/site-packages/gradio/blocks.py", line 1024, in process_api
    result = await self.call_function(
  File "/home/user/.local/lib/python3.10/site-packages/gradio/blocks.py", line 836, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/user/.local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/home/user/.local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/home/user/.local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/home/user/.local/lib/python3.10/site-packages/gradio/helpers.py", line 584, in tracked_fn
    response = fn(*args)
  File "/media/user/Scratch/ai-voice-cloning/src/webui.py", line 78, in run_generation
    raise gr.Error(message)
gradio.exceptions.Error: 'torch.cat(): expected a non-empty list of Tensors'
> ~~Can you provide a full stack trace? I'm pretty sure I won't need one, since I'll just clamp down the batch size to the sample size.~~ Actually, I should be able to replicate it. > > This is with Samples set to 9: ``` Reading from latent: ./voices/JayDee/cond_latents.pth [1/1] Generating line: Your bussy is no match for my rock hard cock, Potter.... Generating autoregressive samples Computing best candidates using CLVP Traceback (most recent call last): File "/media/user/Scratch/ai-voice-cloning/src/webui.py", line 49, in run_generation sample, outputs, stats = generate( File "/media/user/Scratch/ai-voice-cloning/src/utils.py", line 224, in generate gen, additionals = tts.tts(cut_text, **settings ) File "/media/user/Scratch/ai-voice-cloning/tortoise-tts/tortoise/api.py", line 589, in tts clip_results = torch.cat(clip_results, dim=0) RuntimeError: torch.cat(): expected a non-empty list of Tensors During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/user/.local/lib/python3.10/site-packages/gradio/routes.py", line 384, in run_predict output = await app.get_blocks().process_api( File "/home/user/.local/lib/python3.10/site-packages/gradio/blocks.py", line 1024, in process_api result = await self.call_function( File "/home/user/.local/lib/python3.10/site-packages/gradio/blocks.py", line 836, in call_function prediction = await anyio.to_thread.run_sync( File "/home/user/.local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/home/user/.local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/home/user/.local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, *args) File "/home/user/.local/lib/python3.10/site-packages/gradio/helpers.py", line 584, in tracked_fn response = fn(*args) File "/media/user/Scratch/ai-voice-cloning/src/webui.py", line 78, in run_generation raise gr.Error(message) gradio.exceptions.Error: 'torch.cat(): expected a non-empty list of Tensors' ```
mrq commented 2023-02-21 21:51:19 +00:00
Owner
Copy Link

Should be fixed in commit 9e64dad785. I can't do any tests for it on my 2060 since I can't increase it's batch size all that much for higher numbers.

Should be fixed in commit 9e64dad785bc6d6e0e948784b27028d5d4746e25. I can't do any tests for it on my 2060 since I can't increase it's batch size all that much for higher numbers.
mrchucksneed commented 2023-02-21 21:59:02 +00:00
Author
Copy Link

Should be fixed in commit 9e64dad785. I can't do any tests for it on my 2060 since I can't increase it's batch size all that much for higher numbers.

Reading from latent: ./voices/JayDee/cond_latents.pth
Traceback (most recent call last):
  File "/media/user/Scratch/ai-voice-cloning/src/webui.py", line 49, in run_generation
    sample, outputs, stats = generate(
  File "/media/user/Scratch/ai-voice-cloning/src/utils.py", line 156, in generate
    if num_autoregressive_samples < args.sample_batch_size:
TypeError: '<' not supported between instances of 'int' and 'NoneType'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/user/.local/lib/python3.10/site-packages/gradio/routes.py", line 384, in run_predict
    output = await app.get_blocks().process_api(
  File "/home/user/.local/lib/python3.10/site-packages/gradio/blocks.py", line 1024, in process_api
    result = await self.call_function(
  File "/home/user/.local/lib/python3.10/site-packages/gradio/blocks.py", line 836, in call_function
    prediction = await anyio.to_thread.run_sync(
  File "/home/user/.local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
    return await get_asynclib().run_sync_in_worker_thread(
  File "/home/user/.local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
    return await future
  File "/home/user/.local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
    result = context.run(func, *args)
  File "/home/user/.local/lib/python3.10/site-packages/gradio/helpers.py", line 584, in tracked_fn
    response = fn(*args)
  File "/media/user/Scratch/ai-voice-cloning/src/webui.py", line 78, in run_generation
    raise gr.Error(message)
gradio.exceptions.Error: "'<' not supported between instances of 'int' and 'NoneType'"

This was the output after updating.

> Should be fixed in commit 9e64dad785bc6d6e0e948784b27028d5d4746e25. I can't do any tests for it on my 2060 since I can't increase it's batch size all that much for higher numbers. ``` Reading from latent: ./voices/JayDee/cond_latents.pth Traceback (most recent call last): File "/media/user/Scratch/ai-voice-cloning/src/webui.py", line 49, in run_generation sample, outputs, stats = generate( File "/media/user/Scratch/ai-voice-cloning/src/utils.py", line 156, in generate if num_autoregressive_samples < args.sample_batch_size: TypeError: '<' not supported between instances of 'int' and 'NoneType' During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/home/user/.local/lib/python3.10/site-packages/gradio/routes.py", line 384, in run_predict output = await app.get_blocks().process_api( File "/home/user/.local/lib/python3.10/site-packages/gradio/blocks.py", line 1024, in process_api result = await self.call_function( File "/home/user/.local/lib/python3.10/site-packages/gradio/blocks.py", line 836, in call_function prediction = await anyio.to_thread.run_sync( File "/home/user/.local/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/home/user/.local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/home/user/.local/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, *args) File "/home/user/.local/lib/python3.10/site-packages/gradio/helpers.py", line 584, in tracked_fn response = fn(*args) File "/media/user/Scratch/ai-voice-cloning/src/webui.py", line 78, in run_generation raise gr.Error(message) gradio.exceptions.Error: "'<' not supported between instances of 'int' and 'NoneType'" ``` This was the output after updating.
mrq commented 2023-02-21 22:14:00 +00:00
Owner
Copy Link

Oops, a slight oversight. Should be fixed in commit fefc7aba03.

Oops, a slight oversight. Should be fixed in commit fefc7aba0388d061a2db37cd74ebef7a9172439b.
mrchucksneed commented 2023-02-21 22:19:11 +00:00
Author
Copy Link

Oops, a slight oversight. Should be fixed in commit fefc7aba03.

I'm either really thick or I'm missing something, when I read samples in the UI, I'm actually reading the batch size of 16? So when I set 256 samples, it's 16 batches of 16 samples in the progress field on the right hand side of gradio?
Cause like I said a little earlier, when I was on Windows and set say 100 samples, it would show while generating on the right hand side of gradio "1/100 steps" etc.

It is working again however, thank you.

> Oops, a slight oversight. Should be fixed in commit fefc7aba0388d061a2db37cd74ebef7a9172439b. I'm either really thick or I'm missing something, when I read samples in the UI, I'm actually reading the batch size of 16? So when I set 256 samples, it's 16 batches of 16 samples in the progress field on the right hand side of gradio? Cause like I said a little earlier, when I was on Windows and set say 100 samples, it would show while generating on the right hand side of gradio "1/100 steps" etc. It is working again however, thank you.
mrq commented 2023-02-21 22:30:12 +00:00
Owner
Copy Link

The Sample slider in the Generate tab will only affect how many (autoregressive) samples you want.

The progress bar on the right will report [Current Batch / Total Batches], where Total Batches = Autoregressive Samples / Batch Size (or exchange batch with step or iteration).

So when I set 256 samples, it's 16 batches of 16 samples in the progress field on the right hand side of gradio?

Yep. If you do 128 samples, it's 8 batches of 16 samples, while 96 samples will give you 6 batches of 16 samples.

I wouldn't worry much about it (especially in terms of finetuning/training), just know more samples will give better output, while higher batch sizes will make things faster, at the cost of VRAM.

Cause like I said a little earlier, when I was on Windows and set say 100 samples, it would show while generating on the right hand side of gradio "1/100 steps" etc.

Yep, since with DirectML, it'll default to a batch size of 1, as there's no way to gauge VRAM size, as the only real limitation for increasing batch size is from VRAM.

The Sample slider in the Generate tab will only affect how many (autoregressive) samples you want. The progress bar on the right will report `[Current Batch / Total Batches]`, where `Total Batches = Autoregressive Samples / Batch Size` (or exchange batch with step or iteration). > So when I set 256 samples, it's 16 batches of 16 samples in the progress field on the right hand side of gradio? Yep. If you do 128 samples, it's 8 batches of 16 samples, while 96 samples will give you 6 batches of 16 samples. I wouldn't worry much about it (especially in terms of finetuning/training), just know more samples will give better output, while higher batch sizes will make things faster, at the cost of VRAM. > Cause like I said a little earlier, when I was on Windows and set say 100 samples, it would show while generating on the right hand side of gradio "1/100 steps" etc. Yep, since with DirectML, it'll default to a batch size of 1, as there's no way to gauge VRAM size, as the only real limitation for increasing batch size is from VRAM.
mrq closed this issue 2023-02-21 22:31:02 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
help wanted

Need some help

  
insufficient info

Need more information

  
invalid

Something is wrong

  
news

To make up for a lack of a way to provide any news-like

  
not a bug

For non-bug things

  
question

More information is needed

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
insufficient info
invalid
news
not a bug
question
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
2 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: mrq/ai-voice-cloning#21
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?