mrq/ai-voice-cloning
31
74
Fork 43
Code Issues 223 Pull Requests 5 Packages Projects Releases Wiki Activity

Long prompts and random voice #85

New Issue
Closed
opened 2023-03-07 16:39:08 +00:00 by st33lmouse · 6 comments
st33lmouse commented 2023-03-07 16:39:08 +00:00
Copy Link

Long prompts often do not survive the generation to the end of the prompt. I know that a solution is to use the \n delimiter to break it into pieces but...

if you use delimiter and random voice, it changes to a new random voice on every line. I'm not sure about this exactly, but each line sounds different, sometimes even switching genders.

If long prompts can be fixed, that would permit enough material to be generated on a single go with the same random voice to save it as a new, permanent and stable voice, or to turn it into a new model.

OR

if random voice can be prevented from switching to a new voice on each delimited line, that would also do the trick.

OR

Sticking to the same random voice until you generate a new seed.

Any of these solutions would make generating new voices easy,and that would be a very nice feature.

Long prompts often do not survive the generation to the end of the prompt. I know that a solution is to use the \n delimiter to break it into pieces but... if you use delimiter and random voice, it changes to a new random voice on every line. I'm not sure about this exactly, but each line sounds different, sometimes even switching genders. If long prompts can be fixed, that would permit enough material to be generated on a single go with the same random voice to save it as a new, permanent and stable voice, or to turn it into a new model. OR if random voice can be prevented from switching to a new voice on each delimited line, that would also do the trick. OR Sticking to the same random voice until you generate a new seed. Any of these solutions would make generating new voices easy,and that would be a very nice feature.
deviandice commented 2023-03-07 16:59:22 +00:00
Copy Link

Each new line restarts the voice process. IMO, if you find a line you like, you should use that as your voice.

Otherwise you're asking a complete system rework.

Each new line restarts the voice process. IMO, if you find a line you like, you should use that as your voice. Otherwise you're asking a complete system rework.
mrq commented 2023-03-07 17:07:55 +00:00
Owner
Copy Link

Long prompts often do not survive the generation to the end of the prompt.

Yeah, that's more of a limitation with TorToiSe itself.

if you use delimiter and random voice, it changes to a new random voice on every line.

Should be fixed in commit c27ee3ce95 (untested, as I have lunch to eat). I'll spare the gory details.

> Long prompts often do not survive the generation to the end of the prompt. Yeah, that's more of a limitation with TorToiSe itself. > if you use delimiter and random voice, it changes to a new random voice on every line. Should be fixed in commit c27ee3ce9540fa400a989ebe310d74a91cd4e4af (untested, as I have lunch to eat). I'll spare the gory details.
st33lmouse commented 2023-03-08 09:38:32 +00:00
Author
Copy Link

Did the update with update.sh and then tried update-forced.sh. script now starts, but:

regular voices work and generate speech. Random voice produces this error:

raceback (most recent call last):
File "/home/mouse/ai-voice-cloning/venv/lib/python3.10/site-packages/gradio/routes.py", line 384, in run_predict
output = await app.get_blocks().process_api(
File "/home/mouse/ai-voice-cloning/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1032, in process_api
result = await self.call_function(
File "/home/mouse/ai-voice-cloning/venv/lib/python3.10/site-packages/gradio/blocks.py", line 844, in call_function
prediction = await anyio.to_thread.run_sync(
File "/home/mouse/ai-voice-cloning/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync
return await get_asynclib().run_sync_in_worker_thread(
File "/home/mouse/ai-voice-cloning/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread
return await future
File "/home/mouse/ai-voice-cloning/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run
result = context.run(func, *args)
File "/home/mouse/ai-voice-cloning/venv/lib/python3.10/site-packages/gradio/helpers.py", line 584, in tracked_fn
response = fn(*args)
File "/home/mouse/ai-voice-cloning/src/webui.py", line 84, in run_generation
raise gr.Error(message)
gradio.exceptions.Error: "'NoneType' object is not subscriptable"

Did the update with update.sh and then tried update-forced.sh. script now starts, but: regular voices work and generate speech. Random voice produces this error: raceback (most recent call last): File "/home/mouse/ai-voice-cloning/venv/lib/python3.10/site-packages/gradio/routes.py", line 384, in run_predict output = await app.get_blocks().process_api( File "/home/mouse/ai-voice-cloning/venv/lib/python3.10/site-packages/gradio/blocks.py", line 1032, in process_api result = await self.call_function( File "/home/mouse/ai-voice-cloning/venv/lib/python3.10/site-packages/gradio/blocks.py", line 844, in call_function prediction = await anyio.to_thread.run_sync( File "/home/mouse/ai-voice-cloning/venv/lib/python3.10/site-packages/anyio/to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "/home/mouse/ai-voice-cloning/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "/home/mouse/ai-voice-cloning/venv/lib/python3.10/site-packages/anyio/_backends/_asyncio.py", line 867, in run result = context.run(func, *args) File "/home/mouse/ai-voice-cloning/venv/lib/python3.10/site-packages/gradio/helpers.py", line 584, in tracked_fn response = fn(*args) File "/home/mouse/ai-voice-cloning/src/webui.py", line 84, in run_generation raise gr.Error(message) gradio.exceptions.Error: "'NoneType' object is not subscriptable"
mrq commented 2023-03-08 15:55:07 +00:00
Owner
Copy Link

Wrong stack trace.

Wrong stack trace.
st33lmouse commented 2023-03-08 16:03:02 +00:00
Author
Copy Link

Tried again. Is this what you're looking for?

[1/3] Generating line: This is a test
Loading voice: random with model d1f79232
Generating autoregressive samples
Computing best candidates using CLVP
Transforming autoregressive outputs into audio..
Transforming autoregressive outputs into audio..
Transforming autoregressive outputs into audio..
Generating line took 3.33890438079834 seconds
[2/3] Generating line:
Loading voice: random with model d1f79232
Generating autoregressive samples
Computing best candidates using CLVP
Transforming autoregressive outputs into audio..
Transforming autoregressive outputs into audio..
Transforming autoregressive outputs into audio..
Generating line took 2.805288553237915 seconds
[3/3] Generating line:
Loading voice: random with model d1f79232
Generating autoregressive samples
Computing best candidates using CLVP
Transforming autoregressive outputs into audio..
Transforming autoregressive outputs into audio..
Transforming autoregressive outputs into audio..
Generating line took 2.7975049018859863 seconds
Traceback (most recent call last):
File "/home/mouse/ai-voice-cloning/src/webui.py", line 55, in run_generation
sample, outputs, stats = generate(
File "/home/mouse/ai-voice-cloning/src/utils.py", line 400, in generate
'settings': get_info(voice=voice),
File "/home/mouse/ai-voice-cloning/src/utils.py", line 300, in get_info
if latents and settings['conditioning_latents']:
TypeError: 'NoneType' object is not subscriptable

Tried again. Is this what you're looking for? [1/3] Generating line: This is a test Loading voice: random with model d1f79232 Generating autoregressive samples Computing best candidates using CLVP Transforming autoregressive outputs into audio.. Transforming autoregressive outputs into audio.. Transforming autoregressive outputs into audio.. Generating line took 3.33890438079834 seconds [2/3] Generating line: Loading voice: random with model d1f79232 Generating autoregressive samples Computing best candidates using CLVP Transforming autoregressive outputs into audio.. Transforming autoregressive outputs into audio.. Transforming autoregressive outputs into audio.. Generating line took 2.805288553237915 seconds [3/3] Generating line: Loading voice: random with model d1f79232 Generating autoregressive samples Computing best candidates using CLVP Transforming autoregressive outputs into audio.. Transforming autoregressive outputs into audio.. Transforming autoregressive outputs into audio.. Generating line took 2.7975049018859863 seconds Traceback (most recent call last): File "/home/mouse/ai-voice-cloning/src/webui.py", line 55, in run_generation sample, outputs, stats = generate( File "/home/mouse/ai-voice-cloning/src/utils.py", line 400, in generate 'settings': get_info(voice=voice), File "/home/mouse/ai-voice-cloning/src/utils.py", line 300, in get_info if latents and settings['conditioning_latents']: TypeError: 'NoneType' object is not subscriptable
mrq commented 2023-03-08 17:04:28 +00:00
Owner
Copy Link

Should be fixed in commit 0ab091e7ff.

Should be fixed in commit 0ab091e7ff6fac4dee7582d44875493daaeea265.
mrq closed this issue 2023-03-08 17:04:28 +00:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels   
bug

Something is not working

  
duplicate

This issue or pull request already exists

  
enhancement

New feature

  
help wanted

Need some help

  
insufficient info

Need more information

  
invalid

Something is wrong

  
news

To make up for a lack of a way to provide any news-like

  
not a bug

For non-bug things

  
question

More information is needed

  
wontfix

This won't be fixed

No Label
bug
duplicate
enhancement
help wanted
insufficient info
invalid
news
not a bug
question
wontfix
Milestone
Clear milestone
No items
No Milestone
Projects
Clear projects
No project
Assignees
Clear assignees
No Assignees
3 Participants
Notifications
Due Date
The due date is invalid or out of range. Please use the format 'yyyy-mm-dd'.

No due date set.

Dependencies

No dependencies set.

Reference: mrq/ai-voice-cloning#85
No description provided.
Delete Branch "%!s()"

Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?