• https://git.ecker.tech/ aims to provide a place to share my efforts while maintaining true ownership of my code, as I do not trust GitHub.

    XMR: 4B9TQdkAkBFYrbj5ztvTx89e5LpucPeTSPzemCihdDi9EBnx7btn8RDNZTBz2zihWsjMnDkzn5As1LU6gLv3KQy8BLsZ8SG

  • Joined on 2022-10-10
ecker pushed to master at ecker/ai-voice-cloning 2023-04-08 14:04:47 +00:00
b785192dfc Merge pull request 'Make convenient to use with Docker' (#191) from psr/ai-voice-cloning:docker into master
9afafc69c1 docker: add training script
c018bfca9c docker: add ffmpeg for whisper and general cleanup
d64cba667f docker support
Compare 4 commits »
ecker pushed to master at ecker/vall-e 2023-04-07 15:22:40 +00:00
99c1b4cf9f do GC on save/eval because it will OOM with high enough memory pressure
ecker pushed to master at ecker/vall-e 2023-04-07 15:01:22 +00:00
52a4d73970 make GC a config var to either do it every 'step' or 'substep' or neither (as it eats 0.20 seconds every step and isn't necessary)
4768f39e87 one more thing
Compare 2 commits »
ecker pushed to master at ecker/vall-e 2023-04-07 02:53:35 +00:00
5c6ec5a639 jostle the start and end points of when to trim down the training input prompt
ecker pushed to master at ecker/vall-e 2023-04-07 02:44:29 +00:00
b6e4816adf modified how input prompts for training are sampled (adheres more to the VALL-E paper, drastically reduces VRAM usage)
ecker commented on issue ecker/ai-voice-cloning#152 2023-04-07 01:56:09 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

I swear every time I grow a wild hair and dive back into the implementation I forked, there's another twist with how it behaves.

To be brief, I was trying to both have a way to calculate…

ecker commented on issue ecker/ai-voice-cloning#193 2023-04-07 01:44:32 +00:00
Can you add to an already trained model?

You can go back and "add" to finetune by reusing the finetunes existing weights, yeah. I haven't done it specifically for TorToiSe, but I don't see it wouldn't work outside of "quality" concerns. …

ecker commented on issue ecker/ai-voice-cloning#152 2023-04-06 22:59:02 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

Have you checked out the Pony Preservation Project Datasets You can found them here:

Oh right, I forgot I can leverage /mlp/'s autism. I'll nab them too for the next feeding time as well. I'm…

ecker commented on issue ecker/ai-voice-cloning#152 2023-04-06 15:09:46 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

Would you like me to download and link these for you? Or can you do it on your end?

I can rip it with yt-dlp and transcribe from there. I'll add them into a next batch after this one gets…

ecker commented on issue ecker/ai-voice-cloning#152 2023-04-05 15:57:22 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

How many segments minimum?

Not sure; I still have about 20 voices that have sub-50 lines that I'm not too sure how much would help shape things, but I imagine at least 10 lines would be…

ecker commented on issue ecker/ai-voice-cloning#152 2023-04-04 18:37:44 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

So, I was letting the FUD get to me about whether or not I should have backed the newer implementation instead. I was getting the model code transplanted into my fork, and as I was stitching up…

ecker commented on issue ecker/ai-voice-cloning#152 2023-04-03 22:33:32 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

Thoughts?

Unless I'm misunderstanding it:

  • isn't it primarily for speech-to-speech?
  • isn't it also only really best for singing?

I suppose it can fill a specific niche, but those two…

ecker commented on issue ecker/ai-voice-cloning#152 2023-04-01 18:03:13 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

I guess at that point, if what you say can produce the segments, then all we would need to do is feed him the data/anime?

Yeah. I pretty much just need the audio, and WhisperX / the…

ecker commented on issue ecker/ai-voice-cloning#152 2023-04-01 18:01:10 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

tldr You should enforce high example standards

I'm not even sure if I need high standards. WhisperX does a damn decent job now at transcription and timestamping, and the VALL-E paper says…

ecker pushed to master at ecker/vall-e 2023-04-01 16:49:59 +00:00
d0a6e02f24 downcast to uint8/int16 phonemes/quants for inferencing, not sure if that matters
ecker pushed to main at ecker/tortoise-tts 2023-04-01 16:37:58 +00:00
815ae5d707 Merge pull request 'feat: support .flac voice files' (#43) from NtTestAlert/tortoise-tts:support_flac_voice into main
2cd7b72688 feat: support .flac voice files
Compare 2 commits »
ecker deleted branch support_flac_voice from NtTestAlert/tortoise-tts 2023-04-01 16:37:58 +00:00
ecker merged pull request ecker/tortoise-tts#43 2023-04-01 16:37:57 +00:00
feat: support .flac voice files
ecker commented on issue ecker/ai-voice-cloning#152 2023-03-31 18:47:51 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

Progress report:

Inferencing with VALL-E is integrated into the web UI, with some warts.

  • not very many knobs to play with, only a maximum step count and sampling temperatures. I'm sure I…
ecker pushed to master at ecker/ai-voice-cloning 2023-03-31 06:56:02 +00:00