• https://git.ecker.tech/ aims to provide a place to share my efforts while maintaining true ownership of my code, as I do not trust GitHub.

    XMR: 4B9TQdkAkBFYrbj5ztvTx89e5LpucPeTSPzemCihdDi9EBnx7btn8RDNZTBz2zihWsjMnDkzn5As1LU6gLv3KQy8BLsZ8SG

  • Joined on 2022-10-10
ecker commented on issue ecker/ai-voice-cloning#152 2023-07-01 21:43:17 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

Oh yeah, I'll see what I can grab. For sure I'll grab the latest examples to test, and I'll load back the old models and do an eval to spit out audio with Vocos.

I've had the quarter-sized…

ecker commented on issue ecker/ai-voice-cloning#152 2023-06-27 20:07:10 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

I'm really not too sure what to make of these results. There's some things that make sense in hindsight, but there's a lot of weird shit with this new run for the quarter-sized model at 1.5kbps /…

ecker pushed to master at ecker/vall-e 2023-06-26 15:35:00 +00:00
2b70aa4f8f added ability to modify accepted RVQ-bin levels (default 8, for 6kbps Encodec)
ecker commented on issue ecker/ai-voice-cloning#152 2023-06-26 03:21:52 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

The first epoch has concluded for the fullsize model, I'm not too pleased since training definitely petered off, but at least it's still going down. ![image](/attachments/64430d14-fdd4-4dd1-863b-…

ecker pushed to master at ecker/vall-e 2023-06-26 01:57:18 +00:00
e1853c77aa crammed vocos into it
ecker commented on issue ecker/ai-voice-cloning#152 2023-06-24 02:25:20 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

There's a new neural vocoder that might be worth checking out called 'Vocos'. It was made for bark TTS, and sounds like an improvement to bare EnCodec. The demo doesn't compare it to any other…

ecker pushed to master at ecker/vall-e 2023-06-21 03:06:50 +00:00
3b6901634b fixes with the hdf5 creator, added lzf compression to it, and deduced chunking for the audio quants when storing them
ecker commented on issue ecker/ai-voice-cloning#152 2023-06-21 02:45:07 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

Progress report: image

  • 2200+ 3400+ hours is definitely having the loss go down faster than with the 550+ hour dataset, obviously. …
ecker pushed to master at ecker/vall-e 2023-06-21 01:38:56 +00:00
1c091e48a5 moved aggressive (possibly superfluous) optimization to yaml bool, added support for converting/loading from an hdf5 dataset
ecker commented on issue ecker/ai-voice-cloning#269 2023-06-17 19:27:12 +00:00
Custom tokenizer

mmmmmmm

For Russian, I can't imagine you needing to use a new tokenizer definition for Russian. If you want to be extra sure, I believe it's Utilities > Tokenizer, and you can type in…

ecker commented on issue ecker/ai-voice-cloning#270 2023-06-17 19:17:06 +00:00
models merging

If I remember right, the model merging code is shamelessly (but credited) lifted from voldy Automatic1111's Stable…

ecker commented on issue ecker/ai-voice-cloning#152 2023-06-17 19:06:13 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

It being very, very general purpose for being able to do a variety of inputs + infilling seems very, very promising solely in the new model architecture (getting a bit tired of the typical…

ecker commented on issue ecker/ai-voice-cloning#152 2023-06-16 03:39:47 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

Training the quarter sized model with the new dataset has stabilized. I don't know whether it's:

  • skipping "validating" the phonemes, which requires loading every file several times and some…
ecker commented on issue ecker/ai-voice-cloning#152 2023-06-15 02:32:26 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

https://github.com/descriptinc/descript-audio-codec

can do 44.1KHz at 8kbps of bandwidth

Sob. I JUST quantized everything.

ecker commented on issue ecker/ai-voice-cloning#152 2023-06-14 15:19:46 +00:00
VALL-E Integration (and In Response To TorToiSe: a Quick Retrospective)

Training has resumed after a few days to transcribe the audiobooks + re-quantize the LibriTTS_R dataset. I think unironically, quantizing the audio through Encodec on my 4070Ti is much slower than…

ecker commented on issue ecker/ai-voice-cloning#266 2023-06-14 12:45:28 +00:00
"Original Latents Method" - What is this for, and what is the best option?

Couldn't find any mention of them in the wiki or the issues

Whoops, I forgot to add it to the documentation, but a loose idea of it is in mrq/ai-voice-cloning#243

ecker pushed to master at ecker/vall-e 2023-06-14 01:55:42 +00:00
a6c3cd35d6 oops, never actually commited the proper prepare script
ecker commented on issue ecker/ai-voice-cloning#264 2023-06-12 23:10:10 +00:00
Is there anyway to save the voice from a random generation ?

If you have Embed Output Metadata enabled in settings, the latents used for that generation are "embedded" into the result sound file. You can take that into Utilities > Import / Analyze and…

ecker commented on issue ecker/ai-voice-cloning#263 2023-06-12 18:28:27 +00:00
cli.py loads model everytime it's ran

I guess this is intented behaviour

"Intentional" in the sense there's no way to keep it loaded without it being a server such as with the main script, yes, it's intentional. There's no way to…

ecker commented on issue ecker/ai-voice-cloning#242 2023-06-11 04:49:22 +00:00
Is it possible to generate using the command-line?

Sorry it took so long, especially for how simple the cli.py script ended up being. My work ethic has taken quite the nosedive.

Should be added in commit 76ed34ddd2e610d466cf2c6fb76e13eea6df27b…