mrq
18 Followers · 0 Following

  • https://git.ecker.tech/ aims to provide a place to share my efforts while maintaining true ownership of my code, as I do not trust GitHub.

    XMR: 4B9TQdkAkBFYrbj5ztvTx89e5LpucPeTSPzemCihdDi9EBnx7btn8RDNZTBz2zihWsjMnDkzn5As1LU6gLv3KQy8BLsZ8SG

  • Joined on 2022-10-10
Repositories
15
 Projects Packages Public Activity Starred Repositories
mrq pushed to master at mrq/vall-e 2025-03-26 02:57:16 +00:00
ef0fd0c8ac fixes for non-phonemized text input
mrq pushed to master at mrq/vall-e 2025-03-26 02:28:07 +00:00
476d87d4aa cringe fix because I guess I moved which logit gets trained for len duration (I should probably rethink this)
mrq pushed to master at mrq/vall-e 2025-03-25 23:51:22 +00:00
a1eb96e6c1 re-enable this (nothing could go wrong)
mrq pushed to master at mrq/vall-e 2025-03-25 23:48:12 +00:00
aa8b32d97e added more notes (although I could have sworn I have had more notes that i can't recall)
mrq pushed to master at mrq/vall-e 2025-03-24 15:16:55 +00:00
df5b870908 added remark about not using sliding attention
02a8bcbe29 fixed errant index error (although it makes me wonder if my segmented masking is still flawed)
Compare 2 commits »
mrq pushed to master at mrq/vall-e 2025-03-22 00:01:32 +00:00
d1d91295b3 add segmented sliding attention, also found a bug with prom-less segments in the attention mask generation.........
mrq pushed to master at mrq/vall-e 2025-03-20 20:57:30 +00:00
8068f24e35 cleaned up parallel nar, i think it's slightly faster but even the smallest model is still slower than ar+nar-len-llama-8...
mrq pushed to master at mrq/vall-e 2025-03-20 03:36:49 +00:00
9a7458cf17 fixed inferencing since I did delete the len_emb, some more notes on the model since it seems I just had bad experimental settings
mrq pushed to master at mrq/vall-e 2025-03-19 19:15:34 +00:00
61de653ad9 now causal training should work again
mrq pushed to master at mrq/vall-e 2025-03-19 18:27:18 +00:00
85b9dd47c1 ugh
mrq pushed to master at mrq/vall-e 2025-03-19 01:57:16 +00:00
81acd565b3 re-enable these
mrq pushed to master at mrq/vall-e 2025-03-19 00:29:59 +00:00
5479d2eacc more tweaks to the new implementation (properly trim the len stuff to save some params, decoder to d_ffn expansion to 2 to maybe also make it faster, etc.)
mrq pushed to master at mrq/vall-e 2025-03-18 23:28:50 +00:00
9a8a8e3195 off by one bateman
mrq pushed to master at mrq/vall-e 2025-03-18 02:44:45 +00:00
0280e72257 ugh
b0dba9db07 this may bite me in the ass
Compare 2 commits »
mrq pushed to master at mrq/vall-e 2025-03-16 16:26:08 +00:00
2dfef693c4 comments for clarity
mrq pushed to master at mrq/vall-e 2025-03-16 01:13:56 +00:00
c5475ebc91 another dataloader optimization
mrq pushed to master at mrq/vall-e 2025-03-15 21:45:45 +00:00
bee2688dea ugh
mrq pushed to master at mrq/vall-e 2025-03-15 18:09:39 +00:00
2053580838 updated dataloader to hopefully reduce RAM usage
mrq pushed to master at mrq/vall-e 2025-03-15 01:25:43 +00:00
9cfbf94b1c config-ify the len_loss_factor
mrq pushed to master at mrq/vall-e 2025-03-15 01:13:28 +00:00
ca8cc15271 more tweaks (vall_e.webui --yaml still breaks things, --model needs to deduce what audio backend now that im supporting other ones again // added easy top-sampler settings back for new implementation)