deviandice/DL-Art-School
1
0
Fork 0
forked from mrq/DL-Art-School
Code Issues Pull Requests Packages Projects Releases Wiki Activity
2,094 Commits 1 Branch 0 Tags 5.6 MiB
02ebda42f2
Commit Graph

2094 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
James Betker
02ebda42f2 #yolo 2022-07-21 00:43:03 -06:00
James Betker
b92ff8de78 misc 2022-07-20 23:59:32 -06:00
James Betker
a1743d26aa Revert "Try to squeeze a bit more performance out of this arch" 
This reverts commit 767f963392.
 2022-07-20 23:57:56 -06:00
James Betker
767f963392 Try to squeeze a bit more performance out of this arch 2022-07-20 23:51:11 -06:00
James Betker
b9d0f7e6de simplify parameterization a bit 2022-07-20 23:41:54 -06:00
James Betker
ee8ceed6da rework tfd13 further 
- use a gated activation layer for both attention & convs
- add a relativistic learned position bias. I believe this is similar to the T5 position encodings but it is simpler and learned
- get rid of prepending to the attention matrix - this doesn't really work that well. the model eventually learns to attend one of its heads to these blocks but why not just concat if it is doing that?
 2022-07-20 23:28:29 -06:00
James Betker
40427de8e3 update tfd13 for inference 2022-07-20 21:51:25 -06:00
James Betker
dbebe18602 Fix ts=0 with new formulation 2022-07-20 12:12:33 -06:00
James Betker
82bd62019f diffuse the cascaded prior for continuous sr model 2022-07-20 11:54:09 -06:00
James Betker
b0e3be0a17 transition to nearest interpolation mode for downsampling 2022-07-20 10:56:17 -06:00
James Betker
7b3fc79737 iq checkin 2022-07-20 10:19:32 -06:00
James Betker
9a37f3ba42 reminder to future self 2022-07-20 10:19:15 -06:00
James Betker
15decfdb98 misc 2022-07-20 10:19:02 -06:00
James Betker
2997a640b0 fix mdf 2022-07-19 19:39:29 -06:00
James Betker
c14bf6dfb2 fix conditioning free 2022-07-19 18:04:49 -06:00
James Betker
fc0b291b21 do masking up proper 2022-07-19 16:32:17 -06:00
James Betker
b203a7dc97 And remove unused parameters 2022-07-19 15:05:12 -06:00
James Betker
17a07b2e33 readd one mdf function 2022-07-19 15:04:36 -06:00
James Betker
c00398e955 scope attention in tfd13 as well 2022-07-19 14:59:43 -06:00
James Betker
b157b28c7b tfd14 
hopefully this helps address the positional dependencies of tfd12
 2022-07-19 13:30:05 -06:00
James Betker
4597447178 add assertions to mel generator script 2022-07-19 11:23:54 -06:00
James Betker
1b6fe88bcb spit out overages in GDI 2022-07-19 11:19:59 -06:00
James Betker
73d7211a4c fix script 2022-07-19 11:17:43 -06:00
James Betker
6b1cfe8e66 ugh 2022-07-19 11:14:20 -06:00
James Betker
da9e47ca0e new bounds for MEL normalization and multi-resolution SR in MDF 2022-07-19 11:11:46 -06:00
James Betker
eecb534e66 a few fixes to multiresolution sr 2022-07-19 11:11:15 -06:00
James Betker
2fb85526bc mdf cleanup 2022-07-19 09:57:05 -06:00
James Betker
4aa840a494 be more stringent on min and max ranges in GDI.. this is gonna break somethings probably 2022-07-19 09:14:08 -06:00
James Betker
eab7dc339d iq checkin 
who knows where I'm going with this.. I don't even know sometimes..
 2022-07-19 09:13:27 -06:00
James Betker
625d7b6f38 music joiner checkin 2022-07-18 18:40:25 -06:00
James Betker
0824708dc7 iq checkin 2022-07-18 18:40:14 -06:00
James Betker
df27b98730 ddp doesnt like dropout on checkpointed values 2022-07-18 17:17:04 -06:00
James Betker
8d7692c1e0 uh 2022-07-18 17:15:27 -06:00
James Betker
c959e530cb good ole ddp.. 2022-07-18 17:13:45 -06:00
James Betker
cf57c352c8 Another fix 2022-07-18 17:09:13 -06:00
James Betker
83a4ef4149 default to use input for conditioning & add preprocessed input to GDI 2022-07-18 17:01:19 -06:00
James Betker
1b4d9567f3 tfd13 for multi-resolution superscaling 2022-07-18 16:36:22 -06:00
James Betker
1b648abd7c iq2 2022-07-18 10:12:23 -06:00
James Betker
7a10c3fed8 commit my own version of vq, with a fix for cosine similarity and support for masking 2022-07-18 10:12:17 -06:00
James Betker
4ce5c31705 music_joiner update for new big cosine model 2022-07-17 18:24:56 -06:00
James Betker
cb24bef406 ddim_with_guidance 2022-07-17 18:24:43 -06:00
James Betker
20ef9cc6b4 iq checkin 
yeah - I'm at it again...
 2022-07-17 18:24:33 -06:00
James Betker
a073fbfcb8 force tf32 to be enabled (torch 1.12 disables it) 2022-07-16 13:59:07 -06:00
James Betker
438dcaccc5 misc 2022-07-16 13:58:52 -06:00
James Betker
e13b1adfdb :< 2022-07-14 21:52:23 -06:00
James Betker
51291ab070 Some additional context regularization in tfd 2022-07-14 21:49:47 -06:00
James Betker
7170ccdfa9 Merge branch 'baktfd' 2022-07-14 21:38:45 -06:00
James Betker
3b12d348fc mdf: provide conditioning margin 2022-07-14 21:38:14 -06:00
James Betker
fa352e2744 also some good assert text 2022-07-14 21:26:22 -06:00
James Betker
4d53c66602 simplify span selecting logic in tfdpc 2022-07-14 21:25:03 -06:00