6676c89c0e
I sucked off the hyptothetical wizard again, just using BNB's ADAM optimizer nets HUGE savings, but I don't know the output costs, will need to test
2023-02-23 02:42:17 +00:00
4427d7fb84
initial conversion (errors out)
2023-02-22 23:07:05 +00:00
James Betker
cc74a43675
Checkin
2022-10-10 11:30:20 -06:00
James Betker
4ddd01a7fb
support generating cheaters from the new cheater network
2022-07-29 09:19:20 -06:00
James Betker
f8108cfdb2
update environment and fix a bunch of deps
2022-07-24 23:43:25 -06:00
James Betker
45afefabed
fix booboo
2022-07-24 18:00:14 -06:00
James Betker
cc62ba9cba
few more tfd13 things
2022-07-24 17:39:33 -06:00
James Betker
76464ca063
some fixes to mdf to support new archs
2022-07-21 10:55:50 -06:00
James Betker
24a78bd7d1
update tfd14 too
2022-07-21 00:45:33 -06:00
James Betker
b92ff8de78
misc
2022-07-20 23:59:32 -06:00
James Betker
a1743d26aa
Revert "Try to squeeze a bit more performance out of this arch"
...
This reverts commit 767f963392
2022-07-20 23:57:56 -06:00
James Betker
767f963392
Try to squeeze a bit more performance out of this arch
2022-07-20 23:51:11 -06:00
James Betker
b9d0f7e6de
simplify parameterization a bit
2022-07-20 23:41:54 -06:00
James Betker
ee8ceed6da
rework tfd13 further
...
- use a gated activation layer for both attention & convs
- add a relativistic learned position bias. I believe this is similar to the T5 position encodings but it is simpler and learned
- get rid of prepending to the attention matrix - this doesn't really work that well. the model eventually learns to attend one of its heads to these blocks but why not just concat if it is doing that?
2022-07-20 23:28:29 -06:00
James Betker
40427de8e3
update tfd13 for inference
2022-07-20 21:51:25 -06:00
James Betker
dbebe18602
Fix ts=0 with new formulation
2022-07-20 12:12:33 -06:00
James Betker
82bd62019f
diffuse the cascaded prior for continuous sr model
2022-07-20 11:54:09 -06:00
James Betker
b0e3be0a17
transition to nearest interpolation mode for downsampling
2022-07-20 10:56:17 -06:00
James Betker
7b3fc79737
iq checkin
2022-07-20 10:19:32 -06:00
James Betker
15decfdb98
misc
2022-07-20 10:19:02 -06:00
James Betker
c14bf6dfb2
fix conditioning free
2022-07-19 18:04:49 -06:00
James Betker
fc0b291b21
do masking up proper
2022-07-19 16:32:17 -06:00
James Betker
c00398e955
scope attention in tfd13 as well
2022-07-19 14:59:43 -06:00
James Betker
b157b28c7b
tfd14
...
hopefully this helps address the positional dependencies of tfd12
2022-07-19 13:30:05 -06:00
James Betker
73d7211a4c
fix script
2022-07-19 11:17:43 -06:00
James Betker
6b1cfe8e66
ugh
2022-07-19 11:14:20 -06:00
James Betker
da9e47ca0e
new bounds for MEL normalization and multi-resolution SR in MDF
2022-07-19 11:11:46 -06:00
James Betker
eecb534e66
a few fixes to multiresolution sr
2022-07-19 11:11:15 -06:00
James Betker
eab7dc339d
iq checkin
...
who knows where I'm going with this.. I don't even know sometimes..
2022-07-19 09:13:27 -06:00
James Betker
0824708dc7
iq checkin
2022-07-18 18:40:14 -06:00
James Betker
df27b98730
ddp doesnt like dropout on checkpointed values
2022-07-18 17:17:04 -06:00
James Betker
c959e530cb
good ole ddp..
2022-07-18 17:13:45 -06:00
James Betker
cf57c352c8
Another fix
2022-07-18 17:09:13 -06:00
James Betker
83a4ef4149
default to use input for conditioning & add preprocessed input to GDI
2022-07-18 17:01:19 -06:00
James Betker
1b4d9567f3
tfd13 for multi-resolution superscaling
2022-07-18 16:36:22 -06:00
James Betker
1b648abd7c
iq2
2022-07-18 10:12:23 -06:00
James Betker
20ef9cc6b4
iq checkin
...
yeah - I'm at it again...
2022-07-17 18:24:33 -06:00
James Betker
e13b1adfdb
:<
2022-07-14 21:52:23 -06:00
James Betker
51291ab070
Some additional context regularization in tfd
2022-07-14 21:49:47 -06:00
James Betker
fa352e2744
also some good assert text
2022-07-14 21:26:22 -06:00
James Betker
4d53c66602
simplify span selecting logic in tfdpc
2022-07-14 21:25:03 -06:00
James Betker
4d5688be47
fix compatibility
2022-07-13 21:28:20 -06:00
James Betker
def70cd444
Merge remote-tracking branch 'origin/master'
2022-07-13 21:26:59 -06:00
James Betker
711c53c1f0
music script
2022-07-13 21:26:55 -06:00
James Betker
15831b2576
some stuff
2022-07-13 21:26:25 -06:00
James Betker
e23c322089
uhh2.0
2022-07-12 22:48:46 -06:00
James Betker
ebfe72d502
fix obo
2022-07-12 22:28:20 -06:00
James Betker
f46d6645da
tfdpcv5 updates
2022-07-12 21:48:18 -06:00
James Betker
7b4dcbf136
Support causal diffusion!
2022-07-08 12:30:05 -06:00
James Betker
28d5b6a80a
optionally disable checkpointing in x_transformers (and make it so with the cond_encoder in tfdpc_v5)
2022-07-06 16:55:57 -06:00
