James Betker
a3da7f186e
add tfd audio diffusion
2022-06-12 13:59:22 -06:00
James Betker
38a00f29c0
now theres deprecation warnings, fml
2022-06-10 15:41:39 -06:00
James Betker
561a6b8ff7
damn this sucks
2022-06-10 15:38:59 -06:00
James Betker
0316063e2d
.
2022-06-10 15:37:02 -06:00
James Betker
ee2827dee9
Debug warmup state
2022-06-10 15:23:31 -06:00
James Betker
d98b895307
loss aware fix and report gumbel temperature
2022-06-09 21:56:47 -06:00
James Betker
07bdd865dc
some checks
2022-06-09 21:46:32 -06:00
James Betker
34005367fd
setup for partial channel diffusion
2022-06-09 21:41:20 -06:00
James Betker
47b34f5cb9
mup work checkin
2022-06-09 21:15:09 -06:00
James Betker
e67e82be2d
misc
2022-06-09 21:14:48 -06:00
James Betker
5a54d7db11
unet with ar prior
2022-06-07 17:52:36 -06:00
James Betker
0a9d4d4afc
bunch of new stuff
2022-06-04 22:23:08 -06:00
James Betker
4819f15521
undo quantile sampler in music_diffusion_fid
2022-06-04 10:15:31 -06:00
James Betker
a9387179db
add channel loss balancing
2022-06-03 15:19:23 -06:00
James Betker
b2a83efe50
a few fixes
2022-06-01 16:35:15 -06:00
James Betker
f7d237a50a
train quantizer with diffusion
2022-05-30 16:25:33 -06:00
James Betker
27f347cdd3
gotta project that shit!
2022-05-28 23:19:36 -06:00
James Betker
6b43915eb8
support projecting to vectors
2022-05-28 22:27:45 -06:00
James Betker
76aeba7843
fail gracefully from mfd
2022-05-27 20:24:16 -06:00
James Betker
f691f5faa1
f
2022-05-27 13:47:05 -06:00
James Betker
031769150d
clip adf test dataset
2022-05-27 13:42:52 -06:00
James Betker
31dec016e0
adf
2022-05-27 12:28:04 -06:00
James Betker
b4269af61b
fix circular deps
2022-05-27 11:44:27 -06:00
James Betker
34ee1d0bc3
mdf
2022-05-27 11:40:47 -06:00
James Betker
9852599b34
tfd5 - with clvp!
2022-05-27 09:49:10 -06:00
James Betker
3db862dd32
adf update
2022-05-27 09:25:53 -06:00
James Betker
48aab2babe
ressurect ctc code gen with some cool new ideas
2022-05-24 14:02:33 -06:00
James Betker
f4a97ca0a7
and this
2022-05-23 10:38:28 -06:00
James Betker
874de1775d
Update mdf spectral
2022-05-23 10:37:15 -06:00
James Betker
36dd4eb61f
no grads for mel injectors
2022-05-23 10:34:53 -06:00
James Betker
4093e38717
revert flat diffusion back...
2022-05-22 23:10:58 -06:00
James Betker
07f7be24ce
:/
2022-05-22 20:00:55 -06:00
James Betker
c0bc466aad
mdf for shortened mel test
2022-05-22 19:29:20 -06:00
James Betker
2798bfab8c
Revert "update mdf for legacy try"
...
This reverts commit 693b19ea3b
2022-05-22 19:22:44 -06:00
James Betker
693b19ea3b
update mdf for legacy try
2022-05-22 16:37:02 -06:00
James Betker
57d6f6d366
Big rework of flat_diffusion
...
Back to the drawing board, boys. Time to waste some resources catching bugs....
2022-05-22 08:09:33 -06:00
James Betker
db38672dae
precompute diffusion embeddings for from_codes
2022-05-22 06:45:57 -06:00
James Betker
ea21a8b107
Update music_diffusion_fid to support waveform diffusion from codes
2022-05-22 05:23:54 -06:00
James Betker
e0bf3a0ddc
Save myself some time in the future
2022-05-20 17:18:35 -06:00
James Betker
e9fb2ead9a
m2v stuff
2022-05-20 11:01:17 -06:00
James Betker
c9c16e3b01
misc updates
2022-05-19 13:39:32 -06:00
James Betker
7213ad2b89
Do grad reduction
2022-05-17 17:59:40 -06:00
James Betker
8202b9f39c
some stuff
2022-05-15 21:50:54 -06:00
James Betker
ab5acead0e
add exp loss for diffusion models
2022-05-15 21:50:38 -06:00
James Betker
9118f58849
uncomment music projector..
2022-05-09 09:19:26 -06:00
James Betker
74dd095326
a
2022-05-08 18:54:09 -06:00
James Betker
1177c35dec
music fid updates
2022-05-08 18:49:39 -06:00
James Betker
6c8032b4be
more work
2022-05-06 21:56:49 -06:00
James Betker
d8925ccde5
few things with gap filling
2022-05-06 14:33:44 -06:00
James Betker
b83b53cf84
norm mel
2022-05-06 00:49:54 -06:00
James Betker
47662b9ec5
some random crap
2022-05-04 20:29:23 -06:00
James Betker
6655f7845a
add pixel shuffling for 1d cases
2022-05-04 08:03:09 -06:00
James Betker
c42c53e75a
Add a trainable network for converting a normal distribution into a latent space
2022-05-02 09:47:30 -06:00
James Betker
f4254609c1
MDF
...
around and around in circles........
2022-05-01 23:04:56 -06:00
James Betker
e208d9fb80
gate augmentations with a flag
2022-04-28 10:09:22 -06:00
James Betker
3f67cb2023
music diffusion fid adjustments
2022-04-28 10:08:55 -06:00
James Betker
f02b01bd9d
reverse univnet classifier
2022-04-20 21:37:55 -06:00
James Betker
b1c2c48720
music diffusion fid
2022-04-20 00:28:03 -06:00
James Betker
3cad1b8114
more fixes
2022-04-11 15:18:44 -06:00
James Betker
6dea7da7a8
another fix
2022-04-11 12:29:43 -06:00
James Betker
f2c172291f
fix audio_diffusion_fid for autoregressive latent inputs
2022-04-11 12:08:15 -06:00
James Betker
8ea5c307fb
Fixes for training the diffusion model on autoregressive inputs
2022-04-11 11:02:44 -06:00
James Betker
048f6f729a
remove lightweight_gan
2022-04-07 23:12:08 -07:00
James Betker
6fc4f49e86
some dumb stuff
2022-04-07 11:32:34 -06:00
James Betker
035bcd9f6c
fwd fix
2022-04-01 16:03:07 -06:00
James Betker
9b90472e15
feed direct inputs into gd
2022-03-26 08:36:19 -06:00
James Betker
2a29a71c37
attempt to force meaningful codes by adding a surrogate loss
2022-03-26 08:31:40 -06:00
James Betker
45804177b8
more stuff
2022-03-25 00:03:18 -06:00
James Betker
d4218d8443
mods
2022-03-24 23:31:20 -06:00
James Betker
9c79fec734
update adf
2022-03-24 21:20:29 -06:00
James Betker
07731d5491
Fix ET
2022-03-24 21:20:22 -06:00
James Betker
b0d2827fad
flat0
2022-03-24 11:30:40 -06:00
James Betker
be5f052255
misc
2022-03-22 11:40:56 -06:00
James Betker
963f0e9cee
fix unscaler
2022-03-22 11:40:02 -06:00
James Betker
1ad18d29a8
Flat fixes
2022-03-21 14:43:52 -06:00
James Betker
c5000420f6
more arbitrary fixes
2022-03-17 17:45:44 -06:00
James Betker
c14fc003ed
flat diffusion
2022-03-17 17:45:27 -06:00
James Betker
428911cd4d
flat diffusion network
2022-03-17 10:53:56 -06:00
James Betker
bf08519d71
fixes
2022-03-17 10:53:39 -06:00
James Betker
95ea0a592f
More cleaning
2022-03-16 12:05:56 -06:00
James Betker
d186414566
More spring cleaning
2022-03-16 12:04:00 -06:00
James Betker
8b376e63d9
More improvements
2022-03-16 10:16:34 -06:00
James Betker
54202aa099
fix mel normalization
2022-03-16 09:26:55 -06:00
James Betker
8437bb0c53
fixes
2022-03-15 23:52:48 -06:00
James Betker
3f244f6a68
add mel_norm to std injector
2022-03-15 22:16:59 -06:00
James Betker
f563a8dd41
fixes
2022-03-15 21:43:00 -06:00
James Betker
1e3a8554a1
updates to audio_diffusion_fid
2022-03-15 11:35:09 -06:00
James Betker
7929fd89de
Refactor audio-style models into the audio folder
2022-03-15 11:06:25 -06:00
James Betker
e045fb0ad7
fix clip grad norm with scaler
2022-03-13 16:28:23 -06:00
James Betker
08599b4c75
fix random_audio_crop injector
2022-03-12 20:42:29 -07:00
James Betker
c4e4cf91a0
add support for the original vocoder to audio_diffusion_fid; also add a new "intelligibility" metric
2022-03-08 15:53:27 -07:00
James Betker
3e5da71b16
add grad scaler scale to metrics
2022-03-08 15:52:42 -07:00
James Betker
d1dc8dbb35
Support tts9
2022-03-05 20:14:36 -07:00
James Betker
93a3302819
Push training_state data to CPU memory before saving it
...
For whatever reason, keeping this on GPU memory just doesn't work.
When you load it, it consumes a large amount of GPU memory and that
utilization doesn't go away. Saving to CPU should fix this.
2022-03-04 17:57:33 -07:00
James Betker
6000580e2e
df
2022-03-04 13:47:00 -07:00
James Betker
382681a35d
Load diffusion_fid DVAE into the correct cuda device
2022-03-04 13:42:14 -07:00
James Betker
e1052a5e32
Move log consensus to train for efficiency
2022-03-04 13:41:32 -07:00
James Betker
ce6dfdf255
Distributed "fixes"
2022-03-04 12:46:41 -07:00
James Betker
3ff878ae85
Accumulate loss & grad_norm metrics from all entities within a distributed graph
2022-03-04 12:01:16 -07:00
James Betker
f87e10ffef
Make deterministic sampler work with distributed training & microbatches
2022-03-04 11:50:50 -07:00
