James Betker
aac92b01b3
fix
2022-06-05 01:27:28 -06:00
James Betker
38d8b17d18
tfd8 gets real verbose grad norm metrics
2022-06-04 23:09:54 -06:00
James Betker
0a9d4d4afc
bunch of new stuff
2022-06-04 22:23:08 -06:00
James Betker
8f8b189025
Support legacy vqvae quantizer in music_quantizer
2022-06-04 10:16:24 -06:00
James Betker
40ba802104
padding
2022-06-03 12:09:59 -06:00
James Betker
581bc7ac5c
udmc update
2022-06-03 12:02:22 -06:00
James Betker
9d8c2bddb1
classical unet for music
2022-06-03 11:03:14 -06:00
James Betker
2f4d990ad1
tfd7
2022-06-02 09:27:40 -06:00
James Betker
b2a83efe50
a few fixes
2022-06-01 16:35:15 -06:00
James Betker
712e0e82f7
fix bug
2022-06-01 14:21:44 -06:00
James Betker
de54be5570
propagate diversity loss
2022-06-01 14:18:50 -06:00
James Betker
4c6ef42b38
freeze quantizer until step
2022-06-01 08:06:05 -06:00
James Betker
64b6ae2f4a
fix
2022-06-01 01:01:32 -06:00
James Betker
1ac02acdc3
tfd7
2022-06-01 00:50:40 -06:00
James Betker
e8cb93a4e9
fix size issues
2022-05-31 21:23:26 -06:00
James Betker
8a1b8e3e62
add checkpointing
2022-05-31 21:09:05 -06:00
James Betker
c0db85bf4f
music quantizer
2022-05-31 21:06:54 -06:00
James Betker
29b55d42a5
one more
2022-05-30 16:33:49 -06:00
James Betker
71cf654957
fix unused parameters
2022-05-30 16:31:40 -06:00
James Betker
f7d237a50a
train quantizer with diffusion
2022-05-30 16:25:33 -06:00
James Betker
136021bf8d
tfd6
2022-05-30 09:09:42 -06:00
James Betker
eab1162d2b
hmm..
2022-05-29 22:32:25 -06:00
James Betker
2e72fddaeb
td_tts_2
2022-05-29 22:22:14 -06:00
James Betker
536c8558ae
fix
2022-05-28 22:32:38 -06:00
James Betker
da367da411
df5
2022-05-28 22:30:23 -06:00
James Betker
6b43915eb8
support projecting to vectors
2022-05-28 22:27:45 -06:00
James Betker
86694aef4e
tfd5
2022-05-28 22:27:04 -06:00
James Betker
b6b4f10e1b
...
2022-05-28 10:59:03 -06:00
James Betker
0d3b831cf9
big fatty
2022-05-28 10:55:43 -06:00
James Betker
490d39b967
some stuff
2022-05-27 11:40:31 -06:00
James Betker
5efeee6b97
fix type bug
2022-05-27 11:19:30 -06:00
James Betker
0659fe3d1e
tfd3 mods
2022-05-27 11:16:26 -06:00
James Betker
bed3df4888
propagate type
2022-05-27 11:12:03 -06:00
James Betker
c46da0285c
Move stuff around
2022-05-27 11:06:58 -06:00
James Betker
9852599b34
tfd5 - with clvp!
2022-05-27 09:49:10 -06:00
James Betker
3db862dd32
adf update
2022-05-27 09:25:53 -06:00
James Betker
8587a18717
fd fix
2022-05-26 20:19:09 -06:00
James Betker
dd13b883ac
td4
2022-05-26 14:56:03 -06:00
James Betker
1dbe0b6b2e
a
2022-05-26 10:13:27 -06:00
James Betker
aa653115f1
tfd3
2022-05-26 10:09:11 -06:00
James Betker
36c68692a6
forgot to add rotary embeddings
2022-05-26 09:25:42 -06:00
James Betker
8ce48f04ff
transformer diffusion 2
2022-05-26 09:08:35 -06:00
James Betker
56f19a23cd
fix nh
2022-05-25 12:31:56 -06:00
James Betker
52a20f3aa3
und10
2022-05-25 12:19:21 -06:00
James Betker
8b4b5ffa72
slight rework
2022-05-24 14:38:37 -06:00
James Betker
48aab2babe
ressurect ctc code gen with some cool new ideas
2022-05-24 14:02:33 -06:00
James Betker
65b441d74e
transformer diffusion
2022-05-24 14:02:05 -06:00
James Betker
1e1bbe1a27
whoops
2022-05-23 12:28:36 -06:00
James Betker
560b83e770
default to residual encoder
2022-05-23 12:24:00 -06:00
James Betker
f432bdf7ae
deeper resblock encoder
2022-05-23 11:46:40 -06:00
James Betker
dc471f5c6d
residual features
2022-05-23 09:58:30 -06:00
James Betker
1f521d6a1d
add reconstruction loss to m2v
2022-05-23 09:28:41 -06:00
James Betker
2270c89fdc
.
2022-05-23 08:47:15 -06:00
James Betker
40f844657b
tolong
2022-05-23 08:27:54 -06:00
James Betker
10f4a742bd
reintroduce attention masks
2022-05-23 08:16:04 -06:00
James Betker
68c0afcbcc
m2v frequency masking
2022-05-23 07:04:12 -06:00
James Betker
4093e38717
revert flat diffusion back...
2022-05-22 23:10:58 -06:00
James Betker
8f28404645
another fix
2022-05-22 21:32:43 -06:00
James Betker
41809a6330
Add 8x dim reductor
2022-05-22 20:23:16 -06:00
James Betker
1095248caf
Revert "retest"
...
This reverts commit ed7768c73b
2022-05-22 19:23:01 -06:00
James Betker
ed7768c73b
retest
2022-05-22 16:30:09 -06:00
James Betker
2dd0b9e6e9
mel_head should be optional
2022-05-22 12:25:45 -06:00
James Betker
0c60f22197
fix unused parameters
2022-05-22 08:16:31 -06:00
James Betker
57d6f6d366
Big rework of flat_diffusion
...
Back to the drawing board, boys. Time to waste some resources catching bugs....
2022-05-22 08:09:33 -06:00
James Betker
be937d202e
new attempt
2022-05-20 17:04:22 -06:00
James Betker
968660c248
another update
2022-05-20 11:25:00 -06:00
James Betker
28f950b7d3
fix
2022-05-20 11:18:52 -06:00
James Betker
b317c68ac9
fix
2022-05-20 11:12:53 -06:00
James Betker
3121bc4e43
flat diffusion
2022-05-20 11:01:48 -06:00
James Betker
e9fb2ead9a
m2v stuff
2022-05-20 11:01:17 -06:00
James Betker
c9c16e3b01
misc updates
2022-05-19 13:39:32 -06:00
James Betker
10378fc37f
make codebooks specifiable
2022-05-18 11:07:12 -06:00
James Betker
efc2657b48
fiddle with init
2022-05-18 10:56:01 -06:00
James Betker
208a703080
use gelu act
2022-05-18 09:34:01 -06:00
James Betker
b2b37453df
make the codebook bigger
2022-05-17 20:58:56 -06:00
James Betker
9a9c3cafba
Make feature encoder a bit more descriptive
2022-05-17 18:14:52 -06:00
James Betker
ee364f4eeb
just take the mean...
2022-05-17 18:09:23 -06:00
James Betker
6130391a85
fix div
2022-05-17 18:04:20 -06:00
James Betker
7213ad2b89
Do grad reduction
2022-05-17 17:59:40 -06:00
James Betker
7c82e18c6c
darn mpi
2022-05-17 17:16:09 -06:00
James Betker
88ec0512f7
Scale losses
2022-05-17 17:12:20 -06:00
James Betker
a6397ce84a
Fix incorrect projections
2022-05-17 16:53:52 -06:00
James Betker
c37fc3b4ed
m2v grad norm groups
2022-05-17 16:29:36 -06:00
James Betker
c1bdb4f9a1
degrade gumbel softmax over time
2022-05-17 16:23:04 -06:00
James Betker
3853f37257
stable layernorm
2022-05-17 16:07:03 -06:00
James Betker
519151d83f
m2v
2022-05-17 15:37:59 -06:00
James Betker
d1de94d75c
Stash mel2vec work (gonna throw it all away..)
2022-05-17 12:35:01 -06:00
James Betker
ee218ab9b7
uv3
2022-05-13 17:57:47 -06:00
James Betker
545453077e
uv3
2022-05-09 15:36:22 -06:00
James Betker
96a5cc66ee
uv3
2022-05-09 15:35:51 -06:00
James Betker
b42b4e18de
clean up unified voice
...
- remove unused code
- fix inference model to use the terms "prior" and "posterior" to properly define the modeling order (they were inverted before)
- default some settings I never intend to change in the future
2022-05-09 14:45:49 -06:00
James Betker
7812c23c7a
revert fill_gaps back to old masking behavior
2022-05-08 00:10:19 -06:00
James Betker
58ed27d7a8
new gap_filler
2022-05-07 12:44:23 -06:00
James Betker
6c8032b4be
more work
2022-05-06 21:56:49 -06:00
James Betker
79543e5488
Simpler form of the wavegen model
2022-05-06 16:37:04 -06:00
James Betker
d8925ccde5
few things with gap filling
2022-05-06 14:33:44 -06:00
James Betker
b13d983c24
and mel_head
2022-05-06 00:25:27 -06:00
James Betker
d5fb79564a
remove mel_pred
2022-05-06 00:24:05 -06:00
James Betker
e9bb692490
fixed aligned_latent
2022-05-06 00:20:21 -06:00
James Betker
1609101a42
musical gap filler
2022-05-05 16:47:08 -06:00
