mrq/DL-Art-School
1
3
Fork 5
Code Issues Pull Requests Packages Projects Releases Wiki Activity
234 Commits 1 Branch 0 Tags 5.6 MiB
8f92c0a088
Commit Graph

10 Commits

Author SHA1 Message Date
James Betker
8f92c0a088 Interpolate attention well before softmax 2020-07-06 09:18:30 -06:00
James Betker
10f7e49214 Add ConvBnSilu to replace ConvBnRelu 
Relu produced good performance gains over LeakyRelu, but
GAN performance degraded significantly. Try SiLU as an alternative
to see if it's the leaky-ness we are looking for or the smooth activation
curvature.
 2020-07-05 13:39:08 -06:00
James Betker
9934e5d082 Move SRG1 to identical to new 2020-07-05 08:49:34 -06:00
James Betker
416538f31c SRG1 conjoined except ConvBnRelu 2020-07-05 08:44:17 -06:00
James Betker
c58c2b09ca Back to remove all biases (looks like a ConvBnRelu made its way in..) 2020-07-04 22:41:02 -06:00
James Betker
86cda86e94 Re-add biases, also add new init 
A/B testing where we lost our GAN competitiveness.
 2020-07-04 22:24:42 -06:00
James Betker
b03741f30e Remove all biases from generator 
Continuing to investigate loss of GAN competitiveness, this is a big difference
between "old" SRG1 and "new".
 2020-07-04 22:19:55 -06:00
James Betker
726e946e79 Turn BN off in SRG1 
This wont work well but just testing if GAN performance comes back
 2020-07-04 14:51:27 -06:00
James Betker
0ee39d419b OrderedDict not needed 2020-07-04 14:09:27 -06:00
James Betker
9048105b72 Break out SRG1 as separate network 
Something strange is going on. These networks do not respond to
discriminator gradients properly anymore. SRG1 did, however so
reverting back to last known good state to figure out why.
 2020-07-04 13:28:50 -06:00