James Betker
2636d3b620
Fix assertion error
2020-07-06 09:23:53 -06:00
James Betker
8f92c0a088
Interpolate attention well before softmax
2020-07-06 09:18:30 -06:00
James Betker
10f7e49214
Add ConvBnSilu to replace ConvBnRelu
...
Relu produced good performance gains over LeakyRelu, but
GAN performance degraded significantly. Try SiLU as an alternative
to see if it's the leaky-ness we are looking for or the smooth activation
curvature.
2020-07-05 13:39:08 -06:00
James Betker
9934e5d082
Move SRG1 to identical to new
2020-07-05 08:49:34 -06:00
James Betker
416538f31c
SRG1 conjoined except ConvBnRelu
2020-07-05 08:44:17 -06:00
James Betker
c58c2b09ca
Back to remove all biases (looks like a ConvBnRelu made its way in..)
2020-07-04 22:41:02 -06:00
James Betker
86cda86e94
Re-add biases, also add new init
...
A/B testing where we lost our GAN competitiveness.
2020-07-04 22:24:42 -06:00
James Betker
b03741f30e
Remove all biases from generator
...
Continuing to investigate loss of GAN competitiveness, this is a big difference
between "old" SRG1 and "new".
2020-07-04 22:19:55 -06:00
James Betker
726e946e79
Turn BN off in SRG1
...
This wont work well but just testing if GAN performance comes back
2020-07-04 14:51:27 -06:00
James Betker
0ee39d419b
OrderedDict not needed
2020-07-04 14:09:27 -06:00
James Betker
9048105b72
Break out SRG1 as separate network
...
Something strange is going on. These networks do not respond to
discriminator gradients properly anymore. SRG1 did, however so
reverting back to last known good state to figure out why.
2020-07-04 13:28:50 -06:00
