Continuing to investigate loss of GAN competitiveness, this is a big difference between "old" SRG1 and "new".
This wont work well but just testing if GAN performance comes back
Something strange is going on. These networks do not respond to discriminator gradients properly anymore. SRG1 did, however so reverting back to last known good state to figure out why.
