Commit Graph

326 Commits

Author SHA1 Message Date
James Betker
1596a98493 Get rid of skip layers from vgg disc 2020-05-12 10:08:12 -06:00
James Betker
ef48e819aa Allow resgen to have a conditional number of upsamples applied to it 2020-05-10 10:48:37 -06:00
James Betker
aa0305def9 Resnet discriminator overhaul
It's been a tough day figuring out WTH is going on with my discriminators.
It appears the raw FixUp discriminator can get into an "defective" state where
they stop trying to learn and just predict as close to "0" D_fake and D_real as
possible. In this state they provide no feedback to the generator and never
recover. Adding batch norm back in seems to fix this so it must be some sort
of parameterization error.. Should look into fixing this in the future.
2020-05-06 17:27:30 -06:00
James Betker
574e7e882b Fix up OOM issues when running a disjoint D update ratio and megabatches 2020-05-06 17:25:25 -06:00
James Betker
eee9d6d9ca Support skip connections in vgg arch discriminator. 2020-05-06 17:24:34 -06:00
James Betker
3cd85f8073 Implement ResGen arch
This is a simpler resnet-based generator which performs mutations
on an input interspersed with interpolate-upsampling. It is a two
part generator:
1) A component that "fixes" LQ images with a long string of resnet
    blocks. This component is intended to remove compression artifacts
    and other noise from a LQ image.
2) A component that can double the image size. The idea is that this
    component be trained so that it can work at most reasonable
    resolutions, such that it can be repeatedly applied to itself to
    perform multiple upsamples.

The motivation here is to simplify what is being done inside of RRDB.
I don't believe the complexity inside of that network is justified.
2020-05-05 11:59:46 -06:00
James Betker
3b4e54c4c5 Add support for passthrough disc/gen
Add RRDBNetXL, which performs processing at multiple image sizes.
Add DiscResnet_passthrough, which allows passthrough of image at different sizes for discrimination.
Adjust the rest of the repo to allow generators that return more than just a single image.
2020-05-04 14:01:43 -06:00
James Betker
9e1acfe396 Fixup upconv for the next attempt! 2020-05-01 19:56:14 -06:00
James Betker
7eaabce48d Full resnet corrupt, no BN
And it works! Thanks fixup..
2020-04-30 19:17:30 -06:00
James Betker
b6e036147a Add more batch norms to FlatProcessorNet_arch 2020-04-30 11:47:21 -06:00
James Betker
66e91a3d9e Revert "Enable skip-through connections from disc to gen"
This reverts commit b7857f35c3.
2020-04-30 11:45:07 -06:00
James Betker
b7857f35c3 Enable skip-through connections from disc to gen 2020-04-30 11:30:11 -06:00
James Betker
bf634fc9fa Make resnet w/ BN discriminator use leaky relus 2020-04-30 11:28:59 -06:00
James Betker
3781ea725c Add Resnet Discriminator with BN 2020-04-29 20:51:57 -06:00
James Betker
a5188bb7ca Remover fixup code from arch_util
Going into it's own arch.
2020-04-29 15:17:43 -06:00
James Betker
5b8a77f02c Discriminator part 1
New discriminator. Includes spectral norming.
2020-04-28 23:00:29 -06:00
James Betker
2c145c39b6 Misc changes 2020-04-28 11:50:16 -06:00
James Betker
8ab595e427 Add FlatProcessorNet
After doing some thinking and reading on the subject, it occurred to me that
I was treating the generator like a discriminator by focusing the network
complexity at the feature levels. It makes far more sense to process each conv
level equally for the generator, hence the FlatProcessorNet in this commit. This
network borrows some of the residual pass-through logic from RRDB which makes
the gradient path exceptionally short for pretty much all model parameters and
can be trained in O1 optimization mode without overflows again.
2020-04-28 11:49:21 -06:00
James Betker
b8f67418d4 Retool HighToLowResNet
The receptive field of the original was *really* low. This new one has a
receptive field of 36x36px patches. It also has some gradient issues
that need to be worked out
2020-04-26 01:13:42 -06:00
James Betker
02ff4a57fd Enable HighToLowResNet to do a 1:1 transform 2020-04-25 21:36:32 -06:00
James Betker
35bd1ecae4 Config changes for discriminator advantage run
Still going from high->low, discriminator discerns on low. Next up disc works on high.
2020-04-25 11:24:28 -06:00
James Betker
d95808f4ef Implement downsample GAN
This bad boy is for a workflow where you train a model on disjoint image sets to
downsample a "good" set of images like a "bad" set of images looks. You then
use that downsampler to generate a training set of paired images for supersampling.
2020-04-24 00:00:46 -06:00
James Betker
05aafef938 Support variant input sizes and scales 2020-04-22 00:39:55 -06:00
James Betker
cc834bd5a3 Support >128px image squares 2020-04-21 16:32:59 -06:00
XintaoWang
866a858e59 add deform_conv_cuda_kernel.cu 2019-08-27 17:49:12 +08:00
XintaoWang
037933ba66 mmsr 2019-08-23 21:42:47 +08:00