mrq/DL-Art-School
1
3
Fork 5
Code Issues Pull Requests Packages Projects Releases Wiki Activity
920 Commits 1 Branch 0 Tags 5.6 MiB
04961b91cf
Commit Graph

894 Commits

Author SHA1 Message Date
James Betker
04961b91cf Add random-crop injector 2021-01-07 12:14:55 -07:00
James Betker
61a86a3c1e VQVAE 2021-01-07 10:20:15 -07:00
James Betker
01a589e712 Adjustments to pixpro & resnet-unet 
I'm not really satisfied with what I got out of these networks on round 1.
Lets try again..
 2021-01-06 15:00:46 -07:00
James Betker
9680294430 Move byol scripts around 2021-01-06 14:52:17 -07:00
James Betker
2f2f87bbea Styled SR fixes 2021-01-05 20:14:39 -07:00
James Betker
9fed90393f Add lucidrains pixpro trainer 2021-01-05 20:14:22 -07:00
James Betker
39a94c74b5 Allow BYOL resnet playground to produce a latent dict 2021-01-04 20:11:29 -07:00
James Betker
ade2732c82 Transfer learning for styleSR 
This is a concept from "Lifelong Learning GAN", although I'm skeptical of it's novelty -
basically you scale and shift the weights for the generator and discriminator of a pretrained
GAN to "shift" into new modalities, e.g. faces->birds or whatever. There are some interesting
applications of this that I would like to try out.
 2021-01-04 20:10:48 -07:00
James Betker
2c65b6b28e More mods to support styledsr 2021-01-04 11:32:28 -07:00
James Betker
2225fe6ac2 Undo lucidrains changes for new discriminator 
This "new" code will live in the styledsr directory from now on.
 2021-01-04 10:57:09 -07:00
James Betker
40ec71da81 Move styled_sr into its own folder 2021-01-04 10:54:34 -07:00
James Betker
5916f5f7d4 Misc fixes 2021-01-04 10:53:53 -07:00
James Betker
4d8064c32c Modifications to allow partially trained stylegan discriminators to be used 2021-01-03 16:37:18 -07:00
James Betker
5e7ade0114 ImageFolderDataset - corrupt lq images alongside each other 2021-01-03 16:36:38 -07:00
James Betker
ce6524184c Do the last commit but in a better way 2021-01-02 22:24:12 -07:00
James Betker
edf9c38198 Make ExtensibleTrainer set the starting step for the LR scheduler 2021-01-02 22:22:34 -07:00
James Betker
bdbab65082 Allow optimizers to train separate param groups, add higher dimensional VGG discriminator 
Did this to support training 512x512px networks off of a pretrained 256x256 network.
 2021-01-02 15:10:06 -07:00
James Betker
193cdc6636 Move discriminators to the create_model paradigm 
Also cleans up a lot of old discriminator models that I have no intention
of using again.
 2021-01-01 15:56:09 -07:00
James Betker
7976a5825d srfid is incorrectly labeled 2021-01-01 13:00:59 -07:00
James Betker
f39179e85a styled_sr: fix bug when using initial_stride 2021-01-01 12:13:21 -07:00
James Betker
913fc3b75e Need init to pick up styled_sr 2021-01-01 12:10:32 -07:00
James Betker
aae65e6ed8 Mods to byol_resnet_playground for large batches 2021-01-01 11:59:54 -07:00
James Betker
e992e18767 Add initial_stride term to style_sr 
Also fix fid and a networks.py issue.
 2021-01-01 11:59:36 -07:00
James Betker
9864fe4c04 Fix for train.py 2021-01-01 11:59:00 -07:00
James Betker
e214e6ce33 Styled SR model 2020-12-31 20:54:18 -07:00
James Betker
0eb1f4dd67 Revert "Get rid of CUDA_VISIBLE_DEVICES" 
It is actually necessary for training in distributed mode. Only
do it then.
 2020-12-31 10:31:40 -07:00
James Betker
8de5a02a48 byol_resnet_playground 
Similar to the spinenet playground, but tinkers with resnet instead
 2020-12-31 10:15:04 -07:00
James Betker
8f18b2709e Get rid of CUDA_VISIBLE_DEVICES 
It is not clear to me what the purpose of this is, but it has recently
started causing failures.
 2020-12-31 10:13:58 -07:00
James Betker
1de1fa30ac Disable refs and centers altogether in single_image_dataset 
I suspect that this might be a cause of failures on parallel datasets.
Plus it is unnecessary computation.
 2020-12-31 10:13:24 -07:00
James Betker
8f0984cacf Add sr_fid evaluator 2020-12-30 20:18:58 -07:00
James Betker
b1fb82476b Add gp debug (fix) 2020-12-30 15:26:54 -07:00
James Betker
9c53314ea2 Add gradient penalty visual debug 2020-12-30 09:51:59 -07:00
James Betker
63cf3d3126 Injector auto-registration 
I love it!
 2020-12-29 20:58:02 -07:00
James Betker
a777c1e4f9 Misc script fixes 2020-12-29 20:25:09 -07:00
James Betker
9dc3c8f0ff Script updates 2020-12-29 20:24:41 -07:00
James Betker
ba543d1152 Glean mods 
- Fixes fixed upscale factor issues
- Refines a few ops to decrease computation & parameterization
 2020-12-27 12:25:06 -07:00
James Betker
5e2e605a50 Merge remote-tracking branch 'origin/gan_lab' into gan_lab 2020-12-26 13:51:19 -07:00
James Betker
f9be049adb GLEAN mod to support custom initial strides 2020-12-26 13:51:14 -07:00
James Betker
2706a84f15 Merge remote-tracking branch 'origin/gan_lab' into gan_lab 2020-12-26 13:50:34 -07:00
James Betker
90e2362c00 Fix bug with full_image_dataset 2020-12-26 13:50:27 -07:00
James Betker
3fd627fc62 Mods to support image classification & filtering 2020-12-26 13:49:27 -07:00
James Betker
10fdfa1563 Migrate generators to dynamic model registration 2020-12-24 23:02:10 -07:00
James Betker
29db7c7a02 Further mods to BYOL 2020-12-24 09:28:41 -07:00
James Betker
036684893e Add LARS optimizer & support for BYOL idiosyncrasies 
- Added LARS and SGD optimizer variants that support turning off certain
  features for BN and bias layers
- Added a variant of pytorch's resnet model that supports gradient checkpointing.
- Modify the trainer infrastructure to support above
- Fix bug with BYOL (should have been nonfunctional)
 2020-12-23 20:33:43 -07:00
James Betker
1bbcb96ee8 Implement a few changes to support training BYOL networks 2020-12-23 10:50:23 -07:00
James Betker
2437b33e74 Fix srflow_latent_space_playground bug 2020-12-22 15:42:38 -07:00
James Betker
e7aeb17404 ImageFolder dataset: allow intermediary downscale before corrupt 
For massive upscales (ex: 8x), corruption does almost nothing when applied
at the HQ level. This patch adds support to perform corruption at a specified
intermediary scale. The dataset downscales to this level, performs the corruption,
then downscales the rest of the way to get the LQ image.
 2020-12-22 15:42:21 -07:00
James Betker
7938f9f50b Fix bug with single_image_dataset which prevented working on multiple directories from working 2020-12-19 15:13:46 -07:00
James Betker
ae666dc520 Fix bugs with srflow after refactor 2020-12-19 10:28:23 -07:00
James Betker
4328c2f713 Change default ReLU slope to .2 BREAKS COMPATIBILITY 
This conforms my ConvGnLelu implementation with the generally accepted negative_slope=.2. I have no idea where I got .1. This will break backwards compatibility with some older models but will likely improve their performance when freshly trained. I did some auditing to find what these models might be, and I am not actively using any of them, so probably OK.
 2020-12-19 08:28:03 -07:00