James Betker
036684893e
Add LARS optimizer & support for BYOL idiosyncrasies
...
- Added LARS and SGD optimizer variants that support turning off certain
features for BN and bias layers
- Added a variant of pytorch's resnet model that supports gradient checkpointing.
- Modify the trainer infrastructure to support above
- Fix bug with BYOL (should have been nonfunctional)
2020-12-23 20:33:43 -07:00
James Betker
1bbcb96ee8
Implement a few changes to support training BYOL networks
2020-12-23 10:50:23 -07:00
James Betker
2437b33e74
Fix srflow_latent_space_playground bug
2020-12-22 15:42:38 -07:00
James Betker
e7aeb17404
ImageFolder dataset: allow intermediary downscale before corrupt
...
For massive upscales (ex: 8x), corruption does almost nothing when applied
at the HQ level. This patch adds support to perform corruption at a specified
intermediary scale. The dataset downscales to this level, performs the corruption,
then downscales the rest of the way to get the LQ image.
2020-12-22 15:42:21 -07:00
James Betker
b7158af23f
More documentation updates
2020-12-20 16:52:57 -07:00
James Betker
bbc677dc7b
Add ESRGAN docs
2020-12-20 11:50:31 -07:00
James Betker
7938f9f50b
Fix bug with single_image_dataset which prevented working on multiple directories from working
2020-12-19 15:13:46 -07:00
James Betker
ae666dc520
Fix bugs with srflow after refactor
2020-12-19 10:28:23 -07:00
James Betker
4328c2f713
Change default ReLU slope to .2 BREAKS COMPATIBILITY
...
This conforms my ConvGnLelu implementation with the generally accepted negative_slope=.2. I have no idea where I got .1. This will break backwards compatibility with some older models but will likely improve their performance when freshly trained. I did some auditing to find what these models might be, and I am not actively using any of them, so probably OK.
2020-12-19 08:28:03 -07:00
James Betker
9377d34ac3
glean mods
2020-12-19 08:26:07 -07:00
James Betker
f35c034fa5
Add trainer readme
2020-12-18 16:52:16 -07:00
James Betker
e82f4552db
Update other docs with dumb config options
2020-12-18 16:21:28 -07:00
James Betker
81256d553a
GLEAN docs
2020-12-18 16:21:01 -07:00
James Betker
92f9a129f7
GLEAN!
2020-12-18 16:04:19 -07:00
James Betker
c717765bcb
Notes for lucidrains converter.
2020-12-18 09:55:38 -07:00
James Betker
b4720ea377
Move stylegan to new location
2020-12-18 09:52:36 -07:00
James Betker
1708136b55
Commit my attempt at "conforming" the lucidrains stylegan implementation to the reference spec. Not working. will probably be abandoned.
2020-12-18 09:51:48 -07:00
James Betker
209332292a
Rosinality stylegan fix
2020-12-18 09:50:41 -07:00
James Betker
d875ca8342
More refactor changes
2020-12-18 09:24:31 -07:00
James Betker
5640e4efe4
More refactoring
2020-12-18 09:18:34 -07:00
James Betker
b905b108da
Large cleanup
...
Removed a lot of old code that I won't be touching again. Refactored some
code elements into more logical places.
2020-12-18 09:10:44 -07:00
James Betker
2f0a52b7db
misc changes
2020-12-18 08:53:45 -07:00
James Betker
a8179ff53c
Image label work
2020-12-18 08:53:18 -07:00
James Betker
da095199fb
gitignore update
2020-12-18 08:52:13 -07:00
James Betker
3074f41877
Get rosinality model converter to work
...
Mostly, just needed to remove the custom cuda ops, not so bueno on Windows.
2020-12-17 16:03:39 -07:00
James Betker
e838c6e75b
Rosinality stylegan2 port
2020-12-17 14:18:46 -07:00
James Betker
12cf052889
Add an image patch labeling UI
2020-12-17 10:16:21 -07:00
James Betker
daee1b5572
Update srflow recipe
2020-12-16 10:36:27 -07:00
James Betker
49327b99fe
SRFlow outputs RRDB output
2020-12-16 10:28:02 -07:00
James Betker
c25b49bb12
Clean up of SRFlowNet_arch
2020-12-16 10:27:38 -07:00
James Betker
42ac8e3eeb
Remove unnecessary comment from SRFlowNet
2020-12-16 09:43:07 -07:00
James Betker
fb2cfc795b
Update requirements, add image_patch_classifier tool
2020-12-16 09:42:50 -07:00
James Betker
09de3052ac
Add softmax to spinenet classification head
2020-12-16 09:42:15 -07:00
James Betker
4310e66848
Fix bug in 'corrupt_before_downsize=true'
2020-12-16 09:41:59 -07:00
James Betker
8661207d57
Merge branch 'gan_lab' of https://github.com/neonbjb/DL-Art-School into gan_lab
2020-12-15 17:16:48 -07:00
James Betker
fc376d34b2
Spinenet with logits head
2020-12-15 17:16:19 -07:00
James Betker
8e0e883050
Mods to support labeled datasets & random augs for those datasets
2020-12-15 17:15:56 -07:00
James Betker
e5a3e6b9b5
srflow latent space misc
2020-12-14 23:59:49 -07:00
James Betker
1e14635d88
Add exclusions to extract_subimages_with_ref
2020-12-14 23:59:41 -07:00
James Betker
0a19e53df0
BYOL mods
2020-12-14 23:59:11 -07:00
James Betker
ef7eabf457
Allow RRDB to upscale 8x
2020-12-14 23:58:52 -07:00
James Betker
087e9280ed
Add labeling feature to image_folder_dataset
2020-12-14 23:58:37 -07:00
James Betker
ec0ee25f4b
Structural latents checkpoint
2020-12-11 12:01:09 -07:00
James Betker
26ceca68c0
BYOL with structure!
2020-12-10 15:07:35 -07:00
James Betker
9c5e272a22
Script to extract models from a wrapped BYOL model
2020-12-10 09:57:52 -07:00
James Betker
a5630d282f
Get rid of 2nd trainer
2020-12-10 09:57:38 -07:00
James Betker
8e4b9f42fd
New BYOL dataset which uses a form of RandomCrop that lends itself to
...
structural guidance to the latents.
2020-12-10 09:57:18 -07:00
James Betker
c203cee31e
Allow swapping to torch DDP as needed in code
2020-12-09 15:03:59 -07:00
James Betker
66cbae8731
Add random_dataset for testing
2020-12-09 14:55:05 -07:00
James Betker
97ff25a086
BYOL!
...
Man, is there anything ExtensibleTrainer can't train? :)
2020-12-08 13:07:53 -07:00
