Commit Graph

2154 Commits

Author SHA1 Message Date
mrq
a4afad8837 PIP-ified (credit to https://git.ecker.tech/eschmidbauer) 2023-03-21 15:39:28 +00:00
mrq
fe24641763 PIP-ified part 1 2023-03-21 15:38:42 +00:00
mrq
efd038c076 forgot the other things that were in tortoise implementation but not here 2023-03-17 20:24:17 +00:00
mrq
64a41fde24 added japanese preprocessor for tokenizer 2023-03-17 20:03:57 +00:00
mrq
7b5e0592f8 migrated bitsandbytes out, since AIVC technically uses this more 2023-03-16 20:42:32 +00:00
mrq
0db8ebc543 deduce if preprocessing text by checking the JSON itself instead 2023-03-16 14:41:21 +00:00
mrq
730f56aa87 some day I'll get a commit right on the first try 2023-03-16 04:37:49 +00:00
mrq
730a04708d added flag to disable preprocessing (because some IPAs will turn into ASCII, implicitly enable for using the specific ipa.json tokenizer vocab) 2023-03-16 04:24:32 +00:00
mrq
bea6174a19 fix for torch2.0.0 suddenly being imposed on us 2023-03-15 19:24:35 +00:00
mrq
b253da6e35 save when training completes 2023-03-15 02:47:12 +00:00
mrq
3fdf2a63aa fixes 2023-03-11 01:18:25 +00:00
mrq
b5c6acec9e gutted bloat loggers, now all my useful metrics update per step 2023-03-10 22:34:37 +00:00
mrq
bf94744514 I am going to scream 2023-03-09 22:47:46 +00:00
mrq
84c8196da5 Shamelessly nabbed from ae80992817 (if this is makes a big enough difference in training i'm going to cum) 2023-03-09 03:39:23 +00:00
mrq
0ee0f46596 . 2023-03-09 00:29:25 +00:00
mrq
6eb7ebf847 silence printing the model because it's just useless noise 2023-03-04 16:38:24 +00:00
mrq
71cc43e65c added a flag (thanks gannybal) 2023-02-26 14:56:26 +00:00
mrq
0f04206aa2 added ability to toggle some settings with envvars for later testing without needing to manually edit this file (and some other things like disabling it when a user requests it in the future) 2023-02-24 23:08:56 +00:00
mrq
1433b7c0ea working Embedding override 2023-02-23 07:28:27 +00:00
mrq
94aefa3e4c silence 2023-02-23 07:25:09 +00:00
mrq
fd66c4104b ugh 2023-02-23 07:18:07 +00:00
mrq
7bcedca771 I guess I can't easily toggle it outside of here, but it works 2023-02-23 07:02:06 +00:00
mrq
0ef8ab6872 shut up 2023-02-23 06:12:27 +00:00
mrq
58600274ac Disabling bitsandbytes optimization as default for now, in the off chance that it actually produces garbage (which shouldn't happen, there's no chance, if training at float16 from a model at float16 works fine, then this has to work) 2023-02-23 03:22:59 +00:00
mrq
918473807f Merge pull request 'bitsandbytes' (#2) from bitsandbytes into master
Reviewed-on: mrq/DL-Art-School#2
2023-02-23 03:16:25 +00:00
mrq
6676c89c0e I sucked off the hyptothetical wizard again, just using BNB's ADAM optimizer nets HUGE savings, but I don't know the output costs, will need to test 2023-02-23 02:42:17 +00:00
mrq
01c0941a40 binaries 2023-02-22 23:09:27 +00:00
mrq
4427d7fb84 initial conversion (errors out) 2023-02-22 23:07:05 +00:00
mrq
6c284ef8ec oops 2023-02-18 03:27:04 +00:00
mrq
8db762fa17 thought I copied this over 2023-02-18 03:15:44 +00:00
mrq
73d9c3bd46 set output folder to be sane with the cwd as a reference point 2023-02-18 02:01:09 +00:00
mrq
5ecf7da881 Fix later 2023-02-17 20:49:29 +00:00
mrq
e3e8801e5f Fix I thought wasn't needed since it literally worked without it earlier 2023-02-17 20:41:20 +00:00
mrq
535549c3f3 add some snark about the kludge I had to fix, and the kludge I used to fix it 2023-02-17 19:20:19 +00:00
mrq
a09cf98c7f more cleanup, pip-ifying won't work, got an alternative 2023-02-17 15:47:55 +00:00
mrq
6afa2c299e break if your dataset size is smaller than your batch size 2023-02-17 04:08:27 +00:00
mrq
94d0f16608 Necessary fixes to get it to work 2023-02-17 02:03:00 +00:00
mrq
49e23b226b pip-ify 2023-02-17 00:33:50 +00:00
James Betker
f31a333c4f more sampling fixes 2022-10-10 20:11:28 -06:00
James Betker
5d172fbf7e Fix eval 2022-10-10 14:22:36 -06:00
James Betker
9502e0755e ugh 2022-10-10 12:15:51 -06:00
James Betker
fce2c8f5db and listify them 2022-10-10 12:13:49 -06:00
James Betker
3cf78e3c44 train mel head even when not 2022-10-10 12:10:56 -06:00
James Betker
cc74a43675 Checkin 2022-10-10 11:30:20 -06:00
James Betker
3cb14123bc glc fix 2022-07-29 11:24:36 -06:00
James Betker
4ddd01a7fb support generating cheaters from the new cheater network 2022-07-29 09:19:20 -06:00
James Betker
27a9b1b750 rename perplexity->log perplexity 2022-07-28 09:48:40 -06:00
James Betker
1d68624828 fix some imports.. 2022-07-28 02:35:32 -06:00
James Betker
cfe907f13f i like this better 2022-07-28 02:33:23 -06:00
James Betker
d44ed5d12d probably too harsh on ninfs 2022-07-28 01:33:54 -06:00