 mrq
|
a4afad8837
|
PIP-ified (credit to https://git.ecker.tech/eschmidbauer)
|
2023-03-21 15:39:28 +07:00 |
|
mrq
|
fe24641763
|
PIP-ified part 1
|
2023-03-21 15:38:42 +07:00 |
|
mrq
|
efd038c076
|
forgot the other things that were in tortoise implementation but not here
|
2023-03-17 20:24:17 +07:00 |
|
mrq
|
64a41fde24
|
added japanese preprocessor for tokenizer
|
2023-03-17 20:03:57 +07:00 |
|
mrq
|
7b5e0592f8
|
migrated bitsandbytes out, since AIVC technically uses this more
|
2023-03-16 20:42:32 +07:00 |
|
mrq
|
0db8ebc543
|
deduce if preprocessing text by checking the JSON itself instead
|
2023-03-16 14:41:21 +07:00 |
|
mrq
|
730f56aa87
|
some day I'll get a commit right on the first try
|
2023-03-16 04:37:49 +07:00 |
|
mrq
|
730a04708d
|
added flag to disable preprocessing (because some IPAs will turn into ASCII, implicitly enable for using the specific ipa.json tokenizer vocab)
|
2023-03-16 04:24:32 +07:00 |
|
mrq
|
bea6174a19
|
fix for torch2.0.0 suddenly being imposed on us
|
2023-03-15 19:24:35 +07:00 |
|
mrq
|
b253da6e35
|
save when training completes
|
2023-03-15 02:47:12 +07:00 |
|
mrq
|
3fdf2a63aa
|
fixes
|
2023-03-11 01:18:25 +07:00 |
|
mrq
|
b5c6acec9e
|
gutted bloat loggers, now all my useful metrics update per step
|
2023-03-10 22:34:37 +07:00 |
|
mrq
|
bf94744514
|
I am going to scream
|
2023-03-09 22:47:46 +07:00 |
|
mrq
|
84c8196da5
|
Shamelessly nabbed from ae80992817 (if this is makes a big enough difference in training i'm going to cum)
|
2023-03-09 03:39:23 +07:00 |
|
mrq
|
0ee0f46596
|
.
|
2023-03-09 00:29:25 +07:00 |
|
mrq
|
6eb7ebf847
|
silence printing the model because it's just useless noise
|
2023-03-04 16:38:24 +07:00 |
|
mrq
|
71cc43e65c
|
added a flag (thanks gannybal)
|
2023-02-26 14:56:26 +07:00 |
|
mrq
|
0f04206aa2
|
added ability to toggle some settings with envvars for later testing without needing to manually edit this file (and some other things like disabling it when a user requests it in the future)
|
2023-02-24 23:08:56 +07:00 |
|
mrq
|
1433b7c0ea
|
working Embedding override
|
2023-02-23 07:28:27 +07:00 |
|
mrq
|
94aefa3e4c
|
silence
|
2023-02-23 07:25:09 +07:00 |
|
mrq
|
fd66c4104b
|
ugh
|
2023-02-23 07:18:07 +07:00 |
|
mrq
|
7bcedca771
|
I guess I can't easily toggle it outside of here, but it works
|
2023-02-23 07:02:06 +07:00 |
|
mrq
|
0ef8ab6872
|
shut up
|
2023-02-23 06:12:27 +07:00 |
|
mrq
|
58600274ac
|
Disabling bitsandbytes optimization as default for now, in the off chance that it actually produces garbage (which shouldn't happen, there's no chance, if training at float16 from a model at float16 works fine, then this has to work)
|
2023-02-23 03:22:59 +07:00 |
mrq
|
918473807f
|
Merge pull request 'bitsandbytes' (#2) from bitsandbytes into master
Reviewed-on: #2
|
2023-02-23 03:16:25 +07:00 |
|
mrq
|
6676c89c0e
|
I sucked off the hyptothetical wizard again, just using BNB's ADAM optimizer nets HUGE savings, but I don't know the output costs, will need to test
|
2023-02-23 02:42:17 +07:00 |
|
mrq
|
01c0941a40
|
binaries
|
2023-02-22 23:09:27 +07:00 |
|
mrq
|
4427d7fb84
|
initial conversion (errors out)
|
2023-02-22 23:07:05 +07:00 |
|
mrq
|
6c284ef8ec
|
oops
|
2023-02-18 03:27:04 +07:00 |
|
mrq
|
8db762fa17
|
thought I copied this over
|
2023-02-18 03:15:44 +07:00 |
|
mrq
|
73d9c3bd46
|
set output folder to be sane with the cwd as a reference point
|
2023-02-18 02:01:09 +07:00 |
|
mrq
|
5ecf7da881
|
Fix later
|
2023-02-17 20:49:29 +07:00 |
|
mrq
|
e3e8801e5f
|
Fix I thought wasn't needed since it literally worked without it earlier
|
2023-02-17 20:41:20 +07:00 |
|
mrq
|
535549c3f3
|
add some snark about the kludge I had to fix, and the kludge I used to fix it
|
2023-02-17 19:20:19 +07:00 |
|
mrq
|
a09cf98c7f
|
more cleanup, pip-ifying won't work, got an alternative
|
2023-02-17 15:47:55 +07:00 |
|
mrq
|
6afa2c299e
|
break if your dataset size is smaller than your batch size
|
2023-02-17 04:08:27 +07:00 |
|
mrq
|
94d0f16608
|
Necessary fixes to get it to work
|
2023-02-17 02:03:00 +07:00 |
|
mrq
|
49e23b226b
|
pip-ify
|
2023-02-17 00:33:50 +07:00 |
James Betker
|
f31a333c4f
|
more sampling fixes
|
2022-10-10 20:11:28 +07:00 |
|
James Betker
|
5d172fbf7e
|
Fix eval
|
2022-10-10 14:22:36 +07:00 |
|
James Betker
|
9502e0755e
|
ugh
|
2022-10-10 12:15:51 +07:00 |
|
James Betker
|
fce2c8f5db
|
and listify them
|
2022-10-10 12:13:49 +07:00 |
|
James Betker
|
3cf78e3c44
|
train mel head even when not
|
2022-10-10 12:10:56 +07:00 |
|
James Betker
|
cc74a43675
|
Checkin
|
2022-10-10 11:30:20 +07:00 |
|
James Betker
|
3cb14123bc
|
glc fix
|
2022-07-29 11:24:36 +07:00 |
|
James Betker
|
4ddd01a7fb
|
support generating cheaters from the new cheater network
|
2022-07-29 09:19:20 +07:00 |
|
James Betker
|
27a9b1b750
|
rename perplexity->log perplexity
|
2022-07-28 09:48:40 +07:00 |
|
James Betker
|
1d68624828
|
fix some imports..
|
2022-07-28 02:35:32 +07:00 |
|
James Betker
|
cfe907f13f
|
i like this better
|
2022-07-28 02:33:23 +07:00 |
|
James Betker
|
d44ed5d12d
|
probably too harsh on ninfs
|
2022-07-28 01:33:54 +07:00 |