mrq/vall-e
5
9
Fork 0
Code Issues 8 Pull Requests Packages Projects Releases Wiki Activity
An unofficial PyTorch implementation of VALL-E
17 Commits 1 Branch 0 Tags 20 MiB
Python 59.5%
C 29.6%
C++ 10.8%
45fc310876
Go to file
Zhe Niu 45fc310876
Create LICENSE
2023-01-12 15:01:26 +08:00
config Add trainer 2023-01-12 14:41:44 +08:00
data/test Add trainer 2023-01-12 14:41:44 +08:00
vall_e Add trainer 2023-01-12 14:41:44 +08:00
.gitignore Add trainer 2023-01-12 14:41:44 +08:00
.gitmodules Init 2023-01-11 23:03:25 +08:00
LICENSE Create LICENSE 2023-01-12 15:01:26 +08:00
README.md Update README.md 2023-01-12 14:57:28 +08:00
requirements.txt Update docs 2023-01-12 14:56:30 +08:00

README.md

VALL-E

An unofficial (toy) implementation of VALL-E, based on the encodec tokenizer.

"Buy Me A Coffee"

Requirements

1. Clone the project

git clone --recurse-submodules https://github.com/enhuiz/vall-e.git

2. Install requirements

pip install -r requirements.txt

Note: You may need to install additional requirements as you run the script.

Data Preparation

  1. Put your data into a folder, e.g. data/your_data. Audio files should be named with the suffix .wav and text files with .normalized.txt.

  2. Quantize the data:

python -m vall_e.emb.qnt data/your_data
  1. Generate phonemes based on the text:
python -m vall_e.emb.g2p data/your_data

  1. Customize your configuration by creating config/your_data/ar.yml and config/your_data/nar.yml. Refer to the example configs in config/test and vall_e/config.py for details.

  2. Train the AR or NAR model using the following scripts:

python -m vall_e.train yaml=config/your_data/ar_or_nar.yml

TODO

  • AR model for the first quantizer
  • Audio decoding from tokens
  • NAR model for the rest quantizers
  • Trainers for both models
  • Pre-trained checkpoint and demos on LibriTTS