From 54acc621d4878f3dfe8edc6de08e44b684229a5c Mon Sep 17 00:00:00 2001 From: enhuiz Date: Thu, 12 Jan 2023 14:56:30 +0800 Subject: [PATCH] Update docs --- README.md | 54 ++++++++++++++++++++++++++++++++++++++++++------ requirements.txt | 16 ++++++++++++++ 2 files changed, 64 insertions(+), 6 deletions(-) create mode 100644 requirements.txt diff --git a/README.md b/README.md index 2a4f723..3e73017 100644 --- a/README.md +++ b/README.md @@ -1,13 +1,55 @@ # VALL-E -An unofficial (toy) implementation of VALL-E, based on the [encodec](https://github.com/facebookresearch/encodec) tokenizer. +An unofficial (toy) implementation of [VALL-E](https://valle-demo.github.io/), based on the [encodec](https://github.com/facebookresearch/encodec) tokenizer. [!["Buy Me A Coffee"](https://www.buymeacoffee.com/assets/img/custom_images/orange_img.png)](https://www.buymeacoffee.com/enhuiz) +## Requirements + +### 1. Clone the project + +``` +git clone --recurse-submodules https://github.com/enhuiz/vall-e.git +``` + +### 2. Install requirements + +``` +pip install -r requirements.txt +``` + +Note: You may need to install additional requirements as you run the script. + +## Data Preparation + +1. Put your data into a folder, e.g. `data/your_data`. Audio files should be named with the suffix `.wav` and text files with `.normalized.txt`. + +2. Quantize the data: + +``` +python -m vall_e.emb.qnt data/your_data +``` + +3. Generate phonemes based on the text: + +``` +python -m vall_e.emb.g2p data/your_data +``` + +4. Customize your configuration by creating `config/your_data/ar.yml` and `config/your_data/nar.yml`. + +- Refer to the example configs in `config/test` and `vall_e/config.py` for details. + +5. Train the AR or NAR model using the following scripts: + +``` +python -m vall_e.train yaml=config/your_data/ar_or_nar.yml +``` + ## TODO -- [x] AR model for the first quantizer. -- [x] Audio decoding from tokens. -- [x] NAR model for the rest quantizers. -- [x] Trainers for both models. -- [ ] Pre-trained checkpoint. +- [x] AR model for the first quantizer +- [x] Audio decoding from tokens +- [x] NAR model for the rest quantizers +- [x] Trainers for both models +- [ ] Pre-trained checkpoint and demos on LibriTTS diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..922a21a --- /dev/null +++ b/requirements.txt @@ -0,0 +1,16 @@ +coloredlogs==15.0.1 +deepspeed==0.7.7 +diskcache==5.4.0 +einops==0.6.0 +encodec==0.1.1 +g2p_en==2.1.0 +humanize==4.4.0 +matplotlib==3.6.0 +numpy==1.23.3 +omegaconf==2.2.3 +openTSNE==0.6.2 +pandas==1.5.0 +soundfile==0.11.0 +torch==1.13.0+cu116 +torchaudio==0.13.0+cu116 +tqdm==4.64.1