add DATASETS.md

2019-08-23 21:45:54 +08:00 · 2019-08-23 21:45:54 +08:00 · dfdaaef492
commit dfdaaef492
parent 037933ba66
1 changed files with 255 additions and 0 deletions
--- a/datasets/DATASETS.md
+++ b/datasets/DATASETS.md
@ -0,0 +1,255 @@
+There are three kinds of datasets: training dataset, validation dataset, and testing dataset. Usually, we do not explicitly distinguish between the validation and testing datasets in image/video restoration. So we use the validation/testing dataset in our description. <br/>
+We recommend to use [LMDB](https://lmdb.readthedocs.io/en/release/) (Lightning Memory-Mapped Database) formats for the training datasets, and directly read images (using image folder) during validation/testing. So there is no need to prepare LMDB files for evaluation/testing datasets.
+
+---
+We organize the training datasets in LMDB format for **faster training IO speed**. If you do not want to use LMDB, you can also use the **image folder**.<br/>
+Besides the standard LMDB folder, we add an extra `meta_info.pkl` file to record the **meta information** of the dataset, such as the dataset name, keys and resolution of each image in the dataset.
+
+Take the DIV2K dataset in LMDB for example, the folder structure and meta information are as follows:
+#### folder structure
+```
+- DIV2K800_sub.lmdb
+|--- data.mdb
+|--- lock.mdb
+|--- meta_info.pkl
+```
+#### meta information in `meta_info.pkl`
+`meta_info.pkl` is a python-pickled dict.
+
+|     Key    |                           Value                           |
+|:----------:|:---------------------------------------------------------:|
+|    name    |                `DIV2K800_sub_GT`                    |
+|    keys    |   [ `0001_s001`, `0001_s002`, ..., `0800_s040` ] |
+| resolution |                       [ `3_480_480` ]                    |
+
+If all the images in the LMDB file have the same resolution, only one copy of `resolution` is stored. Otherwise, each key has its corresponding `resolution`.
+
+----
+
+## Table of Contents
+1. [Prepare DIV2K](#prepare-div2k)
+1. [Common Image SR Datasets](#common-image-sr-datasets)
+1. [Prepare Vimeo90K](#prepare-vimeo90k)
+1. [Prepare REDS](#prepare-reds)
+
+The following shows how to prepare the datasets in detail.<br/>
+It is recommended to symlink the dataset root to $MMSR/datasets. If your folder structure is different, you may need to change the corresponding paths in config files.
+
+## Prepare DIV2K
+[DIV2K](https://data.vision.ee.ethz.ch/cvl/DIV2K/) is a widely-used dataset in image super-resolution. In many research works, a MATLAB bicubic downsampling kernel is assumed. It may not be practical because the MATLAB bicubic downsampling kernel is not a good approximation for the implicit degradation kernels in real-world scenarios. And there is another topic named **blind restoration** that deals with this gap.
+
+We provide a demo script for DIV2K X4 datasets preparation.
+```
+cd codes/data_scripts
+bash prepare_DIV2K_x4_dataset.sh
+```
+The specific steps are  as follows:
+
+**Step 1**: Download the GT images and corresponding LR images from the [official DIV2K website](https://data.vision.ee.ethz.ch/cvl/DIV2K/).<br/>
+Here are shortcuts for the download links:
+
+| Name | links (training) | links (validation)|
+|:----------:|:----------:|:----------:|
+|Ground-Truth|[DIV2K_train_HR](http://data.vision.ee.ethz.ch/cvl/DIV2K/DIV2K_train_HR.zip)|[DIV2K_valid_HR](http://data.vision.ee.ethz.ch/cvl/DIV2K/DIV2K_valid_HR.zip)|
+|LRx2 (MATLAB bicubic)|[DIV2K_train_LR_bicubic_X2](http://data.vision.ee.ethz.ch/cvl/DIV2K/DIV2K_train_LR_bicubic_X2.zip)|[DIV2K_valid_LR_bicubic_X2](http://data.vision.ee.ethz.ch/cvl/DIV2K/DIV2K_valid_LR_bicubic_X2.zip)|
+|LRx3 (MATLAB bicubic)|[DIV2K_train_LR_bicubic_X3](http://data.vision.ee.ethz.ch/cvl/DIV2K/DIV2K_train_LR_bicubic_X3.zip)|[DIV2K_valid_LR_bicubic_X3](http://data.vision.ee.ethz.ch/cvl/DIV2K/DIV2K_valid_LR_bicubic_X3.zip)|
+|LRx4 (MATLAB bicubic)|[DIV2K_train_LR_bicubic_X4](http://data.vision.ee.ethz.ch/cvl/DIV2K/DIV2K_train_LR_bicubic_X4.zip)|[DIV2K_valid_LR_bicubic_X4](http://data.vision.ee.ethz.ch/cvl/DIV2K/DIV2K_valid_LR_bicubic_X4.zip)|
+|LRx8 (MATLAB bicubic)|[DIV2K_train_LR_bicubic_X8](http://data.vision.ee.ethz.ch/cvl/DIV2K/DIV2K_train_LR_x8.zip)|[DIV2K_valid_LR_bicubic_X8](http://data.vision.ee.ethz.ch/cvl/DIV2K/DIV2K_valid_LR_x8.zip)|
+
+**Step 2**: Rename the downloaded LR images to have the same name as those of GT.<br/> Run the script `data_scripts/rename.py`. Remember to modify the folder path.
+
+**Step 3 (optional)**: Generate low-resolution counterparts. <br/>If you have downloaded the LR datasets, skip this step. Otherwise, you can use the script `data_scripts/generate_mod_LR_bic.m` or `data_scripts/generate_mod_LR_bic.py` to generate LR images. Make sure the LR and GT pairs have the same name.
+
+**Step 4**: Crop to sub-images. <br/>DIV2K has 2K resolution (e.g., 2048 × 1080) images but the training patches are usually very small (e.g., 128x128). So there is a waste if reading the whole image but only using a very small part of it. In order to accelerate the IO speed during training, we crop the 2K resolution images to sub-images (here, we crop to 480x480 sub-images). You can skip this step if your have a high IO speed.<br/>
+Note that the size of sub-images is different from the training patch size (`GT_size`) defined in the config file. Specifically, the sub-images with 480x480 are stored in the LMDB files. The dataloader will further randomly crop the sub-images to `GT_size x GT_size` patches for training. <br/>
+Use the script `data_scripts/extract_subimages.py` with `mode = 'pair'`. Remember to modify the following configurations if you have different settings:
+```
+GT_folder = '../../datasets/DIV2K/DIV2K800'
+LR_folder = '../../datasets/DIV2K/DIV2K800_bicLRx4'
+save_GT_folder = '../../datasets/DIV2K/DIV2K800_sub'
+save_LR_folder = '../../datasets/DIV2K/DIV2K800_sub_bicLRx4'
+scale_ratio = 4
+```
+**Step 5**: Create LMDB files. <br/>You need to run the script `data_scripts/create_lmdb.py` separately for GT and LR images.<br/>
+
+**Step 6**: Test the dataloader with the script `data_scripts/test_dataloader.py`.
+
+This procedure is also applied to other datasets, such as 291 images, or your custom datasets.
+```
+@InProceedings{Agustsson_2017_CVPR_Workshops,
+ author = {Agustsson, Eirikur and Timofte, Radu},
+ title = {NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study},
+ booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops},
+ month = {July},
+ year = {2017}
+}
+```
+## Common Image SR Datasets
+We provide a list of common image super-resolution datasets. You can download the images from the official website or Google Drive or Baidu Drive.
+
+<table>
+  <tr>
+    <th>Name</th>
+    <th>Datasets</th>
+    <th>Short Description</th>
+    <th>Google Drive</th>
+    <th>Baidu Drive</th>
+  </tr>
+  <tr>
+    <td rowspan="3">Classical SR Training</td>
+    <td>T91</td>
+    <td><sub>91 images for training</sub></td>
+    <td rowspan="9"><a href="https://drive.google.com/drive/folders/1pRmhEmmY-tPF7uH8DuVthfHoApZWJ1QU?usp=sharing">Google Drive</a></td>
+    <td rowspan="9"><a href="https://pan.baidu.com/s/1q_1ERCMqALH0xFwjLM0pTg">Baidu Drive</a></td>
+  </tr>
+ <tr>
+    <td><a href="https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/">BSDS200</a></td>
+    <td><sub>A subset (train) of BSD500 for training</sub></td>
+  </tr>
+  <tr>
+    <td><a href="http://mmlab.ie.cuhk.edu.hk/projects/FSRCNN.html">General100</a></td>
+    <td><sub>100 images for training</sub></td>
+  </tr>
+  <tr>
+    <td rowspan="6">Classical SR Testing</td>
+    <td>Set5</td>
+    <td><sub>Set5 test dataset</sub></td>
+  </tr>
+  <tr>
+    <td>Set14</td>
+    <td><sub>Set14 test dataset</sub></td>
+  </tr>
+  <tr>
+    <td><a href="https://www2.eecs.berkeley.edu/Research/Projects/CS/vision/bsds/">BSDS100</a></td>
+    <td><sub>A subset (test) of BSD500 for testing</sub></td>
+  </tr>
+  <tr>
+    <td><a href="https://sites.google.com/site/jbhuang0604/publications/struct_sr">urban100</a></td>
+    <td><sub>100 building images for testing (regular structures)</sub></td>
+  </tr>
+  <tr>
+    <td><a href="http://www.manga109.org/en/">manga109</a></td>
+    <td><sub>109 images of Japanese manga for testing</sub></td>
+  </tr>
+  <tr>
+    <td>historical</td>
+    <td><sub>10 gray LR images without the ground-truth</sub></td>
+  </tr>
+
+  <tr>
+    <td rowspan="3">2K Resolution</td>
+    <td><a href="https://data.vision.ee.ethz.ch/cvl/DIV2K/">DIV2K</a></td>
+    <td><sub>proposed in <a href="http://www.vision.ee.ethz.ch/ntire17/">NTIRE17</a> (800 train and 100 validation)</sub></td>
+    <td rowspan="3"><a href="https://drive.google.com/drive/folders/1B-uaxvV9qeuQ-t7MFiN1oEdA6dKnj2vW?usp=sharing">Google Drive</a></td>
+    <td rowspan="3"><a href="https://pan.baidu.com/s/1CFIML6KfQVYGZSNFrhMXmA">Baidu Drive</a></td>
+  </tr>
+ <tr>
+    <td><a href="https://github.com/LimBee/NTIRE2017">Flickr2K</a></td>
+    <td><sub>2650 2K images from Flickr for training</sub></td>
+  </tr>
+ <tr>
+    <td>DF2K</td>
+    <td><sub>A merged training dataset of DIV2K and Flickr2K</sub></td>
+  </tr>
+
+  <tr>
+    <td rowspan="2">OST (Outdoor Scenes)</td>
+    <td>OST Training</td>
+    <td><sub>7 categories images with rich textures</sub></td>
+    <td rowspan="2"><a href="https://drive.google.com/drive/u/1/folders/1iZfzAxAwOpeutz27HC56_y5RNqnsPPKr">Google Drive</a></td>
+    <td rowspan="2"><a href="https://pan.baidu.com/s/1neUq5tZ4yTnOEAntZpK_rQ#list/path=%2Fpublic%2FSFTGAN&parentPath=%2Fpublic">Baidu Drive</a></td>
+  </tr>
+ <tr>
+    <td>OST300</td>
+    <td><sub>300 test images of outdoor scences</sub></td>
+  </tr>
+
+  <tr>
+    <td >PIRM</td>
+    <td>PIRM</td>
+    <td><sub>PIRM self-val, val, test datasets</sub></td>
+    <td rowspan="2"><a href="https://drive.google.com/drive/folders/17FmdXu5t8wlKwt8extb_nQAdjxUOrb1O?usp=sharing">Google Drive</a></td>
+    <td rowspan="2"><a href="https://pan.baidu.com/s/1gYv4tSJk_RVCbCq4B6UxNQ">Baidu Drive</a></td>
+  </tr>
+</table>
+
+## Prepare Vimeo90K
+The description of the Vimeo90K can be found in [Open-VideoRestoration](https://xinntao.github.io/open-videorestoration/rst_src/datasets_sr.html#vimeo90k) and [the official webpage](http://toflow.csail.mit.edu/).<br/>
+
+**Step 1**: Download the dataset<br/>
+Download the [`Septuplets dataset --> The original training + test set (82GB)`](http://data.csail.mit.edu/tofu/dataset/vimeo_septuplet.zip). This is the Ground-Truth (GT). There is a `sep_trainlist.txt` file recording the training samples in the download zip file.
+
+**Step 2**: Generate the low-resolution images<br/>
+The low-resolution images in the Vimeo90K test dataset are generated with the MATLAB bicubic downsampling kernel. Use the script `data_scripts/generate_LR_Vimeo90K.m` (run in MATLAB) to generate the low-resolution images.
+
+**Step 3**: Create LMDB files<br/>
+Use the script `data_scripts/create_lmdb.py` to generate the lmdb files separately for GT and LR images. You need to modify the configurations in the script:
+1) For GT
+```
+dataset = 'vimeo90K'
+mode = 'GT'
+```
+2) For LR
+```
+dataset = 'vimeo90K'
+mode = 'LR'
+```
+
+**Step 4**: Test the dataloader with the script `data_scripts/test_dataloader.py`.
+
+```
+@Article{xue2017video,
+  author    = {Xue, Tianfan and Chen, Baian and Wu, Jiajun and Wei, Donglai and Freeman, William T},
+  title     = {Video enhancement with task-oriented flow},
+  journal   = {International Journal of Computer Vision},
+  year      = {2017}
+}
+```
+
+## Prepare REDS
+We re-group the REDS training and validation sets as follows:
+
+| name | from | total number |
+|:----------:|:----------:|:----------:|
+| REDS training | the original training (except 4 clips) and validation sets | 266 clips |
+| REDS4 testing | 000, 011, 015 and 020 clips from the *original training set* | 4 clips |
+
+The description of the REDS dataset can be found in [Open-VideoRestoration](https://xinntao.github.io/open-videorestoration/rst_src/datasets_sr.html#reds) and the [official website](https://seungjunnah.github.io/Datasets/reds.html).
+
+**Step 1**: Download the datasets<br/>
+You can download the REDS datasets from the [official website](https://seungjunnah.github.io/Datasets/reds.html). The download links are also sorted as follows:
+
+| track | links (training) | links (validation)|links (testing)|
+|:----------:|:----------:|:----------:|:----------:|
+| Ground-truth| [train_sharp - part1](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/train_sharp_part1.zip), [part2](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/train_sharp_part2.zip), [part3](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/train_sharp_part3.zip) |[val_sharp](https://cv.snu.ac.kr/~snah/Deblur/dataset/REDS/REDS_validation_set/val_sharp.zip) | Not Available |- |
+| SR-clean | [train_sharp_bicubic](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/train_sharp_bicubic.zip) | [val_sharp_bicubic](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/val_sharp_bicubic.zip) |[test_sharp_bicubic](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/test_sharp_bicubic.zip) |
+| SR-blur)  | [train_blur_bicubic](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/train_blur_bicubic.zip) | [val_blur_bicubic](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/val_blur_bicubic.zip) |[test_blur_bicubic](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/test_blur_bicubic.zip) |
+| Deblurring  | [train_blur - part1](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/train_blur_part1.zip),  [part2](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/train_blur_part2.zip), [part3](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/train_blur_part3.zip) | [val_blur](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/val_blur.zip) |[test_blur](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/test_blur.zip) |
+| Deblurring - Compression  | [train_blur_comp - part1](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/train_blur_comp_part1.zip),  [part2](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/train_blur_comp_part2.zip), [part3](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/train_blur_comp_part3.zip) | [val_blur_comp](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/val_blur_comp.zip) |[test_blur_comp](https://data.vision.ee.ethz.ch/timofter/NTIRE19video/test_blur_comp.zip) |
+
+**Step 2**: Re-group the datasets<br/>
+We rename the clips in the original validation set, starting from 240 ... It can be accomplished by `data_scripts/regroup_REDS.py`.
+Note that the REDS4 will be excluded in the data loader, so there is no need to remove the REDS4 explicitly.
+
+**Step 3**: Create LMDB files<br/>
+Use the script `data_scripts/create_lmdb.py` to generate the lmdb files separately for GT and LR frames. You need to modify the configurations in the script:
+1) For GT (train_sharp)
+```
+dataset = 'REDS'
+mode = 'train_sharp'
+```
+2) For LR (train_sharp_bicubic)
+```
+dataset = 'REDS'
+mode = 'train_sharp_bicubic'
+```
+**Step 4**: Test the dataloader with the script `data_scripts/test_dataloader.py`.
+
+```
+@InProceedings{nah2019reds,
+  author    = {Nah, Seungjun and Baik, Sungyong and Hong, Seokil and Moon, Gyeongsik and Son, Sanghyun and Timofte, Radu and Lee, Kyoung Mu},
+  title     = {NTIRE 2019 challenges on video deblurring and super-resolution: Dataset and study},
+  booktitle = {The IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)},
+  year      = {2019}
+}
+```