CResMD

Interactive Multi-Dimension Modulation with Dynamic Controllable Residual Learning for Image Restoration Paper

By Jingwen He*, Chao Dong*, and Yu Qiao (* indicates equal contribution)

two-dimension modulation

three-dimension modulation

Demo video of two-dimension modulation.

Dependencies and Installation

• Python 3 (Recommend to use Anaconda)
• PyTorch >= 1.0
• NVIDIA GPU + CUDA
• Python packages: pip install numpy opencv-python lmdb pyyaml
• TensorBoard:
  • PyTorch >= 1.1: pip install tb-nightly future
  • PyTorch == 1.0: pip install tensorboardX

How to Test

• Prepare the test dataset

  1. Download LIVE1 dataset and CBSD68 dataset from Google Drive
  2. Generate LQ images with different combinations of degradations using matlab codes/data_scripts/generate_2D_val.m, codes/data_scripts/generate_3D_val.m.

• Modulation Testing

  1. (optional) Modify the configuration file options/test/modulation_CResMD.yml.
  • dataroot_GT, dataroot_LQ, pretrain_model_G.
  • cond_init: The starting point of modulation, usually set to [0, 0].
  • modulation_dim: The dimension you would like to modulate.
  • modulation_stride: The stride for modulation process, usually set to 0.1.
  1. Run command:

   cd codes
   python modulation_CResMD.py -opt options/test/modulation_CResMD.yml

• Test CResMD

  1. (optional) Modify the configuration file options/test/test_CResMD.yml.
  • dataroot_GT, dataroot_LQ
  • cond_norm (This paper uses [40, 50] for 2D modulation, [40, 50, 92] for 3D modulation).
  • mode (LQGT or LQGT_cond. if it is set to LQGT, you should specify cond, which is the degradation levels.)
  1. Run command:

   python test.py -opt options/test/test_CResMD.yml

• Test base network

  1. Modify the configuration file options/test/test_Base.yml.
  2. Run command:

   python test.py -opt options/test/test_Base.yml

How to Train

• CResMD

  1. Prepare datasets, usually the DIV2K dataset. More details are in codes/data.
  2. Modify the configuration file options/train/train_CResMD.yml
  • dataroot_GT, dataroot_LQ
  • cond_norm (This paper uses [40, 50] for 2D modulation, [40, 50, 92] for 3D modulation).
  1. Run command:

   python train_CResMD.py -opt options/train/train_CResMD.yml

• base network

  1. Prepare datasets, usually the DIV2K dataset. More details are in codes/data.
  2. Modify the configuration file options/train/train_Base.yml
  3. Run command:

   python train.py -opt options/train/train_Base.yml

Acknowledgement

• This code is based on mmsr.