Sagiri: Low Dynamic Range Image Enhancement with Generative Diffusion Prior

Paper | Project Page

Baiang Li^{1, 5}, Sizhuo Ma³, Yanhong Zeng¹, Xiaogang Xu^{2, 4}, Youqing Fang¹, Zhao Zhang⁵, Jian Wang^3✝, Kai Chen^1✝
✝Corresponding Authors.
¹Shanghai AI Laboratory
²The Chinese University of Hong Kong
³Snap Inc.
⁴Zhejiang University
⁵Hefei University of Technology

Our task:

📖Table Of Contents

• Visual Results
• Update
• Installation
• Pretrained Models
• Inference
• Train

Visual results on entire real-world image

Left: Input image; Medium: After previous methods; Right: LS-Sagiri(Ours)

Visual results on selected region

Left: Input image; Medium: Previous methods; Right: LS-Sagiri(Ours)

Sagiri plugged after other methods

Left: Input image; Medium: SingleHDR; Right: SingleHDR+Sagiri(Ours)

Left: Input image; Medium: LCDPNet; Right: LCDPNet+Sagiri(Ours)

Controlling where and what to generate

First: Input image; Second: SingleHDR; Third: SingleHDR+Sagiri (with prompt a); Fourth: SingleHDR+Sagiri (with prompt b);
Prompt a: ``A white waterfall is flowing down from the cliff, surrounded by rocks and trees.'';
Prompt b: ``The sun is setting, and the sky is filled with clouds.''

Update

• 2024.06: This repo is released.

Start installation

# clone this repo
git clone https://github.com/ztMotaLee/Sagiri.git
cd Sagiri

# create an environment with python >= 3.9
conda create -n sagiri python=3.9
conda activate sagiri
pip install -r requirements.txt

Pretrained Models

Model Name	Description	BaiduNetdisk
stage1.ckpt	Stage1 for brightness and color adjustment.	download
stage2.ckpt	Sagiri for conditional image generation.	download

Inference

Stage 1 inference

Note that we can use other restoration models to finish stage 1's process.

python scripts/inference_stage1.py \
--config configs/model/swinir.yaml \
--ckpt /path/to/stage1/model \
--input /path/to/input/images \
--output /path/to/output/images

Sagiri inference

python infer_Sagiri.py \
--config configs/model/cldm.yaml \
--ckpt  /path/to/stage2/model\
--steps 30 \
--input /path/to/input/images \
--output /path/to/output/images \
--disable_preprocess_model \
--device cuda

LS-Sagiri inference

python infer_LSSagiri.py \
--config configs/model/cldm.yaml \
--ckpt  /path/to/stage2/model\
--steps 30 \
--input /path/to/input/images \
--output /path/to/output/images \
--device cuda

Start training

```shell
wget https://huggingface.co/stabilityai/stable-diffusion-2-1-base/resolve/main/v2-1_512-ema-pruned.ckpt --no-check-certificate

```

and

```shell
python train.py --config [training_config_path]
```

Citation

Please cite us if our work is useful for your research.

@article{li2024sagiri,
  author    = {Baiang Li and Sizhuo Ma and Yanhong Zeng and Xiaogang Xu and Youqing Fang and Zhao Zhang and Jian Wang and Kai Chen},
  title     = {Sagiri: Low Dynamic Range Image Enhancement with Generative Diffusion Prior},
  journal   = {arxiv},
  year      = {2024},
}

License

This project is released under the Apache 2.0 license.

Acknowledgement

This project is based on ControlNet, BasicSR and DiffBIR. Thanks for their awesome work.

Contact

Should you have any questions, please feel free to contact with me at [email protected].