Implementing LORA framework on a Merged Diffusion Model for Image Synthesis

Abstract

In this paper, we have tried to explore and understand the working of state-of-art Diffusion models which are being widely used for image synthesis. In addition to this, we have also explored the area for merging two diffusion models with different parameters setting and weightage to build a merged diffusion model which exhibits the features of both the models. To enhance the experimentation further we have tried to implement the famous LORA framework on the merged diffusion model which helps in fine tuning of the model, along with reducing the computational power required to finetune a stable diffusion model in the general sense.