Basics
Dreambooth is a new approach for "personalizing" text-to-image synthesis models, allowing them to generate novel photorealistic images of specific subjects in different contexts while preserving their key identifying features.
The approach involves fine-tuning a pre-trained, diffusion-based text-to-image framework using low-resolution versions of the subject's images and text prompts containing a unique identifier followed by the class name of the subject.
A class-specific prior preservation loss is also introduced to prevent overfitting and encourage the generation of diverse instances of the same class.
TLDR: few of your images+model images => good model
Finetuning is the standard approach for big datasets, only the captions of the images are used [filewords].
Class images are not used.
This results in a model that doesn't need instance token, and reacts to any prompt.
Hybrid for a lack of better term, is achieved using a Instance Token in combination to [filewords] as instance prompt. The Trained Dataset will be linked to that Instance Token.
This minimize the bleed but requires the token in every prompt (ohwx french bulldog)
LoRA, fine-tunes the "residual" of a model instead of the entire model by decomposing it into low-rank matrices and only fine-tuning certain parameters, such as the attention layers of a transformer model. This results in a much smaller model.
Model selection is used to select past created models, to either resume training or load the parameters.
Gets pretty boring after a while.
Create tab is used to create the Diffusers version of the ckpt model, making it available for training.
If it's your first model, stable-diffusion-v1-5 it's a good starting point.
Settings tab is the probably the most intimidating tab of the extension, on the flip side most of the setting will be set and forget.
Wrong values here will either result in Out of memory or Fried models.
You have been warned
Concept tab is where the dataset is defined: instance images, captions, class images they are all defined here.
If your instance token doesn't work, this probably is where you messed up.
Saving is well, saving related stuffs
Use half models, and don't get greedy with the settings or you might need to buy a new SSD
If you didn't want to commit your precious free bytes to the ckpt files in the saving tab...
Good news! This is where can try out you model till you are sure it deserves a ckpt, remember half models.
You can even do some other activities to pass the time.
The output section of the extension inform of the status of the extension.
It will show training progress during training, showing the current steps, the total steps, the lifetime steps and the trained epochs
On training completion it will show a generated sample image paired with a loss graph and a VRAM usage graph
