forked from ostris/ai-toolkit
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Modal cloud training support, fixed typo in toolkit/scheduler.py, Sch…
…nell training support for Colab, issue ostris#92 , issue ostris#114 (ostris#115) * issue ostris#76, load_checkpoint_and_dispatch() 'force_hooks' ostris#76 * RunPod cloud config ostris#90 * change 2x A40 to 1x A40 and price per hour referring to ostris#90 (comment) * include missed FLUX.1-schnell setup guide in last commit * huggingface-cli login required auth * ostris#92 peft, ostris#114 colab, schnell training in colab * modal cloud - run_modal.py and .yaml configs * run_modal.py mount path example * modal_examples renamed to modal * Training in Modal README.md setup guide * rename run command in title for consistency
- Loading branch information
Showing
8 changed files
with
817 additions
and
89 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,96 @@ | ||
| --- | ||
| job: extension | ||
| config: | ||
| # this name will be the folder and filename name | ||
| name: "my_first_flux_lora_v1" | ||
| process: | ||
| - type: 'sd_trainer' | ||
| # root folder to save training sessions/samples/weights | ||
| training_folder: "/root/ai-toolkit/modal_output" # must match MOUNT_DIR from run_modal.py | ||
| # uncomment to see performance stats in the terminal every N steps | ||
| # performance_log_every: 1000 | ||
| device: cuda:0 | ||
| # if a trigger word is specified, it will be added to captions of training data if it does not already exist | ||
| # alternatively, in your captions you can add [trigger] and it will be replaced with the trigger word | ||
| # trigger_word: "p3r5on" | ||
| network: | ||
| type: "lora" | ||
| linear: 16 | ||
| linear_alpha: 16 | ||
| save: | ||
| dtype: float16 # precision to save | ||
| save_every: 250 # save every this many steps | ||
| max_step_saves_to_keep: 4 # how many intermittent saves to keep | ||
| datasets: | ||
| # datasets are a folder of images. captions need to be txt files with the same name as the image | ||
| # for instance image2.jpg and image2.txt. Only jpg, jpeg, and png are supported currently | ||
| # images will automatically be resized and bucketed into the resolution specified | ||
| # on windows, escape back slashes with another backslash so | ||
| # "C:\\path\\to\\images\\folder" | ||
| # your dataset must be placed in /ai-toolkit and /root is for modal to find the dir: | ||
| - folder_path: "/root/ai-toolkit/your-dataset" | ||
| caption_ext: "txt" | ||
| caption_dropout_rate: 0.05 # will drop out the caption 5% of time | ||
| shuffle_tokens: false # shuffle caption order, split by commas | ||
| cache_latents_to_disk: true # leave this true unless you know what you're doing | ||
| resolution: [ 512, 768, 1024 ] # flux enjoys multiple resolutions | ||
| train: | ||
| batch_size: 1 | ||
| steps: 2000 # total number of steps to train 500 - 4000 is a good range | ||
| gradient_accumulation_steps: 1 | ||
| train_unet: true | ||
| train_text_encoder: false # probably won't work with flux | ||
| gradient_checkpointing: true # need the on unless you have a ton of vram | ||
| noise_scheduler: "flowmatch" # for training only | ||
| optimizer: "adamw8bit" | ||
| lr: 1e-4 | ||
| # uncomment this to skip the pre training sample | ||
| # skip_first_sample: true | ||
| # uncomment to completely disable sampling | ||
| # disable_sampling: true | ||
| # uncomment to use new vell curved weighting. Experimental but may produce better results | ||
| # linear_timesteps: true | ||
|
|
||
| # ema will smooth out learning, but could slow it down. Recommended to leave on. | ||
| ema_config: | ||
| use_ema: true | ||
| ema_decay: 0.99 | ||
|
|
||
| # will probably need this if gpu supports it for flux, other dtypes may not work correctly | ||
| dtype: bf16 | ||
| model: | ||
| # huggingface model name or path | ||
| # if you get an error, or get stuck while downloading, | ||
| # check https://github.com/ostris/ai-toolkit/issues/84, download the model locally and | ||
| # place it like "/root/ai-toolkit/FLUX.1-dev" | ||
| name_or_path: "black-forest-labs/FLUX.1-dev" | ||
| is_flux: true | ||
| quantize: true # run 8bit mixed precision | ||
| # low_vram: true # uncomment this if the GPU is connected to your monitors. It will use less vram to quantize, but is slower. | ||
| sample: | ||
| sampler: "flowmatch" # must match train.noise_scheduler | ||
| sample_every: 250 # sample every this many steps | ||
| width: 1024 | ||
| height: 1024 | ||
| prompts: | ||
| # you can add [trigger] to the prompts here and it will be replaced with the trigger word | ||
| # - "[trigger] holding a sign that says 'I LOVE PROMPTS!'"\ | ||
| - "woman with red hair, playing chess at the park, bomb going off in the background" | ||
| - "a woman holding a coffee cup, in a beanie, sitting at a cafe" | ||
| - "a horse is a DJ at a night club, fish eye lens, smoke machine, lazer lights, holding a martini" | ||
| - "a man showing off his cool new t shirt at the beach, a shark is jumping out of the water in the background" | ||
| - "a bear building a log cabin in the snow covered mountains" | ||
| - "woman playing the guitar, on stage, singing a song, laser lights, punk rocker" | ||
| - "hipster man with a beard, building a chair, in a wood shop" | ||
| - "photo of a man, white background, medium shot, modeling clothing, studio lighting, white backdrop" | ||
| - "a man holding a sign that says, 'this is a sign'" | ||
| - "a bulldog, in a post apocalyptic world, with a shotgun, in a leather jacket, in a desert, with a motorcycle" | ||
| neg: "" # not used on flux | ||
| seed: 42 | ||
| walk_seed: true | ||
| guidance_scale: 4 | ||
| sample_steps: 20 | ||
| # you can add any additional meta info here. [name] is replaced with config name at top | ||
| meta: | ||
| name: "[name]" | ||
| version: '1.0' |
98 changes: 98 additions & 0 deletions
98
config/examples/modal/modal_train_lora_flux_schnell_24gb.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| --- | ||
| job: extension | ||
| config: | ||
| # this name will be the folder and filename name | ||
| name: "my_first_flux_lora_v1" | ||
| process: | ||
| - type: 'sd_trainer' | ||
| # root folder to save training sessions/samples/weights | ||
| training_folder: "/root/ai-toolkit/modal_output" # must match MOUNT_DIR from run_modal.py | ||
| # uncomment to see performance stats in the terminal every N steps | ||
| # performance_log_every: 1000 | ||
| device: cuda:0 | ||
| # if a trigger word is specified, it will be added to captions of training data if it does not already exist | ||
| # alternatively, in your captions you can add [trigger] and it will be replaced with the trigger word | ||
| # trigger_word: "p3r5on" | ||
| network: | ||
| type: "lora" | ||
| linear: 16 | ||
| linear_alpha: 16 | ||
| save: | ||
| dtype: float16 # precision to save | ||
| save_every: 250 # save every this many steps | ||
| max_step_saves_to_keep: 4 # how many intermittent saves to keep | ||
| datasets: | ||
| # datasets are a folder of images. captions need to be txt files with the same name as the image | ||
| # for instance image2.jpg and image2.txt. Only jpg, jpeg, and png are supported currently | ||
| # images will automatically be resized and bucketed into the resolution specified | ||
| # on windows, escape back slashes with another backslash so | ||
| # "C:\\path\\to\\images\\folder" | ||
| # your dataset must be placed in /ai-toolkit and /root is for modal to find the dir: | ||
| - folder_path: "/root/ai-toolkit/your-dataset" | ||
| caption_ext: "txt" | ||
| caption_dropout_rate: 0.05 # will drop out the caption 5% of time | ||
| shuffle_tokens: false # shuffle caption order, split by commas | ||
| cache_latents_to_disk: true # leave this true unless you know what you're doing | ||
| resolution: [ 512, 768, 1024 ] # flux enjoys multiple resolutions | ||
| train: | ||
| batch_size: 1 | ||
| steps: 2000 # total number of steps to train 500 - 4000 is a good range | ||
| gradient_accumulation_steps: 1 | ||
| train_unet: true | ||
| train_text_encoder: false # probably won't work with flux | ||
| gradient_checkpointing: true # need the on unless you have a ton of vram | ||
| noise_scheduler: "flowmatch" # for training only | ||
| optimizer: "adamw8bit" | ||
| lr: 1e-4 | ||
| # uncomment this to skip the pre training sample | ||
| # skip_first_sample: true | ||
| # uncomment to completely disable sampling | ||
| # disable_sampling: true | ||
| # uncomment to use new vell curved weighting. Experimental but may produce better results | ||
| # linear_timesteps: true | ||
|
|
||
| # ema will smooth out learning, but could slow it down. Recommended to leave on. | ||
| ema_config: | ||
| use_ema: true | ||
| ema_decay: 0.99 | ||
|
|
||
| # will probably need this if gpu supports it for flux, other dtypes may not work correctly | ||
| dtype: bf16 | ||
| model: | ||
| # huggingface model name or path | ||
| # if you get an error, or get stuck while downloading, | ||
| # check https://github.com/ostris/ai-toolkit/issues/84, download the models locally and | ||
| # place them like "/root/ai-toolkit/FLUX.1-schnell" and "/root/ai-toolkit/FLUX.1-schnell-training-adapter" | ||
| name_or_path: "black-forest-labs/FLUX.1-schnell" | ||
| assistant_lora_path: "ostris/FLUX.1-schnell-training-adapter" # Required for flux schnell training | ||
| is_flux: true | ||
| quantize: true # run 8bit mixed precision | ||
| # low_vram is painfully slow to fuse in the adapter avoid it unless absolutely necessary | ||
| # low_vram: true # uncomment this if the GPU is connected to your monitors. It will use less vram to quantize, but is slower. | ||
| sample: | ||
| sampler: "flowmatch" # must match train.noise_scheduler | ||
| sample_every: 250 # sample every this many steps | ||
| width: 1024 | ||
| height: 1024 | ||
| prompts: | ||
| # you can add [trigger] to the prompts here and it will be replaced with the trigger word | ||
| # - "[trigger] holding a sign that says 'I LOVE PROMPTS!'"\ | ||
| - "woman with red hair, playing chess at the park, bomb going off in the background" | ||
| - "a woman holding a coffee cup, in a beanie, sitting at a cafe" | ||
| - "a horse is a DJ at a night club, fish eye lens, smoke machine, lazer lights, holding a martini" | ||
| - "a man showing off his cool new t shirt at the beach, a shark is jumping out of the water in the background" | ||
| - "a bear building a log cabin in the snow covered mountains" | ||
| - "woman playing the guitar, on stage, singing a song, laser lights, punk rocker" | ||
| - "hipster man with a beard, building a chair, in a wood shop" | ||
| - "photo of a man, white background, medium shot, modeling clothing, studio lighting, white backdrop" | ||
| - "a man holding a sign that says, 'this is a sign'" | ||
| - "a bulldog, in a post apocalyptic world, with a shotgun, in a leather jacket, in a desert, with a motorcycle" | ||
| neg: "" # not used on flux | ||
| seed: 42 | ||
| walk_seed: true | ||
| guidance_scale: 1 # schnell does not do guidance | ||
| sample_steps: 4 # 1 - 4 works well | ||
| # you can add any additional meta info here. [name] is replaced with config name at top | ||
| meta: | ||
| name: "[name]" | ||
| version: '1.0' |
Oops, something went wrong.