feat(runner): add support for SDXL-Lightning in image-to-image pipelines #40
Conversation
This commit adds the [SDXL-Lightning](https://huggingface.co/ByteDance/SDXL-Lightning/blob/main/sdxl_lightning_samples.jpg) model to the image-to-image pipelines.
| @@ -37,8 +48,49 @@ def __init__(self, model_id: str): | |||
| kwargs["variant"] = "fp16" | |||
|
|
|||
| self.model_id = model_id | |||
| self.ldm = AutoPipelineForImage2Image.from_pretrained(model_id, **kwargs) | |||
There was a problem hiding this comment.
I've made some minor changes here. First, I combine the pipeline initialization and GPU allocation steps in one line. Moreover, I've transitioned to using torch_device variable.
This commit adds the [modal](https://modal.com/) endpoints for the [SDXL_Lightning](https://huggingface.co/ByteDance/SDXL-Lightning/blob/main/sdxl_lightning_samples.jpg) image-to-image pipelines.
| @@ -164,6 +164,24 @@ def text_to_image_sdxl_lightning_8step_api(): | |||
| return make_api("text-to-image", "ByteDance/SDXL-Lightning-8step") | |||
|
|
|||
|
|
|||
| @stub.function(image=api_image, secrets=[Secret.from_name("api-auth-token")]) | |||
There was a problem hiding this comment.
The Starter plan for Modal only supports 8 deployed endpoints so as-is I think deploying this app would fail. Assuming that is the case, I would recommend removing the following endpoints to stay under the limit:
image_to_video_svd_api()-> can just use the SVD1.1 endpoint
There was a problem hiding this comment.
Good catch. I noticed that error yesterday when I deployed a test endpoint to my modal account. I have now commented out the SVD1.0 endpoint in 7334b3a 👍🏻. We can remove it if we agree with the more fixed 6FPS and Motion Bucket Id 127 hyperparameters on which the SVD1.1 was trained.
This commit comments out the `stabilityai/stable-video-diffusion-img2vid-xt` function to adhere to modal.com's starter plan which only includes 8 deployed endpoints.
This pull request adds the SDXL-Lightning model to the image-to-image pipelines.
Fixes LIV-97