Is there a way to keep models in memory?

[KaruroChori]

It seems to me that for each task models are loaded and offloaded from the GPU memory at each task execution.
Would it be possible to keep them cached and make the generation process faster as long as they are still the same?
Because at the moment this seems to lead to a massive performance hit compared to auto1111, at least on my system.

[comfyanonymous]

You can use this command line option to always keep them in memory: --highvram

[tiago-falves]

I have a problem in which I have two workflows, which I run in turns.
The problem is that if I run the first workflow, it loads the model. I then run the second workflow, and when the first workflow again, it loads the model again. Is there a way to keep the model loaded?
With the --highvram option, it loads the model again.
Thanks in advance! @comfyanonymous

[Expertaz]

@tiago-falves Hey ! I have two workflows, which I run in turns too. Have you found any solution ? Thanks :)

[tiago-falves]

Hey!
The solution is unifying both workflows, and change the first connection, depending on which workflow you want to use!
This way it keeps both models loaded!

[Expertaz]

@tiago-falves haha thanks! Could you please spare one more minute to explain how to change the initial connection depending on the workflow I intend to use? That would be appreciated (:

[tiago-falves]

1. Place Both Workflows on the Same Page: Begin by arranging both workflows on the same workspace in Comfy.
2. Shared "Base Model" Node: Both workflows will use the same "Base Model" node. This node acts as a starting point for the data processing or analysis performed in each workflow. This is just an example, the same logic would apply if you would have different models, but one "preview image" node in common for example.
3. Connecting the Base Model to a Workflow: To run a workflow, connect the output of the "Base Model" node to the starting point (beginning node) of the desired workflow. This connection directs the output data from the model into the selected workflow for further processing.
4. Running the Workflow: Once the connection is made, execute the workflow. The data will flow from the "Base Model" through the connected workflow, processing as specified within that workflow's sequence of nodes.
5. Switching Workflows: If you need to switch to the other workflow:
   a. Disconnect the Current Workflow: First, disconnect the "Base Model" node from the currently active workflow. This stops data flow to this workflow.
   b. Connect to the New Workflow: Next, connect the "Base Model" node’s output to the beginning of the other workflow. This initiates data flow into the new workflow.

[efwfe]

Hello, try this. #3545

[F0xbite]

How can you achieve this when using the API?

[AI-taja]

Did you find a solution?

[F0xbite]

Did you find a solution?

Actually, yes, check out Inspire Shared Checkpoint loader. These nodes are amazing, exactly what we needed.

With his nodes, you can actually cache anything, loras, checkpoints, models, latents, anything! One of my favorite node packs.
https://github.com/ltdrdata/ComfyUI-Inspire-Pack