[Question]: Deployed worker using my own Docker Image gets stuck in queue on RunPod and never executes (Custom Nodes)

[billyberkouwer]

I am trying to use this worker with custom nodes. When I deploy using your docker image, all works smoothly on RunPod as expected. However, when I fork the repo, create a local image, then push and deploy from my own repo, the worker deploys but my requests get stuck in the queue and never execute.

I have followed the documentation for editing the docker image to use custom nodes. However, this issue occurs even when I am using an unedited version of the Dockerfile.

I am sure that I am missing something but I am unsure why even an unedited copy of this repo produces an image that doesn't work when the image you have supplied does.

My process:

1. Clone repo
2. Run command from directory: e.g. docker build --build-arg MODEL_TYPE=sdxl -t billymylesberkouwer/runpod-worker-comfy:dev-sdxl --platform linux/amd64 .
3. Push image to my hub
4. Create a new RunPod template using billymylesberkouwer/runpod-worker-comfy:dev-sdxl
5. Create a new Serverless using this template
6. Use contents of test_input.json to create a request using RunPod request dashboard

Result:

An endless delay, unhealthy workers, no execution

Expected result:

A success or failure message and successful execution

Any help would be greatly appreciated. Many thanks in advance!

[billyberkouwer]

I have found that using this approach suggested by ecornbower has successfully deployed and is returning succesfully using test_input.json. I would still appreciate it if you/anyone else can shed some light on why the approach I took following the documentation is failing it would still be much appreciated!

[TimPietrusky]

@billyberkouwer thanks for reporting this. We will have to dig a bit deeper to make sure to provide a valid response. And also there is also this PR #30 open which should make custom nodes working for everyone. The thing is just time, but we will make sure to include this, as this is super important for a lot of people.

[billyberkouwer]

@TimPietrusky Many thanks for getting back to me on this! This PR looks like a great solution. I'm currently working on another aspect of my project but will comment on the PR to feedback if I end up adopting it in production. For now, a reduction in the number of nodes and some additional installs and updates based on errors that were thrown seem to have provided a solution. But the ability to use ComfyUI manager snapshots and the full range of nodes sounds preferable

[martintomov]

@billyberkouwer did you test out PR #30 in the end? I'm facing the same issue - getting an endless delay, unhealthy workers, no execution. I forked the PR and I loaded a snapshot from ComfyUI-Manager + a workflow.

[billyberkouwer]

@martintomov I didn't test out the PR - my solution was to pull the latest version of runpod-worker-comfy (v3.1.0), edit as needed, build, and then use this approach to build a new version which installs the custom nodes. I still found that some custom node packs didn't install (e.g. WAS and a few others) but the ComfyUI developer I was working with was able to replace these nodes with ones from other packs that did work.

I also found it's important to edit and build the Docker files as well as any python in WSL Linux (I'm on Windows) to ensure there weren't idiosyncrasies produced with line endings and file paths that could fail the build.