Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use entrypoint style for end-user app #310

Open
wants to merge 3 commits into
base: dev
Choose a base branch
from

Conversation

jasl
Copy link

@jasl jasl commented Oct 22, 2023

I want to add extra arg --api to the sd-webui, then I found the current Dockerfile not easy to make.
With the entry point style, I could add extra args by ./run.sh $(./autotag stable-diffusion-webui) --api which convenient for people.

I also checked other images and made the same change.

In addition, I set WORKDIR to the app folder.

What do you think?

@jasl
Copy link
Author

jasl commented Oct 22, 2023

Out of topic: GitHub and my local editor warn that many files don't have end-file-new-line.

Do you accept style PR? Or run git ls-files -z | while IFS= read -rd '' f; do if file --mime-encoding "$f" | grep -qv binary; then tail -c1 < "$f" | read -r _ || echo >> "$f"; fi; done to ensure new-line to all files

@jasl jasl marked this pull request as draft October 22, 2023 16:52
@jasl jasl marked this pull request as ready for review October 22, 2023 21:05
@jasl
Copy link
Author

jasl commented Oct 22, 2023

@dusty-nv is there any way to auto-test all containers? now I tested what I modified and looks good

@dusty-nv
Copy link
Owner

Thanks @jasl, in an effort to understand ENTRYPOINT vs CMD I mapped out these common usages: (a) starting the enduser app (server) with different args and (b) running a different process all together (model downloader)

A. Start server with arguments

CMD style example

./run.sh --workdir /opt/text-generation-webui $(./autotag text-generation-webui) \
   python3 server.py --listen --verbose --api \
	--model-dir=/data/models/text-generation-webui \
	--model=llama-2-13b-chat.ggmlv3.q4_0.bin \
	--loader=llamacpp \
	--n-gpu-layers=128 \
	--n_ctx=4096 \
	--n_batch=4096

ENTRYPOINT style example

./run.sh $(./autotag text-generation-webui) \
	--model=llama-2-13b-chat.ggmlv3.q4_0.bin \
	--loader=llamacpp \
	--n-gpu-layers=128 \
	--n_ctx=4096 \
	--n_batch=4096

B. Start different process

CMD style, download model

./run.sh --workdir=/opt/text-generation-webui $(./autotag text-generation-webui) /bin/bash -c \
  'python3 download-model.py --output=/data/models/text-generation-webui TheBloke/Llama-2-7b-Chat-GPTQ'

ENTRYPOINT style, download model

./run.sh --workdir=/opt/text-generation-webui --entrypoint 'python3 download-model.py' \
         $(./autotag text-generation-webui) \
	 --output=/data/models/text-generation-webui TheBloke/Llama-2-7b-Chat-GPTQ

I agree that if user is only adding args, ENTRYPOINT is nicer. What I don't prefer is doing more complicated invocations with it, without needing another bash script that you mount in, then run. But in general I am leaning towards ENTRYPOINT for these. Can you confirm that my last example actually works?

@jasl
Copy link
Author

jasl commented Nov 11, 2023

@dusty-nv

It seems the Dockerfile is broken

From https://github.com/haotian-liu/text-generation-webui
 * [new branch]      dev        -> llava2/dev
 * [new branch]      main       -> llava2/main
Auto-merging extensions/multimodal/pipelines/llava/pipelines.py
CONFLICT (content): Merge conflict in extensions/multimodal/pipelines/llava/pipelines.py
Auto-merging extensions/multimodal/pipelines/llava/llava.py
CONFLICT (content): Merge conflict in extensions/multimodal/pipelines/llava/llava.py
error: could not apply 9c085ee... Initial support for LLaVA-LLaMA-2.
hint: after resolving the conflicts, mark the corrected paths
hint: with 'git add <paths>' or 'git rm <paths>'
hint: and commit the result with 'git commit'
The command '/bin/sh -c cd text-generation-webui &&     git remote add llava2 https://github.com/haotian-liu/text-generation-webui &&     git fetch llava2 &&     git config user.email "[email protected]" &&     git config user.name "Dustin Franklin" &&     git cherry-pick 9c085ee' returned a non-zero code: 1

I do a workaround (just let it not block me)

You can do this if you want to call python3 download-model.py.

./run.sh --workdir=/opt/text-generation-webui --entrypoint /usr/bin/env $(./autotag text-generation-webui) python3 download-model.py --output=/data/models/text-generation-webui TheBloke/Llama-2-7b-Chat-GPTQ

The entry point should be /usr/bin/env then you can access full ENV vars (PATH for calling python directly and other env vars for pip staff)

Here's the proof

image

Does this look good to you?

@jasl
Copy link
Author

jasl commented Dec 6, 2023

friendly ping @dusty-nv

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants