Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Container Request]: dorado but without models included, maybe? #1073

Open
kapsakcj opened this issue Sep 30, 2024 · 2 comments · May be fixed by #1109
Open

[Container Request]: dorado but without models included, maybe? #1073

kapsakcj opened this issue Sep 30, 2024 · 2 comments · May be fixed by #1109
Labels
enhancement New feature or request

Comments

@kapsakcj
Copy link
Collaborator

What tool are you looking for and why?

We just deployed a dorado docker image in PR #1051 , but this docker image contains all available dorado models so the docker image is quite large at ~8.5 GB compressed and ~15.7GB uncompressed.

I don't have a need for this (as of now!), but perhaps someone else does? We could considering building a docker image that does not contain any model files. This would reduce the size of the docker image (though still somewhat large due to the size of the nvidia base image).

The user could then download the appropriate dorado model at runtime, as dorado has the capability to do this.

Is there anyone that would use a model-free docker image for dorado?
@kapsakcj kapsakcj added the enhancement New feature or request label Sep 30, 2024
@dpark01
Copy link

dpark01 commented Sep 30, 2024

So, it's worth noting that docker pull of an 8.5GB compressed image is much slower than docker pull of a 4GB image immediately followed by a download of 4GB of data (ie using normal workflow engine type machinery to localize large files from cloud buckets). This is because docker pull is rate limited by the single-threaded gzip decompression of its layers more so than the network bandwidth of wherever you're downloading from.

@kapsakcj
Copy link
Collaborator Author

good to know, thanks Danny. Even more of a reason to make a model-less dorado docker image 👍

@Kincekara Kincekara linked a pull request Nov 21, 2024 that will close this issue
Draft
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
enhancement New feature or request
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

adds dorado 0.8.3 no-model Kincekara/docker-builds
2 participants
@kapsakcj @dpark01