Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow user to configure compresslevel for tarfile in sagemaker.util.repack_model #4832

Open
lendle opened this issue Aug 14, 2024 · 0 comments

Comments

@lendle
Copy link

lendle commented Aug 14, 2024

Describe the feature you'd like
Allow the user to configure the compresslevel parameter in tarfile.open, used by repack_model.

tarfile, which is used in repack_model uses compresslevel=9 by default, which is very slow. The default for gzip from the command line is 6, and is much faster, at the expense of some compression.

I'm trying to develop a pytorch model for deployment with a custom inference.py. The model artifact is pretty big, so gzipping it at compresslevel 9 takes minutes. I'm trying to iterate on changes to inference.py, and having to wait additional time to compress the tarball is a pain each time I deploy. Unfortunately I don't own the model so it's not easy to create a smaller version for development.

How would this feature be used? Please describe.
Pass an argument to Model.deploy or set compresslevel via a config that is used in repack_model.

Describe alternatives you've considered

  1. Monkey patch tarfile.open to set compression level. Haven't tried this yet.
  2. Manually create the tarball, create a PyTorchModel object pointing to that, and avoid model repackaging somehow. I have tried various ways of creating the PyTorchModel object, pointing it to the model tarball locally or in s3, but it seems to repackage every time anyway.
  3. Maybe use boto directly? New to sagemaker/aws so that's another can of worms.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant