Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tempo Save(...) of objects fails due to storing remote options #194

Open
axsaucedo opened this issue Sep 7, 2021 · 0 comments
Open

Tempo Save(...) of objects fails due to storing remote options #194

axsaucedo opened this issue Sep 7, 2021 · 0 comments

Comments

@axsaucedo
Copy link
Contributor

axsaucedo commented Sep 7, 2021

When tempo.save(...) is run on the object instead of the class, namely when deployed to kubernetes they no longer re-initialise the underlying models with the runtime options override, and this results in kubernetes pipelines trying to reach out to models using the docker hostnames (and hence failing)

So if you have:

@pipeline(
    # ...,
    models=...,
)
class MyClass():
  def predict():
    self.model.predict(....)

Then in this case this works

save(MyClass) # This works

But this no longer works

save(MyClass()) # This only works in docker but not in k8s as child models have docker rt opts
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant