Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] A dedicated models attribute for Trials #259

Open
eddiebergman opened this issue Feb 16, 2024 · 0 comments
Open

[Feature] A dedicated models attribute for Trials #259

eddiebergman opened this issue Feb 16, 2024 · 0 comments
Labels
feature A new feature

Comments

@eddiebergman
Copy link
Contributor

eddiebergman commented Feb 16, 2024

Is your feature request related to a problem? Please describe.
It's come up a few times that essentially I would like a dedicated interface for "give me the models for this report".

Describe the solution you'd like
Ideally something that you can store a model such as:

trial.store({f"model_{i}.pkl": model})

And retrieve it later from the report as

stored_models: dict[str, Stored[Any]] = report.models
models = {model_name: stored_model.load() for model_name, stored_model in report.models.items()]

The above description isn't enough in trial.store() to know that this item is specifically a model that should be put in report.models. This means we need to do one of two things:

trial.store_model({f"model_{i}.pkl": model})
# Or
trial.store({f"model_{i}.pkl": model}, tag="model")

The second is more flexible as we could essentially provide tagged items in a store but I am afraid that this feature ends up opening up another whole can of worms. As a result, I think the first approach is more scoped.

Describe alternatives you've considered
The current alternative is just having no report.models attribute and manually mapping between. This workaround works absolutely fine but prevents some higher level automations from occuring unless there's a stringly based interface

@eddiebergman eddiebergman added the feature A new feature label Feb 16, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
feature A new feature
Projects
None yet
Development

No branches or pull requests

1 participant