Make metadata easier to use.

[natemcintosh]

• Change formatting of metadata values to be atomic. Test that this is the case. May have to change this at some point, if we ever add non-atomic fields on purpose.
• Add blob_storage_container as a field to the metadata.
• Use empty string for paths when non-existant.

[github-actions]

Thank you for your contribution @natemcintosh 🚀! Your pkgdown-site is ready for download 👉 here 👈!
_{(The artifact expires on 2024-12-20T13:54:02Z. You can re-generate it by re-running the workflow here.)}

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Additional details and impacted files

📢 Thoughts on this report? Let us know!

[athowes]

From the issue:

The metadata also does not store the blob storage container name.

Could my comment above be getting at there being multiple sources of metadata:

1. Executing the model logic
2. Orchestrating the pipeline

I think the blob storage containter is meta data associated with 2. whereas at the moment (this PR) I think it's being put in 1.?

[natemcintosh]

Does anyone know what specifically is causing the CI failure? It looks like a duckdb issue, but I have re-run it multiple times, and it still fails. Usually that flaky duckdb test fixes itself after a re-run.

[zsusswein]

It looks like the flaky test to me (#97). If it's becoming a real problem, I'll try to make time to fix it right after we get Azure CD somewhere stable.

[zsusswein]

I think your proposed change to metadata handling is the right thing to do next, but this should merge more or less as-is. Let's get something onto main and then iterate?

Let's bump the conversation on how to handle passing the model run metadata out to orchestrate_pipeline in an issue?

[athowes]

Agree OK to make new issue for the meta data point. Main thing for me is that I think execute_model_logic should make sense as a function devoid from any Azure interaction.

[natemcintosh]

Added issue #122 for cleaning up the metadata handling.