You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Sometimes, in cases like #675, it is useful to apply a transformation to an entire set of model data before any recipes are applied, such as cutting out a region, or fixing something. CSET does not currently have the capability to do that, as modifying a recipe won't allow it to be done per-model.
Describe the solution you'd like
Add a pre-processing step to the workflow that can optionally run a user defined recipe to prepare the data, per model.
As part of the data time cycling work in #750, I've added the rose config side of a preprocessor system, where transformations can be applied to an entire set of model data before use.
We need to consider how we will define these preprocessors (more recipes?), and how configurable they should be. Do we want YAML recipes, or arbitrary python scripts?
Describe alternatives you've considered
Bundling this into recipes makes too many separate recipes.
Build logic into the recipes, so it can be done conditionally. While powerful this would vastly increase the complexity of the recipes, making them less reliable and harder to reason about.
The text was updated successfully, but these errors were encountered:
What problem does your feature request solve?
Sometimes, in cases like #675, it is useful to apply a transformation to an entire set of model data before any recipes are applied, such as cutting out a region, or fixing something. CSET does not currently have the capability to do that, as modifying a recipe won't allow it to be done per-model.
Describe the solution you'd like
Add a pre-processing step to the workflow that can optionally run a user defined recipe to prepare the data, per model.
As part of the data time cycling work in #750, I've added the rose config side of a preprocessor system, where transformations can be applied to an entire set of model data before use.
We need to consider how we will define these preprocessors (more recipes?), and how configurable they should be. Do we want YAML recipes, or arbitrary python scripts?
Describe alternatives you've considered
Bundling this into recipes makes too many separate recipes.
Build logic into the recipes, so it can be done conditionally. While powerful this would vastly increase the complexity of the recipes, making them less reliable and harder to reason about.
The text was updated successfully, but these errors were encountered: