AiiDA workgraph creation rebased on new IR #45
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
agoscinski
force-pushed
the
workgraph
branch
from
af95c0a to
3b5a990
Compare
agoscinski
force-pushed
the
workgraph
branch
from
a7e50b9 to
f40e4c2
Compare
GeigerJ2
added a commit
that referenced
this pull request
Dec 17, 2024
This PR adds the `_CliArgsBaseModel` class (inspired from `_WhenBaseModel`), which replaces the `command_option` and `input_arg_options` of the `ConfigShellTaskSpecs`. Validation is applied on the correctness of the keyword and positional arguments to ensure they start with `-` or `--`. The three test YAML files are updated accordingly, leading to changes in the pretty-print test text files, which is also part of the PR. Actually making something useful out of these arguments only happens when the WG is created, so will require #45 to be merged.
agoscinski
force-pushed
the
workgraph
branch
from
f40e4c2 to
5718355
Compare
GeigerJ2
added a commit
that referenced
this pull request
Dec 17, 2024
This PR adds the `_CliArgsBaseModel` class (inspired from `_WhenBaseModel`), which replaces the `command_option` and `input_arg_options` of the `ConfigShellTaskSpecs`. Validation is applied on the correctness of the keyword and positional arguments to ensure they start with `-` or `--`. The three test YAML files are updated accordingly, leading to changes in the pretty-print test text files, which is also part of the PR. Actually making something useful out of these arguments only happens when the WG is created, so will require #45 to be merged.
|
OK, @agoscinski, all conflicts should be resolved now. Will branch off from this here, to work on the |
agoscinski
added a commit
that referenced
this pull request
Dec 30, 2024
TODO remove expandvars hack, use by default local folder TODO remove example reference from tests Arbitrary CLI args for shell tasks (#66) This PR adds the `_CliArgsBaseModel` class (inspired from `_WhenBaseModel`), which replaces the `command_option` and `input_arg_options` of the `ConfigShellTaskSpecs`. Validation is applied on the correctness of the keyword and positional arguments to ensure they start with `-` or `--`. The three test YAML files are updated accordingly, leading to changes in the pretty-print test text files, which is also part of the PR. Actually making something useful out of these arguments only happens when the WG is created, so will require #45 to be merged. Fix svgs being created in main directory. (#67) Create `svg` files in subdirectory under `tests/files/svgs` instead of top-level directory, and ignore them via `.gitignore`, while syncing the `svgs` directory. Merge remote-tracking branch 'origin/main' into workgraph --- Pass single argument string to `ShellTasks` (#72) * Try to make current example YAML files run through. * Update class names after rebase/merge * Add `no_icon` config file * First working version apart from flags. * Try fixing argument format. * Current state before branch-off. * Pass one multi-line string as `cli_argument` * Cleanup. * Pass arguments as list. * Remove `workgraph-dev.py` dev file. * Fix issues from `hatch fmt` reimplementing cli_arguments change to unspecified inputs are added as positional cli args automatically delete examples add files required to run the workflow remove old files use rootdir from config move files to confi folder adapt configs to relativ to config folder rm expandvars fix hatch fmt fix hatch fmt update reference rm comment
agoscinski
force-pushed
the
workgraph
branch
from
0cd2ae5 to
6911f6a
Compare
agoscinski
added a commit
that referenced
this pull request
Dec 30, 2024
TODO remove expandvars hack, use by default local folder TODO remove example reference from tests Arbitrary CLI args for shell tasks (#66) This PR adds the `_CliArgsBaseModel` class (inspired from `_WhenBaseModel`), which replaces the `command_option` and `input_arg_options` of the `ConfigShellTaskSpecs`. Validation is applied on the correctness of the keyword and positional arguments to ensure they start with `-` or `--`. The three test YAML files are updated accordingly, leading to changes in the pretty-print test text files, which is also part of the PR. Actually making something useful out of these arguments only happens when the WG is created, so will require #45 to be merged. Fix svgs being created in main directory. (#67) Create `svg` files in subdirectory under `tests/files/svgs` instead of top-level directory, and ignore them via `.gitignore`, while syncing the `svgs` directory. Merge remote-tracking branch 'origin/main' into workgraph --- Pass single argument string to `ShellTasks` (#72) * Try to make current example YAML files run through. * Update class names after rebase/merge * Add `no_icon` config file * First working version apart from flags. * Try fixing argument format. * Current state before branch-off. * Pass one multi-line string as `cli_argument` * Cleanup. * Pass arguments as list. * Remove `workgraph-dev.py` dev file. * Fix issues from `hatch fmt` reimplementing cli_arguments change to unspecified inputs are added as positional cli args automatically delete examples add files required to run the workflow remove old files use rootdir from config move files to confi folder adapt configs to relativ to config folder rm expandvars fix hatch fmt fix hatch fmt update reference rm comment
agoscinski
force-pushed
the
workgraph
branch
from
6911f6a to
1b47ce1
Compare
agoscinski
added a commit
that referenced
this pull request
Dec 31, 2024
TODO remove expandvars hack, use by default local folder TODO remove example reference from tests Arbitrary CLI args for shell tasks (#66) This PR adds the `_CliArgsBaseModel` class (inspired from `_WhenBaseModel`), which replaces the `command_option` and `input_arg_options` of the `ConfigShellTaskSpecs`. Validation is applied on the correctness of the keyword and positional arguments to ensure they start with `-` or `--`. The three test YAML files are updated accordingly, leading to changes in the pretty-print test text files, which is also part of the PR. Actually making something useful out of these arguments only happens when the WG is created, so will require #45 to be merged. Fix svgs being created in main directory. (#67) Create `svg` files in subdirectory under `tests/files/svgs` instead of top-level directory, and ignore them via `.gitignore`, while syncing the `svgs` directory. Merge remote-tracking branch 'origin/main' into workgraph --- Pass single argument string to `ShellTasks` (#72) * Try to make current example YAML files run through. * Update class names after rebase/merge * Add `no_icon` config file * First working version apart from flags. * Try fixing argument format. * Current state before branch-off. * Pass one multi-line string as `cli_argument` * Cleanup. * Pass arguments as list. * Remove `workgraph-dev.py` dev file. * Fix issues from `hatch fmt` reimplementing cli_arguments change to unspecified inputs are added as positional cli args automatically delete examples add files required to run the workflow remove old files use rootdir from config move files to confi folder adapt configs to relativ to config folder rm expandvars fix hatch fmt fix hatch fmt update reference rm comment implement wait_on clean up code clean up add doc
agoscinski
force-pushed
the
workgraph
branch
from
1b47ce1 to
34ee944
Compare
agoscinski
added a commit
that referenced
this pull request
Jan 1, 2025
TODO remove expandvars hack, use by default local folder TODO remove example reference from tests Arbitrary CLI args for shell tasks (#66) This PR adds the `_CliArgsBaseModel` class (inspired from `_WhenBaseModel`), which replaces the `command_option` and `input_arg_options` of the `ConfigShellTaskSpecs`. Validation is applied on the correctness of the keyword and positional arguments to ensure they start with `-` or `--`. The three test YAML files are updated accordingly, leading to changes in the pretty-print test text files, which is also part of the PR. Actually making something useful out of these arguments only happens when the WG is created, so will require #45 to be merged. Fix svgs being created in main directory. (#67) Create `svg` files in subdirectory under `tests/files/svgs` instead of top-level directory, and ignore them via `.gitignore`, while syncing the `svgs` directory. Merge remote-tracking branch 'origin/main' into workgraph --- Pass single argument string to `ShellTasks` (#72) * Try to make current example YAML files run through. * Update class names after rebase/merge * Add `no_icon` config file * First working version apart from flags. * Try fixing argument format. * Current state before branch-off. * Pass one multi-line string as `cli_argument` * Cleanup. * Pass arguments as list. * Remove `workgraph-dev.py` dev file. * Fix issues from `hatch fmt` reimplementing cli_arguments change to unspecified inputs are added as positional cli args automatically delete examples add files required to run the workflow remove old files use rootdir from config move files to confi folder adapt configs to relativ to config folder rm expandvars fix hatch fmt fix hatch fmt update reference rm comment implement wait_on clean up code clean up add doc cli parameters update tests test work fix everything
Initialization of all files required to run workflows. The workflow config files are updated to work properly. The $PWD environment is removed because the config_rootdir is now used. To allow a graph like initialization of workgraph tasks first creating the task nodes and then linking the additional information including inpouts outputs and arguments we implemented a temporary hack to overcome internal validations.
agoscinski
commented
Jan 1, 2025
| cli_arguments: "{--restart icon_restart} {--init initial_conditions}" | ||
| env_source_files: data/dummy_source_file.sh |
There was a problem hiding this comment.
Has been added to test this in the workgraph as we do not run the large config where this is also used
| [ | ||
| "tests/cases/small/config/test_config_small.yml", | ||
| "tests/cases/parameters/config/test_config_parameters.yml", | ||
| ], |
There was a problem hiding this comment.
the large config takes 4 minutes to run therefore it is not included in the tests
| @@ -0,0 +1 @@ | |||
| pytest_plugins = ["aiida.tools.pytest_fixtures"] | |||
There was a problem hiding this comment.
This autoloads a temporary aiida profile for the tests. I find pytest is doing here a bit too much in the background. One needs to look into the auto fixtures from aiida to see what is all created. Just keep in mind that the tests are not run with your current profile but create a temporary one.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Please merge first #75
Implementation of running Shell Task with workgraph.
Initialization of all files required to run workflows. The workflow config files are updated to work properly. The $PWD environment is removed because the config_rootdir is now used.
To allow a graph like initialization of workgraph tasks first creating the task nodes and then linking the additional information including inpouts outputs and arguments we implemented a temporary hack to overcome internal validations.
TODO: