From 240010c281a216e7ae2ab90049758aa62693e6f2 Mon Sep 17 00:00:00 2001 From: Hendrik Cannoodt Date: Wed, 16 Aug 2023 08:58:21 +0200 Subject: [PATCH] Prepare 0 7 5 release (#54) * rework scripts such that it uses the jinja package instead of jinja cli * refactor script * Prepare for upcoming changes in the config schema change command to generate the cli and config schemas * Remove old intermediate .yaml files Remove `.qmd` from the file names in the `config_pages_settings.yaml`. It's now added in the script. * Tweak empty lines in config pages jinja template * Add environmentVariables to the config_pages_settings.yaml * Update a bunch of automatically generated file changes Mostly adding the default value, some type string improvements or clearing an extra empty line Adds the new test --setup command documentation too config.info, functionality.argumentGroup, environment variables are documented * Use flat list of config schema data instead of map of map Actually now a list of list instead of map of map of list Get title information from the `__this__` parameter Add some minor exceptions in the title that was previously handled by manually defining map keys * Prepare & verify removal of removed parameter documentation Also assuming any mention of 'VDSL3' is supposed to be removed. * Copy static reference pages from viash - Remove all subfolders in reference - Copy static pages - Rebuild cli & config pages * Don't display command arguments if they are hidden We're not (yet?) telling the --colorize and --loglevel options to the user so don't display it in the documentation * refactor guide, add vdsl3 reference docs * Update the pages for the Viash 0.7.5 release. * Add vdsl3 references (#61) * fixes to vdsl3 references * fix link --------- Co-authored-by: Robrecht Cannoodt --- _src/automation/config_pages_settings.yaml | 100 +- .../generate_reference_cli_pages/script.py | 42 +- .../template_cli_page.j2.qmd | 2 +- .../generate_reference_config_pages/script.py | 71 +- .../template_config_page.j2.qmd | 13 +- .../generate_version_blog_pages/script.py | 36 +- _src/render_pages.sh | 10 +- _viash.yaml | 2 +- blog/posts/viash-0.0.1/_index.yaml | 9 - blog/posts/viash-0.1.0/_index.yaml | 64 - blog/posts/viash-0.2.0/_index.yaml | 62 - blog/posts/viash-0.2.1/_index.yaml | 61 - blog/posts/viash-0.2.2/_index.yaml | 24 - blog/posts/viash-0.3.0/_index.yaml | 39 - blog/posts/viash-0.3.1/_index.yaml | 46 - blog/posts/viash-0.3.2/_index.yaml | 22 - blog/posts/viash-0.4.0.1/_index.yaml | 21 - blog/posts/viash-0.4.0/_index.yaml | 51 - blog/posts/viash-0.5.0/_index.yaml | 64 - blog/posts/viash-0.5.1/_index.yaml | 76 - blog/posts/viash-0.5.10.1/_index.yaml | 17 - blog/posts/viash-0.5.10/_index.yaml | 36 - blog/posts/viash-0.5.11/_index.yaml | 38 - blog/posts/viash-0.5.12/_index.yaml | 34 - blog/posts/viash-0.5.13/_index.yaml | 26 - blog/posts/viash-0.5.14/_index.yaml | 55 - blog/posts/viash-0.5.15/_index.yaml | 45 - blog/posts/viash-0.5.2/_index.yaml | 20 - blog/posts/viash-0.5.3/_index.yaml | 29 - blog/posts/viash-0.5.4/_index.yaml | 13 - blog/posts/viash-0.5.5/_index.yaml | 17 - blog/posts/viash-0.5.6/_index.yaml | 26 - blog/posts/viash-0.5.7/_index.yaml | 24 - blog/posts/viash-0.5.8/_index.yaml | 32 - blog/posts/viash-0.5.9/_index.yaml | 24 - blog/posts/viash-0.6.0/_index.yaml | 102 - blog/posts/viash-0.6.1/_index.yaml | 56 - blog/posts/viash-0.6.2/_index.yaml | 20 - blog/posts/viash-0.6.3/_index.yaml | 74 - blog/posts/viash-0.6.4/_index.yaml | 75 - blog/posts/viash-0.6.5/_index.yaml | 13 - blog/posts/viash-0.6.6/_index.yaml | 16 - blog/posts/viash-0.6.7/_index.yaml | 22 - blog/posts/viash-0.7.0/_index.yaml | 78 - blog/posts/viash-0.7.1/_index.yaml | 19 - blog/posts/viash-0.7.2/_index.yaml | 28 - blog/posts/viash-0.7.3/_index.yaml | 23 - blog/posts/viash-0.7.4/_index.yaml | 46 - blog/posts/viash-0.7.5/index.qmd | 96 + guide/nextflow_vdsl3/create-a-pipeline.qmd | 55 +- ...module.qmd => create-and-use-a-module.qmd} | 117 +- guide/nextflow_vdsl3/index.qmd | 22 +- guide/nextflow_vdsl3/introduction.qmd | 11 - index.qmd | 2 +- quickstart/index.qmd | 2 +- reference/cli/_build.yaml | 65 - reference/cli/_config_inject.yaml | 37 - reference/cli/_config_view.yaml | 57 - reference/cli/_ns_build.yaml | 108 - reference/cli/_ns_exec.yaml | 114 - reference/cli/_ns_list.yaml | 93 - reference/cli/_ns_test.yaml | 118 - reference/cli/_run.yaml | 72 - reference/cli/_test.yaml | 67 - reference/cli/build.qmd | 1 - reference/cli/config_inject.qmd | 1 - reference/cli/config_view.qmd | 1 - reference/cli/ns_build.qmd | 1 - reference/cli/ns_exec.qmd | 1 - reference/cli/ns_list.qmd | 1 - reference/cli/ns_test.qmd | 4 +- reference/cli/run.qmd | 1 - reference/cli/test.qmd | 4 +- reference/cli_schema_export.json | 293 +- reference/config/_index.yaml | 45 - reference/config/environmentVariables.qmd | 31 + reference/config/functionality/_author.yaml | 46 - .../_computationalRequirements.yaml | 36 - reference/config/functionality/_index.yaml | 232 - .../config/functionality/argumentGroup.qmd | 26 + .../functionality/arguments/_boolean.yaml | 83 - .../arguments/_boolean_false.yaml | 41 - .../arguments/_boolean_true.yaml | 41 - .../functionality/arguments/_double.yaml | 98 - .../config/functionality/arguments/_file.yaml | 107 - .../functionality/arguments/_index.yaml | 20 - .../functionality/arguments/_integer.yaml | 107 - .../config/functionality/arguments/_long.yaml | 107 - .../functionality/arguments/_string.yaml | 94 - .../functionality/arguments/boolean.qmd | 17 +- .../functionality/arguments/boolean_false.qmd | 9 +- .../functionality/arguments/boolean_true.qmd | 9 +- .../config/functionality/arguments/double.qmd | 21 +- .../config/functionality/arguments/file.qmd | 23 +- .../config/functionality/arguments/index.qmd | 1 - .../functionality/arguments/integer.qmd | 23 +- .../config/functionality/arguments/long.qmd | 23 +- .../config/functionality/arguments/string.qmd | 19 +- reference/config/functionality/author.qmd | 11 +- .../computationalRequirements.qmd | 16 +- reference/config/functionality/index.qmd | 119 +- .../functionality/resources/_bashScript.yaml | 35 - .../resources/_cSharpScript.yaml | 35 - .../functionality/resources/_executable.yaml | 29 - .../functionality/resources/_index.yaml | 26 - .../resources/_javaScriptScript.yaml | 35 - .../resources/_nextflowScript.yaml | 33 - .../functionality/resources/_plainFile.yaml | 29 - .../resources/_pythonScript.yaml | 35 - .../functionality/resources/_rScript.yaml | 35 - .../functionality/resources/_scalaScript.yaml | 35 - .../functionality/resources/bashScript.qmd | 9 +- .../functionality/resources/cSharpScript.qmd | 11 +- .../functionality/resources/executable.qmd | 9 +- .../config/functionality/resources/index.qmd | 1 - .../resources/javaScriptScript.qmd | 11 +- .../resources/nextflowScript.qmd | 11 +- .../functionality/resources/plainFile.qmd | 9 +- .../functionality/resources/pythonScript.qmd | 9 +- .../functionality/resources/rScript.qmd | 9 +- .../functionality/resources/scalaScript.qmd | 9 +- reference/config/index.qmd | 5 +- reference/config/info.qmd | 69 + reference/config/platforms/_index.yaml | 14 - reference/config/platforms/docker/_index.yaml | 306 - reference/config/platforms/docker/index.qmd | 192 +- .../docker/setup/_apkRequirements.yaml | 22 - .../docker/setup/_aptRequirements.yaml | 26 - .../docker/setup/_dockerRequirements.yaml | 70 - .../config/platforms/docker/setup/_index.yaml | 14 - .../docker/setup/_javascriptRequirements.yaml | 48 - .../docker/setup/_pythonRequirements.yaml | 91 - .../docker/setup/_rRequirements.yaml | 84 - .../docker/setup/_rubyRequirements.yaml | 22 - .../docker/setup/_yumRequirements.yaml | 22 - .../docker/setup/apkRequirements.qmd | 3 +- .../docker/setup/aptRequirements.qmd | 5 +- .../docker/setup/dockerRequirements.qmd | 29 +- .../config/platforms/docker/setup/index.qmd | 1 - .../docker/setup/javascriptRequirements.qmd | 13 +- .../docker/setup/pythonRequirements.qmd | 27 +- .../platforms/docker/setup/rRequirements.qmd | 23 +- .../docker/setup/rubyRequirements.qmd | 3 +- .../docker/setup/yumRequirements.qmd | 3 +- reference/config/platforms/index.qmd | 3 +- reference/config/platforms/native/_index.yaml | 31 - reference/config/platforms/native/index.qmd | 3 +- .../config/platforms/nextflow/_auto.yaml | 51 - .../config/platforms/nextflow/_config.yaml | 65 - .../platforms/nextflow/_directives.yaml | 544 -- .../config/platforms/nextflow/_index.yaml | 76 - reference/config/platforms/nextflow/auto.qmd | 9 +- .../config/platforms/nextflow/config.qmd | 7 +- .../config/platforms/nextflow/directives.qmd | 101 +- reference/config/platforms/nextflow/index.qmd | 21 +- .../platforms/nextflowLegacy/_index.yaml | 208 - .../config/platforms/nextflowLegacy/index.qmd | 251 - reference/config_schema_export.json | 7516 +++++++++-------- reference/nextflow_vdsl3/import_module.qmd | 107 + reference/nextflow_vdsl3/index.qmd | 10 + reference/nextflow_vdsl3/run_module.qmd | 61 + reference/project/_index.yaml | 40 - reference/project/index.qmd | 9 +- reference/reference.yml | 4 +- 164 files changed, 5445 insertions(+), 9751 deletions(-) delete mode 100644 blog/posts/viash-0.0.1/_index.yaml delete mode 100644 blog/posts/viash-0.1.0/_index.yaml delete mode 100644 blog/posts/viash-0.2.0/_index.yaml delete mode 100644 blog/posts/viash-0.2.1/_index.yaml delete mode 100644 blog/posts/viash-0.2.2/_index.yaml delete mode 100644 blog/posts/viash-0.3.0/_index.yaml delete mode 100644 blog/posts/viash-0.3.1/_index.yaml delete mode 100644 blog/posts/viash-0.3.2/_index.yaml delete mode 100644 blog/posts/viash-0.4.0.1/_index.yaml delete mode 100644 blog/posts/viash-0.4.0/_index.yaml delete mode 100644 blog/posts/viash-0.5.0/_index.yaml delete mode 100644 blog/posts/viash-0.5.1/_index.yaml delete mode 100644 blog/posts/viash-0.5.10.1/_index.yaml delete mode 100644 blog/posts/viash-0.5.10/_index.yaml delete mode 100644 blog/posts/viash-0.5.11/_index.yaml delete mode 100644 blog/posts/viash-0.5.12/_index.yaml delete mode 100644 blog/posts/viash-0.5.13/_index.yaml delete mode 100644 blog/posts/viash-0.5.14/_index.yaml delete mode 100644 blog/posts/viash-0.5.15/_index.yaml delete mode 100644 blog/posts/viash-0.5.2/_index.yaml delete mode 100644 blog/posts/viash-0.5.3/_index.yaml delete mode 100644 blog/posts/viash-0.5.4/_index.yaml delete mode 100644 blog/posts/viash-0.5.5/_index.yaml delete mode 100644 blog/posts/viash-0.5.6/_index.yaml delete mode 100644 blog/posts/viash-0.5.7/_index.yaml delete mode 100644 blog/posts/viash-0.5.8/_index.yaml delete mode 100644 blog/posts/viash-0.5.9/_index.yaml delete mode 100644 blog/posts/viash-0.6.0/_index.yaml delete mode 100644 blog/posts/viash-0.6.1/_index.yaml delete mode 100644 blog/posts/viash-0.6.2/_index.yaml delete mode 100644 blog/posts/viash-0.6.3/_index.yaml delete mode 100644 blog/posts/viash-0.6.4/_index.yaml delete mode 100644 blog/posts/viash-0.6.5/_index.yaml delete mode 100644 blog/posts/viash-0.6.6/_index.yaml delete mode 100644 blog/posts/viash-0.6.7/_index.yaml delete mode 100644 blog/posts/viash-0.7.0/_index.yaml delete mode 100644 blog/posts/viash-0.7.1/_index.yaml delete mode 100644 blog/posts/viash-0.7.2/_index.yaml delete mode 100644 blog/posts/viash-0.7.3/_index.yaml delete mode 100644 blog/posts/viash-0.7.4/_index.yaml create mode 100644 blog/posts/viash-0.7.5/index.qmd rename guide/nextflow_vdsl3/{create-a-module.qmd => create-and-use-a-module.qmd} (65%) delete mode 100644 guide/nextflow_vdsl3/introduction.qmd delete mode 100644 reference/cli/_build.yaml delete mode 100644 reference/cli/_config_inject.yaml delete mode 100644 reference/cli/_config_view.yaml delete mode 100644 reference/cli/_ns_build.yaml delete mode 100644 reference/cli/_ns_exec.yaml delete mode 100644 reference/cli/_ns_list.yaml delete mode 100644 reference/cli/_ns_test.yaml delete mode 100644 reference/cli/_run.yaml delete mode 100644 reference/cli/_test.yaml delete mode 100644 reference/config/_index.yaml create mode 100644 reference/config/environmentVariables.qmd delete mode 100644 reference/config/functionality/_author.yaml delete mode 100644 reference/config/functionality/_computationalRequirements.yaml delete mode 100644 reference/config/functionality/_index.yaml create mode 100644 reference/config/functionality/argumentGroup.qmd delete mode 100644 reference/config/functionality/arguments/_boolean.yaml delete mode 100644 reference/config/functionality/arguments/_boolean_false.yaml delete mode 100644 reference/config/functionality/arguments/_boolean_true.yaml delete mode 100644 reference/config/functionality/arguments/_double.yaml delete mode 100644 reference/config/functionality/arguments/_file.yaml delete mode 100644 reference/config/functionality/arguments/_index.yaml delete mode 100644 reference/config/functionality/arguments/_integer.yaml delete mode 100644 reference/config/functionality/arguments/_long.yaml delete mode 100644 reference/config/functionality/arguments/_string.yaml delete mode 100644 reference/config/functionality/resources/_bashScript.yaml delete mode 100644 reference/config/functionality/resources/_cSharpScript.yaml delete mode 100644 reference/config/functionality/resources/_executable.yaml delete mode 100644 reference/config/functionality/resources/_index.yaml delete mode 100644 reference/config/functionality/resources/_javaScriptScript.yaml delete mode 100644 reference/config/functionality/resources/_nextflowScript.yaml delete mode 100644 reference/config/functionality/resources/_plainFile.yaml delete mode 100644 reference/config/functionality/resources/_pythonScript.yaml delete mode 100644 reference/config/functionality/resources/_rScript.yaml delete mode 100644 reference/config/functionality/resources/_scalaScript.yaml create mode 100644 reference/config/info.qmd delete mode 100644 reference/config/platforms/_index.yaml delete mode 100644 reference/config/platforms/docker/_index.yaml delete mode 100644 reference/config/platforms/docker/setup/_apkRequirements.yaml delete mode 100644 reference/config/platforms/docker/setup/_aptRequirements.yaml delete mode 100644 reference/config/platforms/docker/setup/_dockerRequirements.yaml delete mode 100644 reference/config/platforms/docker/setup/_index.yaml delete mode 100644 reference/config/platforms/docker/setup/_javascriptRequirements.yaml delete mode 100644 reference/config/platforms/docker/setup/_pythonRequirements.yaml delete mode 100644 reference/config/platforms/docker/setup/_rRequirements.yaml delete mode 100644 reference/config/platforms/docker/setup/_rubyRequirements.yaml delete mode 100644 reference/config/platforms/docker/setup/_yumRequirements.yaml delete mode 100644 reference/config/platforms/native/_index.yaml delete mode 100644 reference/config/platforms/nextflow/_auto.yaml delete mode 100644 reference/config/platforms/nextflow/_config.yaml delete mode 100644 reference/config/platforms/nextflow/_directives.yaml delete mode 100644 reference/config/platforms/nextflow/_index.yaml delete mode 100644 reference/config/platforms/nextflowLegacy/_index.yaml delete mode 100644 reference/config/platforms/nextflowLegacy/index.qmd create mode 100644 reference/nextflow_vdsl3/import_module.qmd create mode 100644 reference/nextflow_vdsl3/index.qmd create mode 100644 reference/nextflow_vdsl3/run_module.qmd delete mode 100644 reference/project/_index.yaml diff --git a/_src/automation/config_pages_settings.yaml b/_src/automation/config_pages_settings.yaml index a27665e7..28e66956 100644 --- a/_src/automation/config_pages_settings.yaml +++ b/_src/automation/config_pages_settings.yaml @@ -1,63 +1,65 @@ structure: - 'config/config': ./index.qmd - 'config/project': ../project/index.qmd + 'Config': ./index + 'Project': ../project/index + 'Info': ./info + 'EnvironmentVariables': ./environmentVariables - 'functionality/functionality': ./functionality/index.qmd - 'functionality/author': ./functionality/author.qmd - 'functionality/computationalRequirements': ./functionality/computationalRequirements.qmd + 'Functionality': ./functionality/index + 'Author': ./functionality/author + 'ComputationalRequirements': ./functionality/computationalRequirements + 'ArgumentGroup': ./functionality/argumentGroup - 'platforms/platform': ./platforms/index.qmd - 'platforms/nativePlatform': ./platforms/native/index.qmd - 'platforms/dockerPlatform': ./platforms/docker/index.qmd - 'platforms/nextflowVdsl3Platform': ./platforms/nextflow/index.qmd - 'platforms/nextflowLegacyPlatform': ./platforms/nextflowLegacy/index.qmd + 'Platform': ./platforms/index + 'NativePlatform': ./platforms/native/index + 'DockerPlatform': ./platforms/docker/index + 'NextflowPlatform': ./platforms/nextflow/index - 'arguments/argument': ./functionality/arguments/index.qmd - 'arguments/boolean_false': ./functionality/arguments/boolean_false.qmd - 'arguments/boolean': ./functionality/arguments/boolean.qmd - 'arguments/boolean_true': ./functionality/arguments/boolean_true.qmd - 'arguments/double': ./functionality/arguments/double.qmd - 'arguments/file': ./functionality/arguments/file.qmd - 'arguments/integer': ./functionality/arguments/integer.qmd - 'arguments/long': ./functionality/arguments/long.qmd - 'arguments/string': ./functionality/arguments/string.qmd + 'Argument': ./functionality/arguments/index + 'BooleanFalseArgument': ./functionality/arguments/boolean_false + 'BooleanArgument': ./functionality/arguments/boolean + 'BooleanTrueArgument': ./functionality/arguments/boolean_true + 'DoubleArgument': ./functionality/arguments/double + 'FileArgument': ./functionality/arguments/file + 'IntegerArgument': ./functionality/arguments/integer + 'LongArgument': ./functionality/arguments/long + 'StringArgument': ./functionality/arguments/string - 'requirements/requirements': ./platforms/docker/setup/index.qmd - 'requirements/apkRequirements': ./platforms/docker/setup/apkRequirements.qmd - 'requirements/aptRequirements': ./platforms/docker/setup/aptRequirements.qmd - 'requirements/dockerRequirements': ./platforms/docker/setup/dockerRequirements.qmd - 'requirements/javascriptRequirements': ./platforms/docker/setup/javascriptRequirements.qmd - 'requirements/pythonRequirements': ./platforms/docker/setup/pythonRequirements.qmd - 'requirements/rRequirements': ./platforms/docker/setup/rRequirements.qmd - 'requirements/rubyRequirements': ./platforms/docker/setup/rubyRequirements.qmd - 'requirements/yumRequirements': ./platforms/docker/setup/yumRequirements.qmd + 'Requirements': ./platforms/docker/setup/index + 'ApkRequirements': ./platforms/docker/setup/apkRequirements + 'AptRequirements': ./platforms/docker/setup/aptRequirements + 'DockerRequirements': ./platforms/docker/setup/dockerRequirements + 'JavaScriptRequirements': ./platforms/docker/setup/javascriptRequirements + 'PythonRequirements': ./platforms/docker/setup/pythonRequirements + 'RRequirements': ./platforms/docker/setup/rRequirements + 'RubyRequirements': ./platforms/docker/setup/rubyRequirements + 'YumRequirements': ./platforms/docker/setup/yumRequirements - 'resources/resource': ./functionality/resources/index.qmd - 'resources/cSharpScript': ./functionality/resources/cSharpScript.qmd - 'resources/pythonScript': ./functionality/resources/pythonScript.qmd - 'resources/plainFile': ./functionality/resources/plainFile.qmd - 'resources/bashScript': ./functionality/resources/bashScript.qmd - 'resources/scalaScript': ./functionality/resources/scalaScript.qmd - 'resources/nextflowScript': ./functionality/resources/nextflowScript.qmd - 'resources/rScript': ./functionality/resources/rScript.qmd - 'resources/executable': ./functionality/resources/executable.qmd - 'resources/javaScriptScript': ./functionality/resources/javaScriptScript.qmd + 'Resource': ./functionality/resources/index + 'CSharpScript': ./functionality/resources/cSharpScript + 'PythonScript': ./functionality/resources/pythonScript + 'PlainFile': ./functionality/resources/plainFile + 'BashScript': ./functionality/resources/bashScript + 'ScalaScript': ./functionality/resources/scalaScript + 'NextflowScript': ./functionality/resources/nextflowScript + 'RScript': ./functionality/resources/rScript + 'Executable': ./functionality/resources/executable + 'JavaScriptScript': ./functionality/resources/javaScriptScript - 'nextflowParameters/nextflowDirectives': ./platforms/nextflow/directives.qmd - 'nextflowParameters/nextflowAuto': ./platforms/nextflow/auto.qmd - 'nextflowParameters/nextflowConfig': ./platforms/nextflow/config.qmd + 'NextflowDirectives': ./platforms/nextflow/directives + 'NextflowAuto': ./platforms/nextflow/auto + 'NextflowConfig': ./platforms/nextflow/config order: - 'config/config': 20 - 'config/project': 30 + 'Config': 20 + 'Project': 30 - 'functionality/functionality': 10 - 'platforms/platform': 20 + 'Functionality': 10 + 'Platform': 20 + 'Info': 30 - 'platforms/nativePlatform': 10 - 'platforms/dockerPlatform': 20 - 'platforms/nextflowVdsl3Platform': 30 - 'platforms/nextflowLegacyPlatform': 40 + 'NativePlatform': 10 + 'DockerPlatform': 20 + 'NextflowPlatform': 30 keywords: mykeyword: /reference/config.html diff --git a/_src/automation/generate_reference_cli_pages/script.py b/_src/automation/generate_reference_cli_pages/script.py index 7e192223..2b584e93 100644 --- a/_src/automation/generate_reference_cli_pages/script.py +++ b/_src/automation/generate_reference_cli_pages/script.py @@ -1,23 +1,29 @@ +import json, re, yaml +from pathlib import Path +import jinja2 ## VIASH START -# The following code has been auto-generated by Viash. par = { - 'input': r'/path/to/file', - 'output': r'reference/cli' + 'input': 'reference/cli_schema_export.json', + 'output': 'reference/cli' +} +meta = { + "resources_dir": "_src/automation/generate_reference_cli_pages" } ## VIASH END -import subprocess, json, re, yaml -from pathlib import Path +output = Path(par["output"]) keyword_regex = r"\@\[(.*?)\]\((.*?)\)" template_file = Path(meta['resources_dir'], "template_cli_page.j2.qmd") config_file = Path(meta['resources_dir'], 'config_pages_settings.yaml') +with open(template_file) as f: + template = jinja2.Template(f.read()) + with open(config_file, 'r') as infile: config_pages_settings = yaml.safe_load(infile) - def generate_pages(): """ Load the generated JSON file and creates pages for every command entry. """ @@ -26,7 +32,7 @@ def generate_pages(): for entry in viash_json: # create_page(entry["name"], entry) - if "bannerCommand" in entry: + if "bannerCommand" in entry: create_page(entry["name"], entry) else: for subcommand in entry['subcommands']: @@ -47,22 +53,14 @@ def create_page(name, json_data): if 'descr' in arg: arg['descr'] = replace_keywords(arg["descr"]) - render_jinja_page(par['output'], name.replace(' ', '_'), page_data) - -def render_jinja_page(folder: str, filename: str, data: dict): - """ Write data to yaml file and run jinja. """ - - full_path = Path(folder, filename) - base_dir = full_path.parent - yaml_file = Path(base_dir, "_" + full_path.name).with_suffix('.yaml') - qmd_file = full_path.with_suffix('.qmd') - - base_dir.mkdir(parents=True, exist_ok=True) - - with open(yaml_file, 'w') as outfile: - yaml.safe_dump(data, outfile, default_flow_style=False) + qmd_file = output / f"{name.replace(' ', '_')}.qmd" + render_jinja_page(qmd_file, page_data) - subprocess.run(["jinja2", template_file, yaml_file, "-o", qmd_file]) +def render_jinja_page(path: Path, data: dict): + """Run jinja on data and write results to file.""" + path.parent.mkdir(parents=True, exist_ok=True) + content = template.render(**data) + path.write_text(content) def replace_keywords(text: str) -> str: """ Finds all keywords in the format @[keyword](text) and returns the replacements based on the keywords settings. """ diff --git a/_src/automation/generate_reference_cli_pages/template_cli_page.j2.qmd b/_src/automation/generate_reference_cli_pages/template_cli_page.j2.qmd index 9a8d8b66..61f42308 100644 --- a/_src/automation/generate_reference_cli_pages/template_cli_page.j2.qmd +++ b/_src/automation/generate_reference_cli_pages/template_cli_page.j2.qmd @@ -17,7 +17,7 @@ search: true | Argument | Description | Type | |-|:----|-: -{% for arg in d.opts|sort(attribute="name") -%} +{% for arg in d.opts|sort(attribute="name") if not arg.hidden -%} {% if arg.name == "config" %}| `{{arg.name}}`{%- else -%}| `--{{arg.name}}`{% endif -%} {%- if arg.short is defined %}, `-{{arg.short}}`{% endif %} | {{arg.descr|replace("$", "\\$")|replace("\n", " ")}} diff --git a/_src/automation/generate_reference_config_pages/script.py b/_src/automation/generate_reference_config_pages/script.py index 10f42be9..074c1072 100644 --- a/_src/automation/generate_reference_config_pages/script.py +++ b/_src/automation/generate_reference_config_pages/script.py @@ -1,54 +1,62 @@ +import json, re, yaml +from pathlib import Path +import jinja2 ## VIASH START -# The following code has been auto-generated by Viash. par = { - 'input': r'/path/to/file', - 'output': r'reference/config' + 'input': 'reference/config_schema_export.json', + 'output': 'reference/cli' +} +meta = { + "resources_dir": "_src/automation/generate_reference_config_pages" } ## VIASH END -import subprocess, json, re, yaml -from pathlib import Path +output = Path(par["output"]) keyword_regex = r"\@\[(.*?)\]\((.*?)\)" template_file = Path(meta['resources_dir'], "template_config_page.j2.qmd") config_file = Path(meta['resources_dir'], 'config_pages_settings.yaml') +with open(template_file) as f: + template = jinja2.Template(f.read()) + with open(config_file, 'r') as infile: config_pages_settings = yaml.safe_load(infile) - def read_json_entries(): """ Load the generated JSON file, split into logical page chunks and generate pages. """ with open(par['input'], 'r') as infile: viash_json = json.load(infile) - for topic, topic_json in viash_json.items(): - if isinstance(topic_json, dict): - for subtopic, subtopic_json in topic_json.items(): - generate_page(topic, subtopic, subtopic_json) - else: - generate_page(".", topic, topic_json) + for topic_json in viash_json: + generate_page(topic_json) -def generate_page(topic: str, subtopic: str, json_data: list): +def generate_page(json_data: list): """ Receives JSON data, does some minor data manipulation and writes to yaml & qmd. """ - # if topic == 'arguments': # Keep title of argument pages as-is - # title = subtopic - # else: - title = re.sub(r"(\w)([A-Z])", r"\1 \2", subtopic).title() # split words and capitalize - + this_parameter = {} # Fix description markdown keywords to links for d in json_data: if d['name'] == '__merge__': d['name'] = '`__merge__' # Bump __merge__ behind __this__ when sorted + if d['name'] == '__this__': + this_parameter = d if 'description' in d: d['description'] = replace_keywords(d["description"]) - page_data = {"topic": topic, "title": title, "data": json_data} + topic = this_parameter['type'] - filename = f"{topic}/{subtopic}" + # split words and capitalize + # do some extra substitutions to clean things up a bit + title = re.sub(r"(\w)([A-Z])", r"\1 \2", topic).title() \ + .replace("Java Script", "JavaScript") \ + .replace("C Sharp", "C#") \ + .replace(" Argument", "") + filename = topic + + page_data = {"title": title, "data": json_data} try: page_data['order'] = config_pages_settings['order'][filename] @@ -59,23 +67,16 @@ def generate_page(topic: str, subtopic: str, json_data: list): filename = config_pages_settings['structure'][filename] except KeyError: print(f"Could not find {filename} in the config pages settings structure") - - render_jinja_page(par['output'], filename, page_data) - -def render_jinja_page(folder: str, filename: str, data: dict): - """ Write data to yaml file and run jinja. """ - - full_path = Path(folder, filename) - base_dir = full_path.parent - yaml_file = Path(base_dir, "_" + full_path.name).with_suffix('.yaml') - qmd_file = full_path.with_suffix('.qmd') - base_dir.mkdir(parents=True, exist_ok=True) + qmd_file = output / (filename + ".qmd") + + render_jinja_page(qmd_file, page_data) - with open(yaml_file, 'w') as outfile: - yaml.safe_dump(data, outfile, default_flow_style=False) - - subprocess.run(["jinja2", template_file, yaml_file, "-o", qmd_file]) +def render_jinja_page(path: Path, data: dict): + """Run jinja on data and write results to file.""" + path.parent.mkdir(parents=True, exist_ok=True) + content = template.render(**data) + path.write_text(content) def replace_keywords(text: str) -> str: """ Finds all keywords in the format @[keyword](text) and returns the replacements based on the keywords settings. """ diff --git a/_src/automation/generate_reference_config_pages/template_config_page.j2.qmd b/_src/automation/generate_reference_config_pages/template_config_page.j2.qmd index ba7f171d..e2b0ab5a 100644 --- a/_src/automation/generate_reference_config_pages/template_config_page.j2.qmd +++ b/_src/automation/generate_reference_config_pages/template_config_page.j2.qmd @@ -9,12 +9,15 @@ order: {{order}} {% if not d.name == "__this__" %} ## {{d.name|replace("`__merge__", "\_\_merge\_\_")}}{# Replace fixes __merge__ being rendered bold and coming before __this__ when sorted #} -{% if "Option of " in d.type %} -**Type**: `{{d.type|replace("Option of ", "")}}` -{% elif "OneOrMore of " in d.type %} -**Type**: `{{d.type|replace("OneOrMore of ", "")}}` / `List of {{d.type|replace("OneOrMore of ", "")}}` +{% if "Option of " in d.niceType %} +**Type**: `{{d.niceType|replace("Option of ", "")}}` +{% elif "OneOrMore of " in d.niceType %} +**Type**: `{{d.niceType|replace("OneOrMore of ", "")}}` / `List of {{d.niceType|replace("OneOrMore of ", "")}}` {% else %} -**Type**: `{{d.type}}` +**Type**: `{{d.niceType}}` +{% endif %} +{%- if d.default is defined %} +**Default**: `{{d.default}}` {% endif %} {% else %} {% endif -%} diff --git a/_src/automation/generate_version_blog_pages/script.py b/_src/automation/generate_version_blog_pages/script.py index 5dabf88c..1516411e 100644 --- a/_src/automation/generate_version_blog_pages/script.py +++ b/_src/automation/generate_version_blog_pages/script.py @@ -1,15 +1,23 @@ -import subprocess, re, yaml +import re from pathlib import Path +import jinja2 ## VIASH START par = { - 'input': "_src/automation/generate_version_blog_pages/test_changelog.md", - 'output': "output/" + 'input': '../viash/CHANGELOG.md', + 'output': 'blog/posts' +} +meta = { + "resources_dir": "_src/automation/generate_version_blog_pages" } ## VIASH END +output = Path(par["output"]) template_file = Path(meta['resources_dir'], "template_blog_page.j2.qmd") +with open(template_file) as f: + template = jinja2.Template(f.read()) + def bump_md_line(string: str) -> str: """ if header, bump header up one level """ if re.match(r"^#+ .+$", string): @@ -43,22 +51,14 @@ def handle_section(lines: list[str]): "changes": "".join(bump_markdown_headers(lines)) } - render_jinja_page(par['output'], f'viash-{version}/index.qmd', data) - -def render_jinja_page(folder: str, filename: str, data: dict): - """ Write data to yaml file and run jinja. """ - - full_path = Path(folder, filename) - base_dir = full_path.parent - yaml_file = Path(base_dir, "_" + full_path.name).with_suffix('.yaml') - qmd_file = full_path.with_suffix('.qmd') - - base_dir.mkdir(parents=True, exist_ok=True) - - with open(yaml_file, 'w') as outfile: - yaml.safe_dump(data, outfile, default_flow_style=False) + qmd_file = output / f'viash-{version}/index.qmd' + render_jinja_page(qmd_file, data) - subprocess.run(["jinja2", template_file, yaml_file, "-o", qmd_file]) +def render_jinja_page(path: Path, data: dict): + """Run jinja on data and write results to file.""" + path.parent.mkdir(parents=True, exist_ok=True) + content = template.render(**data) + path.write_text(content) def load_log(changelog_path: str): """ Load changelog and split into sections """ diff --git a/_src/render_pages.sh b/_src/render_pages.sh index a89a71f3..43ff0759 100755 --- a/_src/render_pages.sh +++ b/_src/render_pages.sh @@ -8,12 +8,18 @@ viash run _src/automation/generate_version_blog_pages/config.vsh.yaml -- \ source renv/python/virtualenvs/renv-python-3.10/bin/activate +echo "Removing reference subfolders but leaving top level files in place" +rm -rf ./reference/*/ + +echo "Copying static reference pages" +cp -r ../viash/docs/reference/* ./reference + echo "Creating cli information" -viash export cli_schema --output ./reference/cli_schema_export.json +viash export cli_schema --output ./reference/cli_schema_export.json --format json viash run _src/automation/generate_reference_cli_pages/config.vsh.yaml -- \ --input ./reference/cli_schema_export.json echo "Creating config information" -viash export config_schema --output ./reference/config_schema_export.json +viash export config_schema --output ./reference/config_schema_export.json --format json viash run _src/automation/generate_reference_config_pages/config.vsh.yaml -- \ --input ./reference/config_schema_export.json diff --git a/_viash.yaml b/_viash.yaml index de04f81b..cd21f68c 100644 --- a/_viash.yaml +++ b/_viash.yaml @@ -1,4 +1,4 @@ -viash_version: 0.7.4 +viash_version: 0.7.5 source: _src target: _target \ No newline at end of file diff --git a/blog/posts/viash-0.0.1/_index.yaml b/blog/posts/viash-0.0.1/_index.yaml deleted file mode 100644 index afac88c5..00000000 --- a/blog/posts/viash-0.0.1/_index.yaml +++ /dev/null @@ -1,9 +0,0 @@ -changes: '' -date: '2020-05-05' -subtitle: Initial release -version: 0.0.1 -whats_new: ' - - * Initial proof of concept. - - ' diff --git a/blog/posts/viash-0.1.0/_index.yaml b/blog/posts/viash-0.1.0/_index.yaml deleted file mode 100644 index 2c1211f7..00000000 --- a/blog/posts/viash-0.1.0/_index.yaml +++ /dev/null @@ -1,64 +0,0 @@ -changes: '### MAJOR CHANGES - - - * Refactoring of the Functionality class as discussed in VIP1 (#1). This has resulted - in a lot of internal changes, but the changes with regard to the yaml definitions - are relatively minor. See the section below for more info. - - - ### MINOR CHANGES - - - * Updated the functionality.yamls under `atoms/` and `src/test/` to reflect these - aforementioned changes. - - * Allow for bioconductor and other repositories in the R environment. - - * Add support for pip versioning syntax. - - - ### BUG FIXES - - - * Do not quote passthrough flags. - - * Allow for spaces inside of Docker volume paths. - - - ### DOCUMENTATION - - - * Updated the README.md. - - * Provide some small examples at `doc/examples`. - - - ' -date: '2020-05-14' -subtitle: Changes to functionality and the native/docker platforms -version: 0.1.0 -whats_new: ' - - ### Changes to functionality.yaml - - - * ftype has been renamed to function_type. The value for this field is also being - checked. - - * platform has been removed. - - * Instead, the first resource listed is expected to have `type: r_script`, `type: - bash_script`, `type: python_script`, or `type: executable`. The other resources - are expected to have `type: file` by default, and are left untouched by Viash. - - * in the arguments, field `flagValue` has been removed. Instead, use `type: boolean_true` - and `type: boolean_false` to achieve the same effect. - - - ### Changes to platform_(docker/native).yaml - - - * The `r: packages:` field has been renamed to `r: cran:`. - - - ' diff --git a/blog/posts/viash-0.2.0/_index.yaml b/blog/posts/viash-0.2.0/_index.yaml deleted file mode 100644 index d0f65d8e..00000000 --- a/blog/posts/viash-0.2.0/_index.yaml +++ /dev/null @@ -1,62 +0,0 @@ -changes: "### NEW FEATURES\n\n* Allow (optional) version attributes in `functionality.yaml`\ - \ and `platform.yaml`.\n* Allow testing a component with the `viash test` functionality.\ - \ Tests are executed in a temporary directory on the specified platform. The temporary\ - \ directory contains all the resource and test files. \n* `viash --version`: Add\ - \ flag for printing the version of viash.\n* Allow fetching resources from URL (http://\ - \ and https://)\n* Allow retrieving functionality and platform YAMLs from URL.\n\ - * For docker containers, autoresolve path names of files. Use `---v path:path` or\ - \ `---volume path:path` to manually mount a specific folder.\n* Implement parameter\ - \ multiplicity. \n Set `multiple: true` to denote an argument to have higher multiplicity.\ - \ \n Run `./cmd --foo one --foo two --foo three:four` in order for multiple values\ - \ to be added to the same parameter list.\n* Added a new format for defining functionality\ - \ in which the user passes the script in which the functionality and platforms are\ - \ listed as yaml headers.\n* A `---chown` flag has been added to Docker executables\ - \ to automatically change the ownership of output files to the current user.\n*\ - \ `viash ns build`: A command for building a whole namespace.\n* `NXF`: Join operations\ - \ are now fully supported by means of `multiple`.\n* `NXF`: Modules that perform\ - \ joins can take either arrays (multiple input files or the same type to be joined)\ - \ or hashes (multiple input files passed using different options on the CLI). Please\ - \ refer to the docs for more info.\n\n### MAJOR CHANGES\n\n* Remove passthrough\ - \ parameters.\n* Since CLI generation is now performed in the outer script, `viash\ - \ pimp` has been deprecated.\n* Write out meta.yaml containing viash run information\ - \ as well as the original `functionality.yaml` and `platform.yaml` content.\n* Renamed\ - \ `viash export` to `viash build`.\n\n### MINOR CHANGES\n\n* `viash run` and `viash\ - \ test`: Allow changing the temporary directory by defining `VIASH_TEMP` as a environment\ - \ variable. Temporary directories are cleaned up after successful executions.\n\ - * `viash run` and `viash test`: Exit(1) when execution or test fails.\n* `viash\ - \ build`: Add -m flag for outputting metadata after build.\n* `viash run`: Required\ - \ parameters can have a default value now. Produce error when a required parameter\ - \ is not passed, even when a default is provided.\n* `NXF`: _Modules_ are now stored\ - \ under `target/nextflow` by default\n\n### BUG FIXES\n\n* `NXF`: Correctly escape\ - \ path variable when running NXF command.\n* `NXF`: Surround parameters with quotes\ - \ when running NXF command.\n\n### INTERNAL CHANGES\n\n* Move CLI from inner script\ - \ to outer script.\n* Renamed Target to Platform\n* Renamed Environment to Requirements\n\ - \n" -date: '2020-09-01' -subtitle: Autoresolve docker paths -version: 0.2.0 -whats_new: "\n### Changes to functionality metadata\n\n- Added version attribute\n\ - \n#### Autoresolve docker paths\n\nArguments of type: file are processed to automatically\ - \ create a mount in docker. More specifically, when you pass an argument value:\ - \ `--input /path/to/file`, this will be processed such that the following parameters\ - \ are passed to docker:\n\n ```bash\n docker run -v /path/to:/viash_automount/path/to\ - \ ... --input /viash_automount/path/to/file\n ```\n\nIf, for some reason, you need\ - \ to manually specify a mount, you can do this with `---mount /path/to/mount:/mymount`.\n\ - \n#### Argument multiplicity\n\nFor all parameter types (except for `boolean_true`\ - \ and `boolean_false`), you can specify `multiple: true` in order to turn this argument\ - \ into an array-based argument. What this does is allow you to pass multiple values\ - \ for this argument, e.g. `--input file1 --input file2 --input file3:file4:file5`.\n\ - \nThe default separator is `:` but this can be overridden by changing the separator\ - \ by setting it to `multiple_sep: \",\"` (for example).\n\n#### New format\n\nViash\ - \ now supports placing the functionality.yaml, platform*.yaml(s) and script into\ - \ a single file. For example, this could be a merged script.R:\n\n ```r\n #' functionality:\n\ - \ #' name: r-estimate\n #' arguments: ...\n #' platforms:\n #' - type: native\n\ - \ #' - type: docker\n #' image: rocker/tidyverse\n library(tidyverse)\n cat(\"\ - Hello world!\\n\")\n ```\n\nInstead of running:\n\n ```bash\n viash run -f functionality.yaml\ - \ -p platform_docker.yaml -- arg1\n ```\n\nWith this format, you can now run:\n\ - \n ```bash\n viash run script.R # run script.R with the first\ - \ platform\n viash run -P docker script.R # run script.R with the platform\ - \ called 'docker' with the large P argument\n # use small p to override the platform\ - \ with a custom yaml:\n viash run -p common_resources/platform_docker.yaml script.R\n\ - \ # note that any arguments for the run command (e.g. -p or -P) should come before\ - \ the script.R, as script.R is considered a trailing argument.\n ```\n\n" diff --git a/blog/posts/viash-0.2.1/_index.yaml b/blog/posts/viash-0.2.1/_index.yaml deleted file mode 100644 index 45da9e9e..00000000 --- a/blog/posts/viash-0.2.1/_index.yaml +++ /dev/null @@ -1,61 +0,0 @@ -changes: '### NEW FEATURES - - - * `NXF`: Data references in Map form can now have values being lists. In other words, - we can have multiple options which have one or more values. - - * `viash ns build`: Added --parallel and --setup flag. - - * `viash build`: Added --setup flag. - - * Allow changing the order of setup commands using the `setup:` variable. - - * (HIDDEN) Do not escape `${VIASH_...}` elements in default values and descriptions! - - - ### MINOR CHANGES - - - * Remove `---chown` flag, move to `platform.docker.chown`; is set to true by default. - - * Perform chown during both run and test using a Docker platform. - - - ### BUG FIXES - - - * Issue trying to parse positional arguments even when none is provided. - - - ' -date: '2020-09-11' -subtitle: Docker chown by default -version: 0.2.1 -whats_new: "\n### Docker chown by default\n\nRunning a script using a Docker platform\ - \ will now chown output files by default, as well as any temporary files. You can\ - \ turn off this feature by specifying `chown: false` in the yaml of a Docker platform.\n\ - \n### [NXF] Data references\n\nData references in Map form can now have values being\ - \ lists. In other words, we can have multiple options which have one or more values.\n\ - \n### viash ns build -P docker --parallel --setup\n\n`viash ns build` has been greatly\ - \ improved! You can automatically build the docker container by adding `--setup`\ - \ to the command, as well as make the whole thing run in parallel using the `--parallel`\ - \ or `-l` flag.\n\nTo build a docker container, you can run either of the following:\n\ - \n ```bash\n viash run -f path/to/config.yaml -P docker -- ---setup\n viash build\ - \ -f path/to/functionality.yaml -P docker -o target/docker/path/to --setup\n ```\n\ - \nNote that the first will only build the docker container, whereas the second will\ - \ build the executable and then build the docker container.\n\nTo build a lot of\ - \ them all at once, run:\n\n ```bash\n viash ns build -P docker --parallel --setup\n\ - \ ```\n\n### Custom order of platform requirements\n\nYou can now choose the order\ - \ in which platform requirements are installed!\n\nBefore:\n\n ```yaml\n type:\ - \ docker\n image: rocker/tidyverse\n target_image: \"viash_test/r\"\n r:\n \ - \ cran:\n - optparse\n github:\n - dynverse/dynutils@devel\n bioc:\n\ - \ - limma\n apt:\n packages:\n - libhdf5-serial-dev\n docker:\n build_arg:\n\ - \ - GITHUB_PAT=\"$GITHUB_PAT\"\n run:\n - git clone --depth 1 https://github.com/data-intuitive/viash_docs.git\ - \ && rm -r viash_docs/.git\n \u2191 in which order will these three components\ - \ be run? Who knows!\n ```\n\nNow:\n\n ```yaml\n type: docker\n image: rocker/tidyverse\n\ - \ target_image: \"viash_test/r\"\n setup:\n - type: docker\n build_arg:\n\ - \ - GITHUB_PAT=\"$GITHUB_PAT\"\n - type: apt\n packages:\n - libhdf5-serial-dev\n\ - \ - type: r\n cran:\n - optparse\n - dynutils\n github:\n - rcannood/princurve@devel\n\ - \ bioc:\n - limma\n - type: docker\n run:\n - git clone --depth 1 https://github.com/data-intuitive/viash_docs.git\ - \ && rm -r viash_docs/.git\n ```\n\nThis will ensure that the setup instructions\ - \ are installed in the given order.\n\n" diff --git a/blog/posts/viash-0.2.2/_index.yaml b/blog/posts/viash-0.2.2/_index.yaml deleted file mode 100644 index fbe74799..00000000 --- a/blog/posts/viash-0.2.2/_index.yaml +++ /dev/null @@ -1,24 +0,0 @@ -changes: '### MINOR CHANGES - - - * Allow generating placeholder without VIASH START/VIASH END blocks. - - - ### BUG FIXES - - - * `viash ns build`: Some platforms would sometimes not be detected. - - * `viash run`: Avoid error when no arguments need to be chowned. - - - ' -date: '2020-09-22' -subtitle: Generation of placeholder code now possible without VIASH START and VIASH - END -version: 0.2.2 -whats_new: "\nAllow generating placeholder without VIASH START/VIASH END blocks.\n\ - \nA script does not need to contain a `VIASH START`/`VIASH END` block in order to\ - \ function.\n\nPreviously, each script had to contain a codeblock as follows:\n\n\ - \ ```r\n ## VIASH START\n par <- list(\n input = \"foo\",\n output = \"\ - bar\n )\n ## VIASH END\n ```\n\n" diff --git a/blog/posts/viash-0.3.0/_index.yaml b/blog/posts/viash-0.3.0/_index.yaml deleted file mode 100644 index b967a117..00000000 --- a/blog/posts/viash-0.3.0/_index.yaml +++ /dev/null @@ -1,39 +0,0 @@ -changes: "### BREAKING CHANGES\n\n* File format `functionality.yaml` is no longer\ - \ supported. Use `config.vsh.yaml` or `script.vsh.R/py/...` instead.\n\n* `viash\ - \ run` and `viash test`: By default, temporary files are removed when the execution\ - \ succeeded, otherwise they are kept. \n This behaviour can be overridden by specifying\ - \ `--keep true` to always keep the temporary files, and `--keep false` to always\ - \ remove them.\n\n* `NXF`: `function_type: todir` now returns the output directory\ - \ on the `Channel` rather than its contents.\n\n### NEW FEATURES\n\n* Added `viash\ - \ ns test`: Run all tests in a particular namespace. For each test, the exit code\ - \ and duration is reported. Results can be written to a tsv file.\n* Added support\ - \ for JavaScript scripts.\n* Added support for Scala scripts.\n* `NXF`: publishing\ - \ has a few more options:\n - `publish`: Publish or yes (default is false)\n -\ - \ `per_id`: Publish results in directories containing the unique (sample) ID (default\ - \ is true)\n - `path`: A prefix path for the results to be published (default is\ - \ empty)\n* Functionality resources and tests: Allow copying whole directories instead\ - \ of only single files. Also allow to rename the destination folder by specifying\ - \ a value for 'dest'.\n* Platform R / Python dependencies: Allow running a simple\ - \ command.\n\n### MAJOR CHANGES\n\n* The `-P ` parameter will be deprecated.\ - \ For now, all `-P` values are simply passed to `-p`.\n* `viash ns build` and `viash\ - \ ns test`: Now use all available platforms if `-p` is not specified.\n* By default,\ - \ python packages will not be installed as user. Use `user: true` to modify this\ - \ behaviour.\n\n### MINOR CHANGES\n\n* Name of autogenerated Docker image is now\ - \ `ns/tool`.\n* Internal changes to make it easier to extend viash with more scripting\ - \ languages.\n* `NXF`: Default image is now `ns/tool` for consistency.\n* `NXF`:\ - \ Repurpose `asis` function type for having simple publishing steps (see docs).\n\ - * `NXF`: Add component name to main `process` name\n* R dependencies: by default,\ - \ do not reinstall Bioconductor packages. Set `bioc_force_install: true` to revert\ - \ this behaviour.\n\n### BUG FIXES\n\n* `viash build`: Do not display error messages\ - \ when pwd is not a git repository.\n\n### TESTING\n\n* `viash test`: Add tests\ - \ for `viash test` functionality.\n\n" -date: '2020-11-24' -subtitle: Combine functionality and platform into one config, remove temporary files -version: 0.3.0 -whats_new: ' - - `config.vsh.yaml` is the new standard format, temporary files are removed when using - run and test commands. - - - ' diff --git a/blog/posts/viash-0.3.1/_index.yaml b/blog/posts/viash-0.3.1/_index.yaml deleted file mode 100644 index a8d9158e..00000000 --- a/blog/posts/viash-0.3.1/_index.yaml +++ /dev/null @@ -1,46 +0,0 @@ -changes: "### NEW FEATURES\n\n* Functionality: Added list of authors field. Example:\n\ - \n ```yaml\n functionality:\n authors:\n - name: Bob Cando\n roles:\ - \ [maintainer, author]\n email: bob@cando.com\n props: {github: bobcando,\ - \ orcid: XXXAAABBB}\n ```\n\n* `Docker`: Allow specifying the registry with `target_registry`.\ - \ Example:\n\n ```yaml\n - type: docker\n image: bash:4.0\n target_registry:\ - \ foo.io\n target_image: bar\n target_tag: 0.1\n ```\n\n* `Docker`: `version`\ - \ is now a synonym for `target_tag`.\n If both `version` and `target_tag` are not\ - \ defined, `functionality.version` will\n be used instead.\n \n* `Docker`: Can\ - \ change the Docker Setup Strategy by specifying\n - in the yaml: `setup_strategy:\ - \ xxx`\n - on command-line: `---docker_setup_strategy xxx` or `---dss xxx`\n \n\ - \ Supported values for the setup strategy are:\n - alwaysbuild / build: build\ - \ the image from the dockerfile (DEFAULT)\n - alwayscachedbuild / cachedbuild:\ - \ build the image from the dockerfile, with caching\n - alwayspull / pull: pull\ - \ the image from a registry\n - alwayspullelsebuild / pullelsebuild: try to pull\ - \ the image from a registry, else build it\n - alwayspullelsecachedbuild / pullelsecachedbuild:\ - \ try to pull the image from a registry, else build it with caching\n - ifneedbebuild:\ - \ if the image does not exist locally, build the image\n - ifneedbecachedbuild:\ - \ if the image does not exist locally, build the image with caching\n - ifneedbepull:\ - \ if the image does not exist locally, pull the image\n - ifneedbepullelsebuild:\ - \ if the image does not exist locally, pull the image else build it\n - ifneedbepullelsecachedbuild:\ - \ if the image does not exist locally, pull the image else build it with caching\n\ - \ - donothing / meh: do not build or pull anything\n \n### MAJOR CHANGES\n\n*\ - \ License: viash is now licensed under GPL-3.\n\n### MINOR CHANGES\n\n* CLI: Allow\ - \ parameters before and after specifying a viash config yaml. For example, \n both\ - \ following commands now work. Up until now, only the latter would work.\n - `viash\ - \ run config.vsh.yaml -p docker`\n - `viash run -p docker config.vsh.yaml`\n\n\ - * Functionality: Arguments field can now be omitted.\n\n* Scripts: Wrapped scripts\ - \ now contain a minimal header at the top.\n\n### BUG FIXES\n\n* `NXF viash build`:\ - \ Do not assume each config yaml has at least one test.\n\n* Scripts: Fix Docker\ - \ `chown` failing when multiple outputs are defined (#21).\n\n* JavaScriptRequirements:\ - \ Fix type getting set to \"python\" when unparsing.\n\n* `viash run . ---debug`:\ - \ Debug session should now work again\n\n* Native `---setup`: Fix missing newlines\ - \ when running native ---setup commands.\n\n* Main: Fix crashing when no arguments\ - \ are supplied.\n\n* Namespace: Show error message when the config file can't be\ - \ parsed.\n\n* Executable resource: Fix Docker automount handling for Executable\ - \ resources.\n\n### TESTING\n\n* YAML: Test invertibility of parsing/unparsing config\ - \ objects.\n\n" -date: '2021-01-26' -subtitle: Add fields for specifying authors and the Docker registry -version: 0.3.1 -whats_new: ' - - Add authors field to config, added registry fields to Docker platform config. - - - ' diff --git a/blog/posts/viash-0.3.2/_index.yaml b/blog/posts/viash-0.3.2/_index.yaml deleted file mode 100644 index 74ddb461..00000000 --- a/blog/posts/viash-0.3.2/_index.yaml +++ /dev/null @@ -1,22 +0,0 @@ -changes: "### BREAKING CHANGES\n\n* `viash build`: Do not automatically generate a\ - \ viash.yaml when creating an executable. \n Instead, you need to add the `-w|--write_meta`\ - \ flag in order to let viash know that it\n should generate a viash.yaml in the\ - \ resources dir.\n\n### MAJOR CHANGES\n\n* `NXF`: Add beta functionality for running\ - \ viash tests in Nextflow.\n\n### MINOR CHANGES\n\n* Resources: Rework the way resources\ - \ paths are converted to absolute URIs, should not have any impact on UX.\n\n###\ - \ BUG FIXES\n\n* `NXF`: Add temporary workaround for determining the used image\ - \ name when running a component.\n\n* Docker Platform: Set default setup strategy\ - \ to \"alwayscachedbuild\" as this used to be the default viash behaviour.\n\n*\ - \ `NXF`: Fix issue where resource dir would not get mounted depending on which inputs\ - \ are provided.\n\n* `NXF`: Accept multiple inputs when component is running as\ - \ standalone.\n\n" -date: '2021-02-04' -subtitle: Don't auto-generate viash.yaml and add beta unit testing in Nextflow -version: 0.3.2 -whats_new: ' - - The viash build command doesn''t generate a viash.yaml automatically anymore, added - beta functionality for running tests in Nextflow. - - - ' diff --git a/blog/posts/viash-0.4.0.1/_index.yaml b/blog/posts/viash-0.4.0.1/_index.yaml deleted file mode 100644 index 95595722..00000000 --- a/blog/posts/viash-0.4.0.1/_index.yaml +++ /dev/null @@ -1,21 +0,0 @@ -changes: '### BUG FIX - - - * `NXF`: Return original_params instead of updated params for now. - - - * `NXF`: Reinstate function_type: asis in line with the refactored module generation - code - - - * `viash ns test`: print header when `--tsv foo.tsv --append true` but foo.tsv doesn''t - exist yet. Fixes #45. - - - ' -date: '2021-05-12' -subtitle: Three small bug fixes. -version: 0.4.0.1 -whats_new: ' - - ' diff --git a/blog/posts/viash-0.4.0/_index.yaml b/blog/posts/viash-0.4.0/_index.yaml deleted file mode 100644 index f81f446e..00000000 --- a/blog/posts/viash-0.4.0/_index.yaml +++ /dev/null @@ -1,51 +0,0 @@ -changes: "### NEW FEATURES\n\n* Config modding: A custom viash DSL allows overriding\ - \ viash config properties at runtime. See online documentation for more information.\ - \ Example:\n\n ```\n viash ns test \\\n -p docker \\\n -c '.functionality.version\ - \ := \"1.0.0\"' \\\n -c '.platforms[.type == \"docker\"].target_registry := \"\ - my.docker-registry.com\"' \\\n -c '.platforms[.type == \"docker\"].setup_strategy\ - \ := \"pull\"' \\\n -l\n ```\n\n* `viash build`: The image can be pushed with\ - \ `--push`. The same can be done by passing `---push` to \n a viash executable.\n\ - \n* `viash ns` can query the name, namespace, or both, with the following arguments:\n\ - \ - `--query_namespace` or `-n`: filter the namespace with a regex.\n - `--query_name`:\ - \ filter the name with a regex.\n - `--query` or `-q`: filter the namespace/name\ - \ with a regex.\n\n* Added the `project_build`, `project_clean`, `project_push`\ - \ and `project_test` components to this repository.\n\n* Added a field `.functionality.info`\ - \ of type `Map[String, String]` in order to be able to specify custom annotations\ - \ to the component.\n\n### BREAKING CHANGES\n\n* `viash ns`: Argument `--namespace`\ - \ has been renamed to `--query_namespace`.\n\n* `viash ns`: Argument `--namespace`\ - \ does not implicitly change the namespace of the functionality anymore. You can\ - \ use the command DSL to reproduce this effect; for example: `-c '.functionality.namespace\ - \ := \"foo\"'`.\n \n* `Docker` & `NXF`: Attribute `version` is deprecated. Instead,\ - \ the default value will be `.functionality.version`, which can be overridden by\ - \ using the `tag` attribute.\n\n* `NXF`: When running a viash component as a Nextflow\ - \ module on its own, you now need to specify all input files on the command line.\ - \ For instance, if `--input` and `--reference` are input file arguments, you need\ - \ to start the process by running `nextflow run main.nf --input <...> --reference\ - \ <...> `. Previously only the input file needed to be specified.\n\ - \ \n* `Docker` & `NXF`: Default separator between namespace and image name has\ - \ been changed from `\"/\"` to `\"_\"`.\n\n### MINOR CHANGES\n\n* `Docker` & `NXF`:\ - \ Parsing of image attributes for both `Docker` and `Nextflow` platforms are better\ - \ aligned. You can define an image by specifying either of the following:\n - `{\ - \ image: 'ubuntu:latest' }` \n - `{ image: ubuntu, tag: latest }`\n \n* `Docker`\ - \ & `NXF`: Allow changing the separator between a namespace and the image name.\n\ - \n### NEXTFLOW REFACTORING\n\nThe generation of Nextflow modules has been refactored\ - \ thoroughly.\n \n* `NXF`: The implicitly generated names for output files/directories\ - \ have been improved leading to less clashes.\n\n* `NXF`: Allow for multiple output\ - \ files/directories from a module while keeping compatibility for single output.\ - \ Please [refer to the docs](/reference/config/platforms/nextflow/#multiple-outputs).\n\ - \n* `NXF`: Allow for zero input files by means of passing an empty list `[]` in\ - \ the triplet\n\n* `NXF`: Remove requirement for `function_type: todir`\n\n* `NXF`:\ - \ It is now possible to not only specify `label: ...` for a nextflow platform but\ - \ also `labels: [ ...]`.\n \n### BUG FIXES\n\n* Allow quotes in functionality descriptions.\n\ - \n* `NXF`: Providing a `default: ...` value for output file arguments is no longer\ - \ necessary.\n\n" -date: '2021-04-14' -subtitle: Config mod DSL and renames to viash ns arguments -version: 0.4.0 -whats_new: ' - - The viash ns command''s --namespace argument has been renamed to --query_namespace, - introduction of custom DSL for overriding config properties at runtime. - - - ' diff --git a/blog/posts/viash-0.5.0/_index.yaml b/blog/posts/viash-0.5.0/_index.yaml deleted file mode 100644 index b7115360..00000000 --- a/blog/posts/viash-0.5.0/_index.yaml +++ /dev/null @@ -1,64 +0,0 @@ -changes: "### BREAKING CHANGES\n\n* `DockerPlatform`: A Docker setup will be performed\ - \ by default. Default strategy has been changed to `ifneedbepullelsecachedbuild`\ - \ (#57).\n `---setup` strategy has been removed and `---docker_setup_strategy`\ - \ has been renamed to `---setup`.\n This change allows running a component for\ - \ the first time. During first time setup, the Docker container will be pulled or\ - \ built automatically. \n\n* `NativePlatform`: Deprecated the native setup field.\n\ - \n### MAJOR CHANGES\n\n* `NXF`: This version changes the handling logic for arguments.\ - \ An argument can be either `required` or not and can have a `default: ...` value\ - \ or not. Checks are implemented to verify that required arguments are effectively\ - \ provided _during_ pipeline running.\n\n* `NXF`: If one sticks to long-option argments\ - \ in the viash config, for all arguments that are _required_, the way of specifying\ - \ the arguments on the CLI is identical for the Docker and Nextflow platforms. Non-required\ - \ arguments can still be accessed from CLI using `--__\ - \ ...`.\n\n* `NXF`: Running a module as a standalone pipeline has become easier.\n\ - \n* `viash run`: Implement verbosity levels (#58). viash executables now have 7\ - \ levels of verbosity: emergency, alert, critical, error, warning, notice, info,\ - \ debug.\n The default verbosity level is 'notice'. Passing `-v` or `--verbose`\ - \ bumps up the verbosity level by one, `-vv` by two. The verbosity level can be\ - \ set manually by passing `--verbosity x`.\n\n### MINOR CHANGES\n\n* `Docker Platform`:\ - \ Added `privileged` argument, allowing to run docker with the `--privileged` flag.\n\ - \n* `Docker Requirements`: Allow specifying environment variables in the Dockerfile.\n\ - \n* Config modding: Added a `+0=` operator to prepend items to a list.\n\n* `viash\ - \ run`: Added a `--version` flag to viash executables for viewing the version of\ - \ the component.\n\n* `Functionality`: Added checks on the functionality and argument\ - \ names.\n\n* `viash run`: Added examples to functionality and arguments. Reworked\ - \ `--help` formatting to include more information and be more consistent (#56).\n\ - \n### BUG FIXES\n\n* `Docker R Requirements`: Install `remotes` when using `{ type:\ - \ r, packages: [ foo ] }`.\n\n* `config`: Throw error when user made a typo in the\ - \ viash config (#62). \n\n### TESTING\n\n* `NXF`: Add an end-to-end test for running\ - \ a nextflow pipeline using viash components.\n\n* `Docker`: Reorganized viash docker\ - \ build testbench into a main testbench with smaller auxiliary testbenches to keep\ - \ them more manageable and clear what happens where.\n\n* `viash ns`: Added a basic\ - \ testbench for namespace tests.\n\n" -date: '2021-08-16' -subtitle: Improvements to running Docker executables, and Nextflow platform argument - changes -version: 0.5.0 -whats_new: "\nHere are the most important changes:\n\n* **Improvements to Docker backend**:\ - \ In the past, you needed to perform `--setup` on your Docker-based components and\ - \ executables in order for the image to be built before you could run the component\ - \ or executable. Now you can simply run your component or executable and Viash will\ - \ do the image building automatically by default if it detects an image isn't present\ - \ yet. This behaviour can be changed by using a Docker setup strategy. For example:\n\ - \n ```bash\n viash build config.vsh.yaml -p docker --setup alwayscachedbuild\n\ - \ ```\n\n* **Nextflow gets some argument changes**: Arguments for the Nextflow\ - \ platform now have optional `required` and `default` values, just like their native\ - \ and Docker counterparts. For example:\n\n ```yaml\n arguments:\n - name:\ - \ --name\n type: string\n description: Input name\n required:\ - \ true\n - name: --repeat\n type: integer\n description: Times\ - \ to repeat the name\n default: 100\n ```\n\n Take a look at the Functionality\ - \ page for more information on arguments and their properties. \n As long as you\ - \ use long-option arguments (e.g. `--my-option`) in the config file for required\ - \ arguments, the way of specifying argument values for the Nextflow platform is\ - \ identical to the Docker platform. You still access non-required arguments via\ - \ this syntax: `--__ `. For example:\n\n \ - \ ```bash\n my_component -- --my_component__input Hello!\n ```\n\n* **Verbosity\ - \ levels for viash run**: Executables now have 8 levels of verbosity\n\n 0. emergency\n\ - \ 1. alert\n 2. critical\n 3. error\n 4. warning\n 5. notice\n 6. info\n \ - \ 7. debug\n\n The default verbosity level is **notice**.\n You can pass the `-v`\ - \ or `--verbose` option to bump up the verbosity by one level. By passing `-vv`\ - \ the verbosity goes up by two levels. You can manually set the verbosity by using\ - \ the `--verbosity ` option. For example, if you wanted to only show\ - \ errors or worse:\n\n ```bash\n viash run config.vsh.yaml -- --verbosity 3\n\ - \ ```\n\n" diff --git a/blog/posts/viash-0.5.1/_index.yaml b/blog/posts/viash-0.5.1/_index.yaml deleted file mode 100644 index debc9405..00000000 --- a/blog/posts/viash-0.5.1/_index.yaml +++ /dev/null @@ -1,76 +0,0 @@ -changes: '### NEW FEATURES - - - * `CSharpScript`: Added support for C# scripts (`type: "csharp_script"`) to viash. - - - ### MINOR CHANGES - - - * `NextflowPlatform`: Added `directive_cpus`, `directive_max_forks`, `directive_memory` - and `directive_time` parameters. - - - ### BUG FIXES - - - * `BashWrapper`: Refactor escaping descriptions, usages, defaults, and examples - (#34). - - - * `NextflowPlatform`: Refactor escaping descriptions, usages, defaults and examples - (#75). - - - * `NextflowPlatform`: Add argument to output path to avoid naming conflicts for - components with multiple output files (#76). - - - * `NextflowPlatform`, `renderCLI()`: Only add flag to rendered command when boolean_true - is actually true (#78). - - - * `DockerPlatform`: Only chown when output file exists. - - - ### TESTING - - - * `viash build`: Capture stdout messages when errors are expected, so that they - don''t clutter the expected output. - - - * `viash build`: Check `--help` description output on the whole text instead of - per letter or word basis. - - - * `TestingAllComponentsSuite`: Only testing bash natively, because other dependencies - might not be available. - - - ' -date: '2021-07-14' -subtitle: Viash 0.5.1 adds support for C# scripts and fixes a few bugs -version: 0.5.1 -whats_new: "\n### C# script support\n\nWe've added C# scripts (.csx) as a supported\ - \ language using **dotnet-script**. \nTo run C# scripts natively, you'll need to\ - \ install .NET Core and execute the following command in a terminal:\n\n ```bash\n\ - \ dotnet tool install -g dotnet-script\n ```\n\nYou can now run C# scripts like\ - \ this:\n\n ```bash\n dotnet script hello_viash.csx\n ```\n\nTo use C# scripts\ - \ as components, use the new `csharp_script` type in the functionality section of\ - \ your config file:\n\n ```yaml\n resources:\n - type: csharp_script\n \ - \ path: script.csx\n ```\n\nHere's an example of a simple C# script with Viash\ - \ in mind:\n\n ```csharp\n // VIASH START\n var par = new {\n input = \"Hello\ - \ World\",\n name = \"Mike\"\n };\n // VIASH END\n\n System.Console.WriteLine(input\ - \ + \", \" + name + \"!\");\n ```\n\nThe language-specific guide for creating C#\ - \ script components will be added in the near future.\n\n### Bug fixes\n\nFirst\ - \ off, these special characters can now be used in the description, usage, default\ - \ and example fields of components:\n\n- \"\n- \\`\n- \\\\\n- \\n\n- $\n\nNextflow\ - \ output files with the same extension won't overwrite each other any more, like\ - \ it was the case for arguments like this:\n\n ```yaml\n functionality:\n name:\ - \ bar\n arguments:\n - name: \"--input\"\n type: file\n example:\ - \ input.txt\n - name: \"--output1\"\n type: file\n direction:\ - \ output\n required: true\n example: output.txt\n - name: \"\ - --output2\"\n type: file\n direction: output\n required: true\n\ - \ example: optional.txt\n ```\n\nIn this case, the two output files would\ - \ have been identical in the past.\n___\n\n" diff --git a/blog/posts/viash-0.5.10.1/_index.yaml b/blog/posts/viash-0.5.10.1/_index.yaml deleted file mode 100644 index ae7a6b6b..00000000 --- a/blog/posts/viash-0.5.10.1/_index.yaml +++ /dev/null @@ -1,17 +0,0 @@ -changes: '### BUG FIX - - - * `NextflowPlatform`: Fix passthrough of `organization` field. - - - ' -date: '2022-03-16' -subtitle: A quick bug fix -version: 0.5.10.1 -whats_new: ' - - This quick release fixes a bug that prevented the correct passthrough of the new - `organization` field. - - - ' diff --git a/blog/posts/viash-0.5.10/_index.yaml b/blog/posts/viash-0.5.10/_index.yaml deleted file mode 100644 index af9282a7..00000000 --- a/blog/posts/viash-0.5.10/_index.yaml +++ /dev/null @@ -1,36 +0,0 @@ -changes: "### MAJOR CHANGES\n\n* `viash_install`:\n - Added `--log_prefix`: This\ - \ prefix is used to determine the path of the log files for `viash_build`, `viash_test`\ - \ and `viash_push`.\n - Added `--organization`: Id of the organisation to be used\ - \ in the Docker image name, i.e. `//`.\n\ - \ - Added `--target_image_source`: Url to the Git repo in which this project resides.\n\ - \ - Removed `--log`.\n\n* `viash_build`:\n - Reduce code duplication by contructing\ - \ the command with Bash Arrays.\n - Renamed `--platforms` to `--platform`.\n -\ - \ Added `--organization`: Id of the organisation to be used in the Docker image\ - \ name, i.e. `//`.\n -\ - \ Added `--target_image_source`: Url to the Git repo in which this project resides.\n\ - \ - Changed default of `--log` from `log.txt` to `.viash_build_log.txt`.\n - Added\ - \ `--verbose`: Print out the underlying `viash ns build` command before running\ - \ it.\n\n* `viash_test`:\n - Reduce code duplication by contructing the command\ - \ with Bash Arrays.\n - Renamed `--platforms` to `--platform`.\n - Added `--organization`:\ - \ Id of the organisation to be used in the Docker image name, i.e. `//`.\n\ - \ - Added `--target_image_source`: Url to the Git repo in which this project resides.\n\ - \ - Changed default of `--log` from `log.txt` to `.viash_test_log.txt`.\n - Changed\ - \ default of `--tsv` from `log.tsv` to `.viash_test_log.tsv`.\n - Added `--verbose`:\ - \ Print out the underlying `viash ns test` command before running it.\n\n* `viash_push`:\n\ - \ - Reduce code duplication by contructing the command with Bash Arrays.\n - Added\ - \ `--organization`: Id of the organisation to be used in the Docker image name,\ - \ i.e. `//`.\n - Changed\ - \ default of `--log` from `log.txt` to `.viash_push_log.txt`.\n - Added `--verbose`:\ - \ Print out the underlying `viash ns build` command before running it.\n\n### MINOR\ - \ CHANGES\n\n* `NextflowPlatform`: Added the `organization` field to the nextflow\ - \ platform as well.\n\n" -date: '2022-03-15' -subtitle: Rework of the Viash helper components -version: 0.5.10 -whats_new: ' - - The `viash_install`, `viash_build`, `viash_test` and `viash_push` components have - been reworked. - - - ' diff --git a/blog/posts/viash-0.5.11/_index.yaml b/blog/posts/viash-0.5.11/_index.yaml deleted file mode 100644 index 97db18d3..00000000 --- a/blog/posts/viash-0.5.11/_index.yaml +++ /dev/null @@ -1,38 +0,0 @@ -changes: "### MAJOR CHANGES\n\n* `Functionality`: Now also accepts 'inputs' and 'outputs'\ - \ in addition to 'arguments'. For inputs and outputs,\n any specified arguments\ - \ will have default `type: file` and `direction: input` or `direction: output` respectively.\n\ - \n### MINOR CHANGES\n\n* `DockerPlatform`: Move description labels to the end of\ - \ the Dockerfile to improve cross-component caching.\n\n* `Functionality`: Arguments\ - \ where `.multiple` is `true` can now have lists as `default` and `example`.\n\n\ - * `viash_build`: Added unit test for this component.\n\n* `viash_test`: Added unit\ - \ test for this component.\n\n* `PythonRequirements`: Allow upgrading dependencies.\ - \ Example: `[ type: python. pypi: anndata, upgrade: true ]`.\n\n* `NextflowLegacyPlatform`:\ - \ Remove annoying messages when building Nxf modules.\n\n* `ConfigMods`: Expanded\ - \ the DSL to allow specifying at which point to apply a config mod.\n This functionality\ - \ was necessary to allow for setting fields which alter the way configs are parsed.\n\ - \ Example of when this is useful: ` .platforms[.type == \"nextflow\"\ - ].variant := \"vdsl3\"`.\n Updating workflow of parsing a config file is:\n \ - \ - read Yaml from file\n - apply preparse config mods\n - parse resulting\ - \ Json as Config, thereby instantiating default values etc.\n - convert Config\ - \ back to Json\n - apply postparse config mods (original config mods)\n -\ - \ convert final Json back to Config\n\n### BETA FUNCTIONALITY\n\n* `NextflowVdsl3Platform`:\ - \ A beta implementation of the next-generation Viash+Nextflow platform.\n See https://github.com/viash-io/viash/issues/82\ - \ for more information. You can access the previous Nextflow\n platform by using\ - \ the `variant` parameter:\n ```yaml\n - type: nextflow\n variant: legacy\n\ - \ separate_multiple_outputs: false\n ```\n\n### BUG FIXES\n\n* `viash_build`\ - \ and `viash_test`: The `query_name` and `query_namespace` arguments were switched\ - \ around. These arguments are now passed correctly.\n\n* `BashScript`, `JavaScriptScript`,\ - \ `PythonScript`, `RScript`: Correctly escape `'` (#113). Update unit tests accordingly.\n\ - \n* `CSharpScript`, `ScalaScript`: Correctly escape `\"` (#113). Update unit tests\ - \ accordingly.\n\n* `viash_build`, `viash_test`, `viash_push`: Don't try to remove\ - \ log files if they don't exist.\n\n### INTERNAL CHANGES\n\n* `DataObject`: \n \ - \ - Renamed `otype` to `flags`.\n - Renamed `oType` to `type`\n - Deprecated `tag`\ - \ (unused feature).\n\n* All abstract / inherited classes: Renamed `oType` to `type`.\n\ - \n### DEPRECATION\n\n* `Functionality`: Deprecated `function_type` and `add_resources_to_path`.\ - \ These should be \n unused features, by now.\n \n" -date: '2022-05-09' -subtitle: Nextflow VDSL3 is here! -version: 0.5.11 -whats_new: "\nThis release contains additional sugar syntax for specifying inputs\ - \ and outputs in a Viash config, \na beta implementation for the next-generation\ - \ Viash platform, and several other minor improvements.\n\n" diff --git a/blog/posts/viash-0.5.12/_index.yaml b/blog/posts/viash-0.5.12/_index.yaml deleted file mode 100644 index d1a7348e..00000000 --- a/blog/posts/viash-0.5.12/_index.yaml +++ /dev/null @@ -1,34 +0,0 @@ -changes: "### MINOR CHANGES\n\n* `--help`: Don't print \"my_component \"\ - \ when no version is specified, \n but instead simply \"my_component\".\n\n* `NextflowVdsl3Platform`:\ - \ Set `mode=copy` for `auto.publish` and `auto.transcript`.\n\n* `NextflowVdsl3Platform`:\ - \ When a module is used multiple times in the same workflow, \n don't throw an\ - \ error anymore, instead simply generate a warning.\n\n* `NextflowVdsl3Platform`:\ - \ Throw an error when an input file was not found.\n\n* `viash build`: Indent auto-generated\ - \ code according the indentation of `VIASH START` when found.\n \n* `Main`: Handle\ - \ not finding the config file or resources in a config file better.\n Display a\ - \ more helpful message instead of a stack trace.\n\n* `BashWrapper`: Add checks\ - \ on parameters for valid integer, double and boolean values.\n\n* `BashWrapper`:\ - \ Add option to limit string and integer values to specific choice values.\n\n*\ - \ `BashWrapper`: Add option to set min and max values for integer and double values.\n\ - \n* Dependencies:\n - Scala was upgraded from 2.12.10 to 2.12.15\n - sbt was upgraded\ - \ from 1.3.4 to 1.6.1\n - sbt-scoverage was upgraded from 1.5.1 to 1.9.3\n\n###\ - \ BUG FIXES\n\n* `viash_test`: Add back `--no_cache` parameter to `viash_test`.\n\ - \n* `viash_test`: Fix `--append` parameter for `viash_test`, was not getting passed\ - \ through.\n\n* `viash ns test`: Fix `--append` parameter, actually start from a\ - \ clean file if append is false.\n\n* `viash_push`: Fix component not being built\ - \ during a release of Viash.\n\n* `PythonRequirements`: Fix packages being mentioned\ - \ twice in a Dockerfile.\n\n* `Main`: Added support spaces in filenames of config\ - \ files and resources\n\n* `BashWrapper`: Display a message when the last parsed\ - \ argument would require more values than are still available.\n Now display a\ - \ message that values are missing, used to silently crash the wrapper.\n\n* `viash\ - \ config inject`: Fix error when file argument is `must_exist: true`.\n \n\n" -date: '2022-05-24' -subtitle: Improvements for VDSL3 and the Bash wrapper + several bug fixes -version: 0.5.12 -whats_new: ' - - This release contains a bunch improvements for VDSL3 and adds some parameters to - the `viash test` and `viash test ns` commands. - - - ' diff --git a/blog/posts/viash-0.5.13/_index.yaml b/blog/posts/viash-0.5.13/_index.yaml deleted file mode 100644 index 60dc01da..00000000 --- a/blog/posts/viash-0.5.13/_index.yaml +++ /dev/null @@ -1,26 +0,0 @@ -changes: "### NEW FUNCTIONALITY\n\n* `NextflowVdsl3Platform`: Allow overriding the\ - \ container registry of all Viash components by \n setting the `params.override_container_registry`\ - \ value. Only works for auto-derived image names.\n\n### MAJOR CHANGES\n\n* `Functionality`:\ - \ renamed `tests` to `test_resources`.\n Backwards compatibility provided but a\ - \ notification message is displayed on the console.\n\n### MINOR CHANGES\n\n* `Functionality`\ - \ and `viash ns`: Added `.enabled` in functionality, set to `true` by default.\n\ - \ Filter for disabled components in namespace commands.\n\n* `DockerPlatform`:\ - \ Add org.opencontainers.image annotations to built docker images.\n\n* `Functionality`:\ - \ when defining text resources, permit defining `path` instead of `dest`.\n If\ - \ both `dest` and `path` are unset, use a default file name depending on the resource\ - \ type, such as `script.sh` or `text.txt`.\n\n* `viash build`: Errors are printed\ - \ in red.\n\n### BUG FIXES\n\n* `NextflowVdsl3Platform`: Undefined input files should\ - \ not inject a `VIASH_PAR_*` variable when `multiple: true`.\n\n* `NextflowVdsl3Platform`:\ - \ Make injected resources dir absolute.\n\n* `NextflowVdsl3Platform`: Fix escaping\ - \ of triple single quotes.\n\n* `NextflowVdsl3Platform`: Also apply auto.simplifyInput\ - \ to Lists.\n\n* `DockerPlatform`: added a `test_setup` that allows adding apt/apk/...\ - \ setup requirements.\n These are only executed when running tests.\n\n" -date: '2022-06-10' -subtitle: Added overriding of the container registry for the VDSL3 + VDSL3 bug fixes -version: 0.5.13 -whats_new: ' - - VDSL3 gets even more improvements and bug fixes. - - - ' diff --git a/blog/posts/viash-0.5.14/_index.yaml b/blog/posts/viash-0.5.14/_index.yaml deleted file mode 100644 index 8d74c472..00000000 --- a/blog/posts/viash-0.5.14/_index.yaml +++ /dev/null @@ -1,55 +0,0 @@ -changes: "### NEW FUNCTIONALITY\n\n* `Functionality`: Allow specifying argument groups.\ - \ Example:\n ```yaml\n functionality:\n ...\n argument_groups:\n -\ - \ name: First group\n arguments: [foo, bar]\n description: Description\n\ - \ ```\n\n* Addition of the `viash_nxf_schema` component for converting a Viash\ - \ config (for a workflow) into a nextflow schema file.\n\n* `NextflowVdsl3Platform`:\ - \ Use `--param_list` to initialise a Nextflow channel with multiple parameter sets.\n\ - \ Possible formats are csv, json, yaml, or simply a yaml_blob.\n A csv should\ - \ have column names which correspond to the different arguments of this pipeline.\n\ - \ A json or a yaml file should be a list of maps, each of which has keys corresponding\ - \ to the arguments of the pipeline.\n A yaml blob can also be passed directly as\ - \ a parameter.\n Inside the Nextflow pipeline code, params.param_list can also\ - \ be used to directly a list of parameter sets.\n When passing a csv, json or yaml,\ - \ relative path names are relativized to the location of the parameter file.\n \ - \ \n Examples: \n ```sh\n nextflow run \"target/foo/bar/main.nf\" --param_list\ - \ '[{\"id\": \"foo\", \"input\": \"/path/to/bar\"}]'\n nextflow run \"target/foo/bar/main.nf\"\ - \ --param_list \"params.csv\" --reference \"/path/to/ref\"\n ```\n\n### MAJOR CHANGES\n\ - \n* `NextflowVdsl3Platform`: The functionality is now slurped from a json instead\ - \ of manually\n taking care of the formatting in Groovy.\n\n* `NextflowVdsl3Platform`:\ - \ The `--help` is auto-generated from the config.\n\n### MINOR CHANGES\n\n* `NextflowVdsl3Platform`:\ - \ Allow both `--publish_dir` and `--publishDir` when `auto.publish = true`.\n\n\ - * `NextflowVdsl3Platform`: Allow passing parameters with multiplicity > 1 from Nextflow\ - \ CLI.\n\n* `Main`: Added `viash --cli_export` which outputs the internal cli construction\ - \ information \n to console. This is to be used to automate populating the documentation\ - \ website.\n\n* `viash ns`: Display success and failure summary statistics, printed\ - \ to stderr.\n\n* `DataObject`: `.alternatives` is now a `OneOrMore[String]` instead\ - \ of `List[String]`, meaning\n you can now specify `{ type: string, name: \"--foo\"\ - , alternatives: \"-f\" }` instead of \n `{ type: string, name: \"--foo\", alternatives:\ - \ [ \"-f\" ] }`\n\n* `BashWrapper`: Added metadata field `meta_executable`, which\ - \ is a shorthand notation for\n `meta_executable=\"$meta_resources_dir/$meta_functionality_name\"\ - `\n\n### INTERNAL CHANGES\n\n* `Arguments`: Internal naming of functionality.arguments\ - \ is changed from DataObject to Arguments. Change is also applied to child classes,\ - \ e.g. StringObject -> StringArgument.\n\n* `Script`: Allow more control over where\ - \ injected code ends up.\n\n* Restructure type system to allow type-specific arguments.\n\ - \n### BUG FIXES\n\n* `DockerPlatform`: Change `org.opencontainers.image.version`\ - \ annotation to `functionality.version` when set.\n Additionally fixed retrieving\ - \ the git tag possibly returning `fatal: No names found, cannot describe anything.`\ - \ or similar.\n\n* `viash config inject`: Fix config inject when `.functionality.inputs`\ - \ or `.functionality.outputs` is used.\n\n* `BashWrapper`: Don't add `bc` as dependency.\ - \ Only perform integer/float min/max checks when bc is available, otherwise ignore.\n\ - \n* `DockerPlatform`: Fix inputs & outputs arguments being present twice.\n\n* `viash\ - \ ns test`: Silently skip Nextflow platforms as these don't support tests and will\ - \ always fail.\n\n* `Testbenches`: Better capture expected error messages while\ - \ running testbenches. Having these show on the console could be confusing.\n\n\ - * `NextflowVdsl3Platform`: Fix issue when running multiple VDSL3 modules concurrently\ - \ on the same channel.\n\n" -date: '2022-06-30' -subtitle: Argument groups can now be defined in the Viash config -version: 0.5.14 -whats_new: ' - - Argument groups allow for grouping arguments together by function or category, making - the `--help` output a lot more clear for components with a lot of arguments. - - - ' diff --git a/blog/posts/viash-0.5.15/_index.yaml b/blog/posts/viash-0.5.15/_index.yaml deleted file mode 100644 index b634290e..00000000 --- a/blog/posts/viash-0.5.15/_index.yaml +++ /dev/null @@ -1,45 +0,0 @@ -changes: "### BREAKING CHANGES\n\n* `WorkflowHelper::helpMessage`: Now only takes\ - \ one argument, namely the config.\n\n### MAJOR CHANGES\n\n* `Namespace`: Changed\ - \ the namespace of viash from `com.dataintuitive.viash` to `io.viash`.\n\n### MINOR\ - \ CHANGES\n\n* `Testbenches`: Add a testbench framework to test lots of character\ - \ sequences, single or repeating to be tested in the yaml config. This can be used\ - \ to later extend to other tests.\n\n* `Testbenches::vdsl3`: Add testbenches to\ - \ verify functionality:\n - Vdsl3's `param_list` (`yamlblob`, `yaml`, `json`, `csv`).\n\ - \ - Nextflow's own `params-file`.\n - Vdsl3's recalculating resource file paths\ - \ to be relative to the `param_list` file instead of the workflow file (only available\ - \ for `yaml`, `json`, `csv`).\n - Vdsl3's wrapping of modules to run these as a\ - \ separate workflow automagically out of the box.\n\n* `Main`: Added `viash --schema_export`\ - \ which outputs a schema of the Viash config file\n to console. This is to be used\ - \ to automate populating the documentation website.\n\n* `Helper`: Split help message\ - \ by argument group.\n\n* `Helper`: Remove unneeded arguments.\n\n* `Functionality`:\ - \ Add default groups `Inputs`, `Outputs` and `Arguments` for all arguments missing\ - \ from user-defined `argument_groups`.\n\n* `WorkflowHelper::helpMessage`: Rewrite\ - \ to bring on par with Viash's help message.\n\n* `BooleanArguments`: Renamed internal\ - \ class names for BooleanArguments to be better in line with how they are named\ - \ in the config yaml.\n `BooleanArgumentRegular` -> `BooleanArgument` (in line\ - \ with `boolean`)\n `BooleanArgumentTrue` -> `BooleanTrueArgument` (in line with\ - \ `boolean_true`)\n `BooleanArgumentFalse` -> `BooleanFalseArgument` (in line with\ - \ `boolean_false`)\n\n### BUG FIXES\n\n* `NextflowVdsl3Platform`: Change how `--id`\ - \ is processed when a VDSL3 module is called from the CLI.\n\n* `NextflowVdsl3Platform`:\ - \ Fix error when param_list is `null`.\n\n* `NextflowVdsl3Platform`: Fix error when\ - \ optional, multiple arguments are set to `null`.\n\n* `Testbenches`: Better capture\ - \ expected error messages while running testbenches again. Code changes right before\ - \ previous release re-introduced some of the messages.\n\n* `NextflowVdsl3Platform`:\ - \ Fix issue where optional parameters aren't removed when `.run(args: [optarg: null])`.\n\ - \n* `WorkflowHelper::readCsv`: Treat empty values as undefined instead of throwing\ - \ an error.\n\n* `NextflowVdsl3Platform`: Use `$NXF_TEMP` or `$VIASH_TEMP` as temporary\ - \ directory if the container engine is not set to `docker`, `podman` or `charlieengine`,\ - \ else set to `/tmp`.\n\n* `Resources`: When adding a resource folder, allow a trailing\ - \ `/` at the end of the path.\n Previously this caused the target folder to be\ - \ erased and the content of the resource folder to be written directly into the\ - \ target folder.\n\n" -date: '2022-07-14' -subtitle: Added testbenches, default argument groups and bugfixes for VDSL3 -version: 0.5.15 -whats_new: ' - - This release introduces testbenches and new default argument groups: `Inputs`, `Outputs` - and `Arguments`. - - - ' diff --git a/blog/posts/viash-0.5.2/_index.yaml b/blog/posts/viash-0.5.2/_index.yaml deleted file mode 100644 index 9518b01c..00000000 --- a/blog/posts/viash-0.5.2/_index.yaml +++ /dev/null @@ -1,20 +0,0 @@ -changes: "### MINOR CHANGES\n\n* `DockerPlatform`: Added `run_args` field to allow\ - \ setting `docker run` arguments.\n\n* `NextflowPlatform`: Added argument `separate_multiple_outputs`\ - \ to allow not separating the outputs generated by a \n component with multiple\ - \ outputs as separate events on the channel.\n\n### BUG FIX\n\n* `IO`: Allow overwriting\ - \ directory resources upon rebuild.\n\n" -date: '2021-08-13' -subtitle: More settings for Docker and Nextflow platform, and a bug fixes for components - with resources -version: 0.5.2 -whats_new: "\nThis is a small release containing two small features and a bug fix.\n\ - The new `run_args` field allows you to add [docker run](https://docs.docker.com/engine/reference/commandline/run/)\ - \ arguments to the [Docker platform](/reference/config/platforms/docker/#) section\ - \ of a [config file](/reference/config/index.html). For example:\n\n ```yaml\n\ - \ platforms:\n - type: docker\n image: bash:4.0\n run_args: \"--expose\ - \ 127.0.0.1:80:8080/tcp --env MY_ENV_VAR=foo\"\n ```\n\nThere's also a new field\ - \ for the [Nextflow platform](/reference/config/platforms/nextflow/#): `separate_multiple_outputs`.\ - \ By default, this is set to `true` and separates the outputs generated by a Nextflow\ - \ component with multiple outputs as separate events on the channel. You can now\ - \ choose to disable this behaviour:\n\n ```yaml\n platforms:\n - type: nextflow\n\ - \ publish: true\n separate_multiple_outputs: false\n ```\n\n" diff --git a/blog/posts/viash-0.5.3/_index.yaml b/blog/posts/viash-0.5.3/_index.yaml deleted file mode 100644 index dca48227..00000000 --- a/blog/posts/viash-0.5.3/_index.yaml +++ /dev/null @@ -1,29 +0,0 @@ -changes: "### NEW FEATURES\n\n* Similar to `par`, each script now also has a `meta`\ - \ list. `meta` contains meta information about the component\n or the execution\ - \ thereof. It currently has the following fields:\n - `meta[\"resources_dir\"]`:\ - \ Path to the directory containing the resources\n - `meta[\"functionality_name\"\ - ]`: Name of the component\n\n* `NextflowPlatform`: Export `VIASH_TEMP` environment\ - \ variable. \n\n### BUG FIXES\n\n* `NextflowPlatform`: Fix output formatting when\ - \ `separate_multiple_outputs` is `false`.\n\n" -date: '2021-09-02' -subtitle: New meta data list for scripts, VIASH_TEMP environment variable for Nextflow, - fixed output formatting with separate outputs -version: 0.5.3 -whats_new: "\nThis release provides more information to scripts with the new `meta`\ - \ list. This list contains two values for now:\n\n - `meta[\"resources_dir\"]`:\ - \ Path to the directory containing the resources\n - `meta[\"functionality_name\"\ - ]`: Name of the component\n\nA new environment variable is now available for export\ - \ when working with the Nextflow platform: `VIASH_TEMP`.\n\n### Resources directory\n\ - \nAll resources defined in the config file are copied over to a temporary location\ - \ right before a Viash component is executed. This location is can now be easily\ - \ accessed in your scripts, allowing you to modify and copy the files as needed.\ - \ \nHere are some examples in different scripting languages on how to access the\ - \ meta data, it works similarly to the `par` list:\n\nBash: \n\n ```bash\n echo\ - \ $meta_resources_dir \n ```\n\nPython: \n\n ```python\n print(meta[\"resources_dir\"\ - ])\n ```\n\nR:\n\n ```r\n cat(meta$resources_dir)\n ```\n\n### Functionality\ - \ name\n\nThe name of the component can now be accessed in the same way as the resources\ - \ directory. This allows you to print the name of the component out to a console\ - \ window for example.\nHere's how to access this data in different scripting languages:\n\ - \nBash:\n\n ```bash\n echo $meta_functionality_name\n ```\n\nPython: \n\n ```python\n\ - \ print(meta[\"functionality_name\"])\n ```\n\nR:\n\n ```r\n cat(meta$functionality_name)\n\ - \ ```\n\n" diff --git a/blog/posts/viash-0.5.4/_index.yaml b/blog/posts/viash-0.5.4/_index.yaml deleted file mode 100644 index 84ee7e54..00000000 --- a/blog/posts/viash-0.5.4/_index.yaml +++ /dev/null @@ -1,13 +0,0 @@ -changes: "### BREAKING CHANGES\n\n* `NextflowPlatform`: The default caching mechanism\ - \ is now what Nextflow uses as default. In order to replicate earlier\n caching,\ - \ `cache: deep` should be specified in the Viash config file.\n\n### NEW FEATURES\n\ - \n* `NextflowPlatform`: Added `cache` directive to specify the typing of caching\ - \ to be performed.\n\n" -date: '2021-09-20' -subtitle: Added cache directive to specify the typing of caching to be performed for - the Nextflow platform -version: 0.5.4 -whats_new: "\nA cache type can now be specified in the config file for the Nextflow\ - \ platform. Previously this was hardcoded to be `deep`, but the default caching\ - \ method is now `default`. \nTo use deep caching again, add this to your config\ - \ file:\n\n ```yaml\n cache: deep\n ```\n\n" diff --git a/blog/posts/viash-0.5.5/_index.yaml b/blog/posts/viash-0.5.5/_index.yaml deleted file mode 100644 index 36852b00..00000000 --- a/blog/posts/viash-0.5.5/_index.yaml +++ /dev/null @@ -1,17 +0,0 @@ -changes: "### BREAKING CHANGES\n\n* `Functionality`: The resources dir no longer automatically\ - \ added to the PATH variable. \n To alter this behaviour, set `.functionality.add_resources_to_path`\ - \ to `true`.\n\n### MINOR CHANGES\n\n* Bash Script: only define variables which\ - \ have values.\n\n* CSharp Test Component: Change Docker image to `dataintuitive/dotnet-script`\ - \ to have more control over the lifecycle of \n versioned tags.\n\n* Updated Code\ - \ of Conduct from v2.0 to v2.1.\n\n### BUG FIXES\n\n* Viash namespace: Fix incorrect\ - \ output path when the parent directory of a Viash component is not equal to the\ - \ value of\n `.functionality.name`.\n\n" -date: '2021-12-17' -subtitle: Resources dir no longer added to PATH automatically and minor changes -version: 0.5.5 -whats_new: "\nThe resources directory is no longer added to the PATH variable by default.\ - \ You can re-enable this behaviour by setting add_resources_to_path to `true` in\ - \ the functionality part of the config file. \nHere's a snippet of a config file\ - \ to illustrate this:\n\n ```yaml\n functionality:\n name: example_component\n\ - \ description: Serve as a simple example.\n add_resources_to_path: true\n\ - \ ...\n ```\n\n" diff --git a/blog/posts/viash-0.5.6/_index.yaml b/blog/posts/viash-0.5.6/_index.yaml deleted file mode 100644 index be1abac9..00000000 --- a/blog/posts/viash-0.5.6/_index.yaml +++ /dev/null @@ -1,26 +0,0 @@ -changes: "### BREAKING CHANGES\n\n* `BashWrapper`: Forbidden flags `-v`, `--verbose`,\ - \ `--verbosity` have been renamed to `---v`, `---verbose`, `---verbosity`.\n\n###\ - \ MINOR CHANGES\n\n* Set version of helper scripts to the same version as Viash.\n\ - \n* `DockerPlatform`: Produce helpful warning message when Docker image can't be\ - \ found remotely (#94).\n\n* `DockerPlatform`: Produce helpful error message when\ - \ Docker isn't installed or the daemon is not running (#94 bis).\n\n### BUG FIXES\n\ - \n* `viash_install`:\n - Passing Viash path as a string instead of as a file to\ - \ ensure the path is not converted to an absolute path\n - Switch from Docker backend\ - \ to a Native backend, 'unzip' and 'wget' are required.\n - Correctly set the log\ - \ file for viash_test.\n \n* `DockerPlatform`: Added sleep workaround to avoid\ - \ concurrency issue where a file is executed to\n build docker containers but apparently\ - \ still in the process of being written.\n \n* `DockerPlatform`: Fix order issue\ - \ of ---verbose flag in combination with ---setup, allowing to run \n `viash run\ - \ config.vsh.yaml -- ---setup cb ---verbose` and actually get output.\n \n\n" -date: '2022-02-03' -subtitle: Forbidden Bash flags have been renamed -version: 0.5.6 -whats_new: ' - - * Viash can now be installed without Docker needing to be installed on your system. - You do need `unzip` and `wget` to complete the installation. - - * The Docker related messages are more user friendly now. - - - ' diff --git a/blog/posts/viash-0.5.7/_index.yaml b/blog/posts/viash-0.5.7/_index.yaml deleted file mode 100644 index 1d144fa1..00000000 --- a/blog/posts/viash-0.5.7/_index.yaml +++ /dev/null @@ -1,24 +0,0 @@ -changes: "### BREAKING CHANGES\n\n* `viash config`: An argument's example now needs\ - \ to be of the same type as the argument itself. \n For example, `[ type: integer,\ - \ name: foo, example: 10 ]` is valid, whereas \n `[ type: integer, name: foo, example:\ - \ bar ]` is not, as 'bar' cannot be cast to an integer.\n\n### NEW FUNCTIONALITY\n\ - \n* `viash config inject`: A command for inserting a Viash header into your script.\n\ - \n* `YumRequirement`: Added a requirement setup for installing through yum. Example:\n\ - \ `setup: [ [ type: yum, packages: [ wget] ] ]`\n\n* `DockerRequirement`: Allow\ - \ using copy and add instructions. Example:\n `setup: [ [ type: docker, add: [\ - \ \"http://foo.bar .\" ]]]`\n\n### BUG FIXES\n\n* `ViashTest`: Fix verbosity passthrough.\n\ - \n* `--help`: Fix repeated usage flag when printing the help.\n\n" -date: '2022-02-16' -subtitle: Argument examples need to be of the same type as the argument itself -version: 0.5.7 -whats_new: "\nExamples for arguments now need to be of the same type as the argument\ - \ itself. You can't provide an `integer` for a `string`-based argument for example.\ - \ \nA handy new command has been added: `viash config inject`. This can be used\ - \ to inject a Viash header into a script based on the arguments of the config file.\n\ - \nThere have been some improvements to the Docker platform as well. \nYou can now\ - \ add yum packages as a requirement:\n\n ```yaml\n platforms:\n - type: docker\n\ - \ image: bash:latest\n setup:\n - type: yum\n packages:\ - \ [ wget ]\n ```\n\nYou can now include ADD and COPY instructions in the config\ - \ file:\n\n ```yaml\n platforms:\n - type: docker\n image: bash:latest\n\ - \ setup:\n - type: docker\n add: [ \"http://foo.bar .\" ]\n\ - \ ```\n\n" diff --git a/blog/posts/viash-0.5.8/_index.yaml b/blog/posts/viash-0.5.8/_index.yaml deleted file mode 100644 index 87cc5b00..00000000 --- a/blog/posts/viash-0.5.8/_index.yaml +++ /dev/null @@ -1,32 +0,0 @@ -changes: "### NEW FUNCTIONALITY\n\n* `DockerPlatform`: Allow defining a container's\ - \ organisation. Example:\n ```yaml\n - type: docker\n registry: ghcr.io\n\ - \ organisation: viash-io\n image: viash\n tag: \"1.0\"\n target_registry:\ - \ ghcr.io\n target_organization: viash-io\n ```\n\n* `DockerRequirement`:\ - \ Add label instructions. Example:\n `setup: [ [ type: docker, label: [ \"foo BAR\"\ - \ ]]]`\n\n* `Config`: In specific places, allow parsing a value as a list of values.\ - \ Fixes #97.\n This mostly applies to list values in `DockerPlatform`, but also\ - \ to author roles.\n Examples:\n ```yaml\n functionality:\n name: foo\n \ - \ authors:\n - name: Alice\n role: author # can be a string or a list\n\ - \ platforms:\n - type: docker\n port: \"80:80\" # can be a string or a\ - \ list\n setup:\n - type: r\n packages: incgraph # can be a\ - \ string or a list\n ```\n \n### BREAKING CHANGES\n\n* `viash test`: This command\ - \ doesn't automatically add the resources dir to the path.\n\n### BUG FIXES\n\n\ - * `Functionality`: Fix `.functionality.add_resources_to_path` not being picked up\ - \ correctly.\n\n* `AptRequirement`: Set `DEBIAN_FRONTEND=noninteractive` by default.\ - \ This can be turned off by specifying:\n ```yaml\n - type: apt\n packages:\ - \ [ foo, bar ]\n interactive: true\n ```\n\n### MINOR CHANGES\n\n* `Main`:\ - \ Slightly better error messages when parsing of viash yaml file fails.\n Before:\n\ - \ ```\n $ viash test src/test/resources/testbash/config_failed_build.vsh.yaml\ - \ \n Exception in thread \"main\" DecodingFailure(Unexpected field: [package];\ - \ valid fields: packages, interactive, type, List(DownField(apt), DownArray, DownField(platforms)))\n\ - \ ```\n \n After:\n ```\n $ viash test src/test/resources/testbash/config_failed_build.vsh.yaml\ - \ \n Error parsing 'file:/path/to/viash/src/test/resources/testbash/config_failed_build.vsh.yaml'.\ - \ Details:\n Unexpected field: [package]; valid fields: packages, interactive,\ - \ type: DownField(apt),DownArray,DownField(platforms)\n ```\n\n" -date: '2022-02-28' -subtitle: Allow defining a Docker image organization, and single values can be used - in place of lists -version: 0.5.8 -whats_new: ' - - ' diff --git a/blog/posts/viash-0.5.9/_index.yaml b/blog/posts/viash-0.5.9/_index.yaml deleted file mode 100644 index 4aa0d575..00000000 --- a/blog/posts/viash-0.5.9/_index.yaml +++ /dev/null @@ -1,24 +0,0 @@ -changes: "### NEW FEATURES\n\n* `viash run`: A long running Viash component can be\ - \ interrupted by pressing \n CTRL-C or by sending it an `INT` or `SIGINT` signal.\n\ - \n* `DockerPlatform`: Automatically add a few labels based on metadata to Dockerfile.\n\ - \n* `DockerPlatform`: Added value `target_image_source` for setting the source of\ - \ \n the target image. This is used for defining labels in the dockerfile.\n Example:\n\ - \ ```yaml\n target_image_source: https://github.com/foo/bar\n ```\n\n### MINOR\ - \ CHANGES\n\n* `viash ns list`: Added `--format yaml/json` argument to be able to\ - \ return the\n output as a json as well. Useful for when `jq` is installed but\ - \ `yq` is not. Example:\n ```\n viash ns list -p docker -f json | jq '.[] |\ - \ .info.config'\n ```\n\n* `viash config view`: Same as above.\n\n### DEPRECATION\n\ - \n* `CLI`: Deprecated `-P` flag use `-p` intead.\n\n* `DockerPlatform`: Deprecated\ - \ `version` value.\n\n" -date: '2022-03-12' -subtitle: Allow interrupting Viash components -version: 0.5.9 -whats_new: ' - - The biggest change in this release is that long running Viash components (VS Code - server or R Studio server for example) can now be interrupted by pressing CTRL-C - or by sending it an `INT` or `SIGINT` signal. Before this release, you had to manually - stop the Docker container to get the component to terminate. - - - ' diff --git a/blog/posts/viash-0.6.0/_index.yaml b/blog/posts/viash-0.6.0/_index.yaml deleted file mode 100644 index 906a8a10..00000000 --- a/blog/posts/viash-0.6.0/_index.yaml +++ /dev/null @@ -1,102 +0,0 @@ -changes: "### BREAKING CHANGES\n\n* `NextflowPlatform`: `variant: vdsl3` is now the\ - \ default NextflowPlatform. `variant: legacy` has been deprecated.\n\n* `Functionality`:\ - \ Fields `.inputs` and `.outputs` has been deprecated. Please use `.argument_groups`\ - \ instead (#186).\n Before:\n ```yaml\n functionality:\n inputs:\n -\ - \ name: \"--foo\"\n outputs:\n - name: \"--bar\"\n ```\n Now:\n ```yaml\n\ - \ functionality:\n argument_groups:\n - name: Inputs\n arguments:\n\ - \ - name: \"--foo\"\n type: file\n - name: Outputs\n \ - \ arguments:\n - name: \"--bar\"\n type: file\n \ - \ direction: output\n ```\n\n* Passing strings to an argument group's arguments\ - \ has been deprecated. Please simply copy the argument itself into the argument\ - \ group (#186).\n Before:\n ```yaml\n functionality:\n arguments:\n -\ - \ name: \"--foo\"\n type: file\n - name: \"--bar\"\n type: file\n\ - \ direction: output\n argument_groups:\n - name: Inputs\n \ - \ arguments: [ foo ]\n - name: Outputs\n arguments: [ bar ]\n ```\n\ - \ Now:\n ```yaml\n functionality:\n argument_groups:\n - name: Inputs\n\ - \ arguments:\n - name: \"--foo\"\n type: file\n \ - \ - name: Outputs\n arguments:\n - name: \"--bar\"\n \ - \ type: file\n direction: output\n ```\n\n### NEW FUNCTIONALITY\n\n\ - * Allow setting the number of processes and memory limit from within the Viash config,\ - \ \n as well as a list of required commands. Example:\n\n ```yaml\n functionality:\n\ - \ name: foo\n requirements:\n cpus: 10\n memory: 10GB\n commands: [ bash,\ - \ r, perl ]\n ```\n \n You can override the default requirements at runtime:\n\ - \n - `./foo ---cpus 4 ---memory 100PB` (for NativePlatform or DockerPlatform)\n\ - \ - By adding `process.cpus = 4` and `process.memory \"100 PB\"` to a nextflow.config\ - \ (for NextflowPlatform)\n\n This results the following meta variables to be injected\ - \ into a script:\n\n - `meta_cpus` (in Bash) or `meta[\"cpus\"]` (in any other\ - \ language): Number of processes the script is allowed to spawn.\n - `meta_memory_b`\ - \ (in Bash) or `meta[\"memory_b\"]` (in any other language): Amount of memory the\ - \ script is allowed to allocate, in bytes.\n - `meta_memory_kb` (in Bash) or `meta[\"\ - memory_kb\"]` (in any other language): Same but in kilobytes, rounded up.\n - `meta_memory_mb`\ - \ (in Bash) or `meta[\"memory_mb\"]` (in any other language): Same but in megabytes,\ - \ rounded up.\n - `meta_memory_gb` (in Bash) or `meta[\"memory_gb\"]` (in any other\ - \ language): Same but in gigabytes, rounded up.\n - `meta_memory_tb` (in Bash)\ - \ or `meta[\"memory_tb\"]` (in any other language): Same but in terabytes, rounded\ - \ up.\n - `meta_memory_pb` (in Bash) or `meta[\"memory_pb\"]` (in any other language):\ - \ Same but in petabytes, rounded up.\n \n* `viash ns exec`: Added a command for\ - \ executing arbitrary commands for all found Viash components.\n The syntax of\ - \ this command is inspired by `find . -exec echo {} \\;`.\n \n The following fields\ - \ are automatically replaced:\n * `{}` | `{path}`: path to the config file\n \ - \ * `{abs-path}`: absolute path to the config file\n * `{dir}`: path to the parent\ - \ directory of the config file\n * `{abs-dir}`: absolute path to the directory\ - \ of the config file\n * `{main-script}`: path to the main script (if any)\n \ - \ * `{abs-main-script}`: absolute path to the main script (if any)\n * `{functionality-name}`:\ - \ name of the component\n \n A command suffixed by `\\;` (or nothing) will execute\ - \ one command for each\n of the Viash components, whereas a command suffixed by\ - \ `+` will execute one\n command for all Viash components.\n\n* `ConfigMod`: Added\ - \ a `del(...)` config mod to be able to delete a value from the yaml. Example: `del(.functionality.version)`.\n\ - \n### MAJOR CHANGES\n\n* `Folder structure`: Adjusted the folder structure to correctly\ - \ reflect the the namespace change of viash from `com.dataintuitive.viash` to `io.viash`.\n\ - \n* `Functionality`: Reworked the `enabled` field from boolean to a `status` field\ - \ which can have the following statusses: `enabled`, `disabled` and `deprecated`.\n\ - \ When parsing a config file which has the `status` field set to `deprecated` a\ - \ warning message is displayed on stderr.\n Backwards for `enabled` is provided\ - \ where `enabled: true` => `status: enabled` and `enabled: false` => `status: false`.\ - \ The `enabled` field is marked deprecated.\n\n### MINOR CHANGES\n\n* `Resources`:\ - \ Handle edge case when no resources are specified in the `vsh.yaml` config file\ - \ and display a warning message.\n\n* `BashWrapper`: Add a warning when an argument\ - \ containing flags (e.g. `--foo`) is not recognized and will be handled as a positional\ - \ argument as this is likely a mistake.\n\n* `Functionality`: Add check to verify\ - \ there are no double argument names or short names in the config `vsh.yaml` declarations.\n\ - \n* `BashWrapper`: Add check to verify a parameter isn't declared twice on the CLI,\ - \ except in the case `multiple: true` is declared as then it's a valid use case.\n\ - \n* `BashWrapper`: For int min/max checking: use native bash functionality so there\ - \ is no dependency to `bc`.\n For double min/max checking: add fallback code to\ - \ use `awk` in case `bc` is not present on the system (most likely to happen when\ - \ running tests in a docker container).\n\n* `viash ns list/viash config view`:\ - \ Allow viewing the post-processed argument groups by passing the `--parse_argument_groups`\ - \ parameter.\n\n### TESTING\n\n* `ConfigMod`: Added unit tests for condition config\ - \ mods.\n\n* `MainTestDockerSuite`: Derive config alternatives from the base `vsh.yaml`\ - \ instead of adding the changes in separate files.\n This both reduces file clutter\ - \ and prevents having to change several files when there are updates in the config\ - \ format.\n\n* `GitTest`: Added unit tests for Git helper (PR #216).\n\n### BUG\ - \ FIXES\n\n* `csharp_script`, `javascript_script`, `python_script`, `r_script`,\ - \ `scala_script`: Make meta fields for `memory` and `cpus` optional.\n\n* `NextflowVdsl3Platform`:\ - \ Don't generate an error when `--publish_dir` is not defined and `-profile no_publish`\ - \ is used.\n\n* `Viash run`: Viash now properly returns the exit code from the executed\ - \ script.\n\n* `Git`: Fix incorrect metadata when git repository is empty (PR #216).\n\ - \n" -date: '2022-09-07' -subtitle: Nextflow VDSL3 is now the default, support for tracking memory and cpu requirements - more elegantly -version: 0.6.0 -whats_new: ' - - The first (major) release this year! The biggest changes are: - - - * Nextflow VDSL3 is now the default Nextflow platform, whereas the legacy Nextflow - platform has been deprecated. - - - * Support for tracking memory and cpu requirements more elegantly. - - - * Grouping arguments in groups more concisely. - - - * The addition of a `viash ns exec` command, to be able to execute commands on Viash - components more easily. - - - ' diff --git a/blog/posts/viash-0.6.1/_index.yaml b/blog/posts/viash-0.6.1/_index.yaml deleted file mode 100644 index 131bde6f..00000000 --- a/blog/posts/viash-0.6.1/_index.yaml +++ /dev/null @@ -1,56 +0,0 @@ -changes: "### BREAKING CHANGES\n\n* Deprecated usage `resources_dir` variable inside\ - \ scripts, use `meta[\"resources_dir\"]` instead (or `$meta_resources_dir` in Bash,\ - \ or `meta$resources_dir` in R).\n\n* Deprecated `meta[\"n_proc\"]` in favour for\ - \ `meta[\"cpus\"]`.\n\n### NEW FUNCTIONALITY\n\n* `viash ns exec`: Added two more\ - \ fields:\n\n - `{platform}`: the platform name (if applicable)\n - `{namespace}`:\ - \ the namespace of the component\n\n* `LongArgument`: Added support for 64-bit integers\ - \ with `type: long` as opposed to `type: integer` which are 32-bit integers.\n\n\ - ### MAJOR CHANGES\n\n* Allow passing integers/doubles/booleans to string parameters\ - \ (#225). Removed the 'Version' helper class.\n\n### MINOR CHANGES\n\n* `meta[\"\ - cpus\"]` is now an integer, `meta[\"memory_*\"]` are now longs (#224).\n\n* `DockerPlatform`:\ - \ Only store author names in the authors metadata.\n\n* `NextflowPlatform`: Only\ - \ store author names in the authors metadata.\n\n* `Argument[_]`: Turn `multiple_sep`\ - \ from `Char` into `String`.\n\n### INTERNAL CHANGES\n\n* All `meta[...]` variables\ - \ are now processed similar to `Argument[_]`s, instead of using custom code to convert\ - \ object types and detect Docker mounts.\n\n* `Escaper`: Make more generic Escaper\ - \ helper class.\n\n### DOCUMENTATION\n\n* Hardcoded URLs pointing to viash.io in\ - \ the documentation annotations were replaced with a new keyword system.\n\n* Replaced\ - \ references to \"DSL\" with \"Dynamic Config Modding\" in the `--help` output.\n\ - \n* Added an example for Ruby based Docker setups.\n\n### BUG FIXES\n\n* `viash\ - \ ns`: Reverse exit code outputs, was returning 1 when everything was OK and 0 when\ - \ errors were detected (PR #227).\n\n* `viash config inject`: Fix processing of\ - \ arguments when argument groups are defined (#231).\n\n* Fixed a few typos in the\ - \ CLI.\n\n* Fixed the formatting of `ns exec` documentation.\n\n* `VDSL3`: Fix stub\ - \ functionality.\n\n* `VDSL3`: Fix error during error message.\n\n* `viash test`:\ - \ Fix issue where `VIASH_TEMP` could not be a relative directory when running `viash\ - \ test` (#242).\n\n* `BashScript`, `CSharpScript`, `JavaScriptScript`, `PythonScript`,\ - \ `RScript`, `ScalaScript`: Fix quoting issues of certain characters (#113).\n\n\ - ### DEPRECATION\n\n* `NextflowPlatform`: Deprecate `--param_list_format` parameter.\n\ - \n### TESTING\n\n* `BashScript`, `CSharpScript`, `JavaScriptScript`, `PythonScript`,\ - \ `RScript`, `ScalaScript`: Implement more rigorous testing of which characters\ - \ are escaped.\n\n* `BashWrapper`: Escape usage of `multiple_sep`. This fixes various\ - \ checks and transformations not working when when `multiple_sep` is set to `\"\ - ;\"` (#235).\n\n" -date: '2022-10-03' -subtitle: Minor improvements in functionality -version: 0.6.1 -whats_new: ' - - This release contains mostly minor improvements of functionality released in Viash - 0.6.0. Most notably: - - - * Support was added for `type: long` arguments - - - * `meta["n_proc"]` has been renamed to `meta["cpus"]`. `meta["cpus"]` is now an - integer, whereas `meta["memory_*"]` are now longs. - - - * `viash ns exec` is able to recognise `{platform}` and `{namespace}` fields. - - - * And various bug fixes and improvements to documentation and unit testing. - - - ' diff --git a/blog/posts/viash-0.6.2/_index.yaml b/blog/posts/viash-0.6.2/_index.yaml deleted file mode 100644 index 938df3d4..00000000 --- a/blog/posts/viash-0.6.2/_index.yaml +++ /dev/null @@ -1,20 +0,0 @@ -changes: '### BUG FIXES - - - * `Git`: Strip credentials from remote repositories when retrieving the path. - - - * `VDSL3`: Allow optional output files to be `null`. - - - ' -date: '2022-10-11' -subtitle: Two bug fixes -version: 0.6.2 -whats_new: ' - - This is a quick release to push two bug fixes related to security and being able - to run Nextflow with optional output files. - - - ' diff --git a/blog/posts/viash-0.6.3/_index.yaml b/blog/posts/viash-0.6.3/_index.yaml deleted file mode 100644 index f0c0f5f6..00000000 --- a/blog/posts/viash-0.6.3/_index.yaml +++ /dev/null @@ -1,74 +0,0 @@ -changes: "### MAJOR CHANGES\n\n* `Config`: Made major internal changes w.r.t. how\ - \ config files are read and at which point a platform (native, docker, nextflow)\n\ - \ is applied to the functionality script. The only visible side effect is that\ - \ \n `viash ns list` will output each config only once instead of multiple times.\n\ - \n* `Functionality`: Structured annotation can be added to a functionality and its\ - \ arguments using the `info` field. Example:\n ```yaml\n functionality:\n name:\ - \ foo\n info:\n site: https://abc.xyz\n tags: [ one, two, three ]\n\ - \ arguments:\n - name: --foo\n type: string\n info:\n \ - \ foo: bar\n a:\n b:\n c\n ```\n\n### MINOR\ - \ CHANGES\n\n* `BashWrapper`: Allow printing the executor command by adding `---verbose\ - \ ---verbose` to a `viash run` or an executable.\n\n* `Testbenches`: Rework `MainBuildAuxiliaryNativeParameterCheck`\ - \ to create stimulus files and loop over the file from bash instead of looping natively.\n\ - \ This prevents creating thousands of new processes which would only test a single\ - \ parameter.\n Note this still calls the main script for each stimulus separately,\ - \ but that was the case anyway, only much much worse.\n\n* `Testbenches`: Split\ - \ some grouped test benches into slightly smaller test benches that group tested\ - \ functionality better.\n\n* `Annotations`: Complete the config schema annotations.\n\ - \ Make sure all arguments are documented.\n Added an annotation `internalFunctionality`\ - \ and `undocumented` for arguments that should not be documented.\n Added a testbench\ - \ that verifies that all arguments are in fact annotated, skipping those that are\ - \ not in the class constructor.\n Adds a hierarchy field in the `__this__` member\ - \ to list the relation of the own and parent classes.\n\n* `Testbenches`: Add exit\ - \ code to helper method `testMainWithStdErr`.\n\n* `Testbenches`: Add testbench\ - \ to verify viash underscore components (viash_build, viash_install, viash_push,\ - \ viash_skeleton, viash_test).\n\n* `Testbenches`: Update viash underscore component\ - \ tests to use `$meta_executable`.\n\n* `viash ns exec`: Allow choosing whether\ - \ the `{platform}` field should be filled in, based on the `--apply_platform` parameter.\n\ - \n### BUG FIXES\n\n* `DockerPlatform`: Remove duplicate auto-mounts (#257).\n\n\ - * `Underscore component tests`: Fix tests for `viash_skeleton` and `viash_test`\ - \ components.\n\n* `NextflowVDSL3Platform`: Fix 'Module scriptPath has not been\ - \ defined yet' error when Nextflow>=22.10 (#269).\n\n* `config inject`: Doesn't\ - \ work when `must_exist == true` (#273).\n\n* `RScript`: Fix compatibility issue\ - \ where the new character escaping in `r_script` required R>=4.0 (#275). Escaping\ - \ is now handled without\n using the new `r'(foo)'` notation.\n\n### DEPRECATION\n\ - \n* `DockerRequirements`: The `resources:` setting has been deprecated and will\ - \ be removed in Viash 0.7.0. Please use `copy:` instead.\n\n* `DockerRequirements`:\ - \ The `privileged:` setting has been deprecated and will be removed in Viash 0.7.0.\ - \ Please use `run_args: \"--privileged\"` instead.\n\n### EXPERIMENTAL FUNCTIONALITY\n\ - \n* `Config`: Any part of a Viash config can use inheritance to fill data (PR #271).\ - \ For example:\n Contents of `src/test/config.vsh.yaml`:\n ```yaml\n __inherits__:\ - \ ../api/base.yaml\n functionality:\n name: test\n resources:\n - type:\ - \ bash_script\n path: script.sh\n text: |\n echo Copying\ - \ $par_input to $par_output\n cp $par_input $par_output\n ```\n Contents\ - \ of `src/api/base.yaml`:\n ```yaml\n functionality:\n arguments:\n -\ - \ name: \"--input\"\n type: file\n - name: \"--output\"\n type:\ - \ file\n direction: output\n ```\n The resulting yaml will be:\n ```yaml\n\ - \ functionality:\n name: test\n arguments:\n - name: \"--input\"\n \ - \ type: file\n - name: \"--output\"\n type: file\n direction:\ - \ output\n resources:\n - type: bash_script\n path: script.sh\n \ - \ text: |\n echo Copying $par_input to $par_output\n cp\ - \ $par_input $par_output\n ```\n\n" -date: '2022-11-09' -subtitle: Quality-of-life improvements in Viash. -version: 0.6.3 -whats_new: ' - - This release features contains mostly quality of life improvements and some experimental - functionality. Most notably: - - - * `viash ns list` now only returns a config just once instead of once per platform. - - - * A functionality''s info field can contain any data structures. An `.info` field - was added to arguments as well. - - - * Bug fixes for using Viash with podman, Nextflow>=22.10 and R<4.0. - - - * Experimental support for inheriting from config partials. - - - ' diff --git a/blog/posts/viash-0.6.4/_index.yaml b/blog/posts/viash-0.6.4/_index.yaml deleted file mode 100644 index 4d5db649..00000000 --- a/blog/posts/viash-0.6.4/_index.yaml +++ /dev/null @@ -1,75 +0,0 @@ -changes: "### BREAKING CHANGES\n\n* Config: Viash configs whose filenames start with\ - \ a `.` are ignored (PR #291).\n\n* `viash build`: `--write_meta/-m` and `--meta/-m`\ - \ arguments have been removed. \n Instead, the `.config.vsh.yaml` file is always\ - \ created when building Viash components (PR #293).\n\n* `FileArgument`: Default\ - \ setting of `must_exist` was changed from `false` to `true`. \n As such, the component\ - \ will throw an error by default if an input file or output file\n is missing (PR\ - \ #295).\n\n* Config merging: `__inherits__` has been renamed to `__merge__`.\n\n\ - ### NEW FUNCTIONALITY\n\n* You can switch versions of Viash using the `VIASH_VERSION`\ - \ \n environment variable (PR #304)! Example:\n \n ```bash\n VIASH_VERSION=0.6.0\ - \ viash --version\n ```\n\n* Traceability: Running `viash build` and `viash test`\ - \ creates a `.config.vsh.yaml` file \n by default, which contains the processed\ - \ config of the component. As a side effect, \n this allows for reading in the\ - \ `.config.vsh.yaml` from within the component to learn \n more about the component\ - \ being tested (PR #291 and PR #293).\n\n* `FileArgument`: Added `create_parent`\ - \ option, which will check if the directory of an output\nfile exists and create\ - \ it if necessary (PR #295).\n\n### MINOR CHANGES\n\n* `viash run`, `viash test`:\ - \ When running or testing a component, Viash will add an extension\n to the temporary\ - \ file that is created. Before: `/tmp/viash-run-wdckjnce`, \n now: `/tmp/viash-run-wdckjnce.py`\ - \ (PR #302).\n\n* NextflowPlatform: Add `DataflowHelper.nf` as a retrievable resource\ - \ in Viash (PR #301).\n\n* NextflowPlatform: During a stubrun, argument requirements\ - \ are turned off and\n the `publishDir`, `cpus`, `memory`, and `label` directives\ - \ are also removed \n from the process (PR #301).\n\n* `NextflowPlatform`: Added\ - \ a `filter` processing argument to filter the incoming channel after \n the `map`,\ - \ `mapData`, `mapId` and `mapPassthrough` have been applied (PR #296).\n\n* `NextflowPlatform`:\ - \ Added the Viash config to the Nextflow module for later introspection (PR #296).\n\ - \ For example:\n ```groovy\n include { foo } from \"$targetDir/path/foo/main.nf\"\ - \n\n foo.run(filter: { tup ->\n def preferredNormalization = foo.config.functionality.info.preferred_normalization\n\ - \ tup.normalization_id == preferredNormalization\n })\n ```\n### BUG FIXES\n\ - \n* `BashWrapper`: Don't overwrite meta values when trailing arguments are provided\ - \ (PR #295).\n\n### EXPERIMENTAL FEATURES\n\n* Viash Project: Viash will automatically\ - \ search for a `_viash.yaml` file in the directory of \n a component and its parent\ - \ directories (PR #294).\n\n Contents of `_viash.yaml`:\n ```yaml\n source: src\n\ - \ target: target\n config_mods: |\n .platforms[.type == 'docker'].target_registry\ - \ := 'ghcr.io'\n .platforms[.type == 'docker'].target_organization := 'viash-io'\n\ - \ .platforms[.type == 'docker'].namespace_separator := '/'\n .platforms[.type\ - \ == 'docker'].target_image_source := 'https://github.com/viash-io/viash'\n ```\n\ - \n* Config merging: Allow specifying the order in which Viash will merge configs\ - \ (PR #289).\n If `.` is not in the list of inherited objects, it will be added\ - \ at the end.\n\n Contents of `config.vsh.yaml`:\n ```yaml\n functionality:\n\ - \ name: foo\n arguments:\n - __merge__: obj_input.yaml\n name:\ - \ \"--one\"\n - __merge__: [., obj_input.yaml]\n name: \"--two\"\n \ - \ - __merge__: [obj_input.yaml, .]\n name: \"--three\"\n ```\n\n Contents\ - \ of `obj_input.yaml`:\n ```yaml\n type: file\n name: --input\n description:\ - \ A h5ad file\n ```\n Output of `viash config view config.vsh.yaml` (stripped\ - \ irrelevant bits):\n ```yaml\n functionality:\n arguments:\n - type: \"\ - file\"\n name: \"--one\"\n description: \"A h5ad file\"\n - type: \"\ - file\"\n name: \"--input\"\n description: \"A h5ad file\"\n - type:\ - \ \"file\"\n name: \"--three\"\n description: \"A h5ad file\"\n ```\n\ - \ \n\n" -date: '2022-11-30' -subtitle: Add backwards compability by supporting switching to older Viash versions -version: 0.6.4 -whats_new: "\nThis release adds features related to managing Viash projects and \n\ - allows for better runtime introspection of Nextflow VDSL3 modules.\n\nThe most notable\ - \ changes are:\n\n* You can switch versions of Viash using the `VIASH_VERSION` \n\ - \ environment variable! Example:\n \n ```bash\n VIASH_VERSION=0.6.0 viash --version\n\ - \ ```\n\n More importantly, you can specify the version of Viash you want\n in\ - \ a project config. See below for more info.\n\n* Introducing Viash project config\ - \ files as an experimental feature.\n It allows storing project-related settings\ - \ in a `_viash.yaml` \n config file which you should store at the root of your\ - \ repository.\n Example:\n\n ```yaml\n viash_version: 0.6.4\n source: src\n\ - \ target: target\n config_mods: |\n .platforms[.type == 'docker'].target_registry\ - \ := 'ghcr.io'\n .platforms[.type == 'docker'].target_organization := 'viash-io'\n\ - \ .platforms[.type == 'docker'].namespace_separator := '/'\n .platforms[.type\ - \ == 'docker'].target_image_source := 'https://github.com/viash-io/viash'\n ```\n\ - \n* It's now possible to specify in which order Viash will merge\n Viash configs.\ - \ Example:\n\n ```yaml\n functionality:\n name: foo\n arguments:\n \ - \ - __merge__: obj_input.yaml\n name: \"--one\"\n - __merge__: [., obj_input.yaml]\n\ - \ name: \"--two\"\n - __merge__: [obj_input.yaml, .]\n name: \"\ - --three\"\n ```\n\nPlease take note of the following breaking changes:\n\n* Passing\ - \ non-existent paths to a Viash component will cause the \n component to generate\ - \ an error when no file or folder is found.\n Set `must_exist` to `false` to revert\ - \ to the previous behaviour.\n\n* The arguments `--write_meta/-w` and `--meta/-m`\ - \ no longer exist,\n because every `viash build/run/test` run will generate a \n\ - \ `.config.vsh.yaml` meta file.\n\n" diff --git a/blog/posts/viash-0.6.5/_index.yaml b/blog/posts/viash-0.6.5/_index.yaml deleted file mode 100644 index e7fd2d81..00000000 --- a/blog/posts/viash-0.6.5/_index.yaml +++ /dev/null @@ -1,13 +0,0 @@ -changes: "### BUG FIXES\n\n* `viash ns list`: When the `-p ` is defined,\ - \ filter the \n output by that platform.\n\n" -date: '2022-12-02' -subtitle: A small bugfix release -version: 0.6.5 -whats_new: ' - - A small update which fixes an issue with `viash ns list` that was - - introduced in Viash 0.6.3. - - - ' diff --git a/blog/posts/viash-0.6.6/_index.yaml b/blog/posts/viash-0.6.6/_index.yaml deleted file mode 100644 index baaadb4b..00000000 --- a/blog/posts/viash-0.6.6/_index.yaml +++ /dev/null @@ -1,16 +0,0 @@ -changes: '### BUG FIXES - - - * Don''t redirect stderr to stdout when switching Viash versions (#312). - - - ' -date: '2022-12-06' -subtitle: A small bugfix release -version: 0.6.6 -whats_new: ' - - This release fixes an issue where stderr was being redirected to stdout. - - - ' diff --git a/blog/posts/viash-0.6.7/_index.yaml b/blog/posts/viash-0.6.7/_index.yaml deleted file mode 100644 index d87cfdcd..00000000 --- a/blog/posts/viash-0.6.7/_index.yaml +++ /dev/null @@ -1,22 +0,0 @@ -changes: "### MINOR CHANGES\n\n* `NextflowPlatform`: Create directories during a stub\ - \ run when output path is a nested directory (PR #314).\n\n* Automatically generate\ - \ a warning for deprecated parameters while parsing a .viash.yaml configuration\ - \ file using the inline documentation deprecation annotations.\n\n* Add a \"planned\ - \ removal\" field in the deprecation annotations.\n\n* Add testbenches to verify\ - \ proper formatting of the deprecation versions and compare current version to the\ - \ planned removal version so no deprecated parameters get to stick around beyond\ - \ what was planned.\n\n* `NextflowPlatform`: Nextflow processes are created lazily;\ - \ that is, only when running\n a Nextflow workflow (PR #321).\n\n### BUG FIXES\n\ - \n* `NextflowPlatform`: Automatically split Viash config strings into strings of\ - \ \n length 65000 since the JVM has a limit (65536) on the length of string constants\ - \ (PR #323).\n\n" -date: '2022-12-14' -subtitle: A minor release with several QoL improvements -version: 0.6.7 -whats_new: ' - - Another minor release which contains several quality of life improvements for the - Nextflow VDSL3 platform, as well as automated warnings for deprecated functionality. - - - ' diff --git a/blog/posts/viash-0.7.0/_index.yaml b/blog/posts/viash-0.7.0/_index.yaml deleted file mode 100644 index 397a5dd5..00000000 --- a/blog/posts/viash-0.7.0/_index.yaml +++ /dev/null @@ -1,78 +0,0 @@ -changes: "### BREAKING CHANGES\n\n* Viash config: Previously deprecated fields are\ - \ now removed.\n - `functionality.inputs`: Use `arguments` or `argument_groups`\ - \ instead.\n - `functionality.outputs`: Use `arguments` or `argument_groups` instead.\n\ - \ - `functionality.tests`: Use `test_resources` instead. No functional difference.\n\ - \ - `functionality.enabled`: Use `status: enabled` instead.\n - `functionality.requirements.n_proc`:\ - \ Use `cpus` instead.\n - `platforms.DockerPlatform.privileged`: Add a `--privileged`\ - \ flag in `run_args` instead.\n - `platforms.DockerPlatform.apk`: Use `setup: [{\ - \ type: apk, packages: ... }]` instead.\n - `platforms.DockerPlatform.apt`: Use\ - \ `setup: [{ type: apt, packages: ... }]` instead.\n - `platforms.DockerPlatform.yum`:\ - \ Use `setup: [{ type: yum, packages: ... }]` instead.\n - `platforms.DockerPlatform.r`:\ - \ Use `setup: [{ type: r, packages: ... }]` instead.\n - `platforms.DockerPlatform.python`:\ - \ Use `setup: [{ type: python, packages: ... }]` instead.\n - `platforms.DockerPlatform.docker`:\ - \ Use `setup: [{ type: docker, run: ... }]` instead.\n - `platforms.DockerPlatform.docker.setup.resources`:\ - \ Use `setup: [{ type: docker, copy: ... }]` instead.\n - `platforms.NextflowLegacy`:\ - \ Use the Nextflow VDSL3 platform instead.\n - `functionality.ArgumentGroups`:\ - \ No longer supports strings referring to arguments in the `arguments:` section.\n\ - \ Instead directly put the arguments inside the argument groups.\n\n* `viash_install`:\ - \ The bootstrap script has been reworked in line with the project config introduced\ - \ in 0.6.4:\n\n * The default location for installing the Viash executable is\ - \ now `./viash` (was: `bin/viash`).\n * The new `viash_install` support `--output`\ - \ and `--tag`.\n * The various settings that existed in `viash_install` (organisation,\ - \ tag, ...) are moved to the project config.\n\n Please note that this new `viash_install`\ - \ bootstrap script can be run from the CLI using:\n\n ```\n curl -fsSL dl.viash.io\ - \ | bash\n ```\n The old `get.viash.io` is still available but points to the\ - \ version 0.6.7 version of this component and is deprecated.\n\n* `WorkflowHelper`:\ - \ `paramsToList`, `paramsToChannel` and `viashChannel` are now deprecated and will\ - \ be removed in a future release.\n\n* `viash (ns) build`: Change the default value\ - \ of the namespace separator in a Docker platform from `_` to `/`. \n Add `\".platforms[.type\ - \ == 'docker'].namespace_separator := '_'\"` to the project config `_viash.yaml`\ - \ to revert to the previous behaviour.\n\n### MAJOR CHANGES\n\n* `VDSL3`: now uses\ - \ the newly implemented `channelFromParams` and `preprocessInputs` instead of `viashChannel`.\n\ - \n### NEW FEATURES\n\n* `WorkflowHelper`: Added `preprocessInputs` and `channelFromParams`\ - \ to replace `paramsToList`, `paramsToChannel` and `viashChannel`. This refactor\ - \ allows processing parameters that are already in a Channel using `preprocessInputs`,\ - \ which is necessary when passing parameters from a workflow to a subworkflow in\ - \ a Nextflow pipeline.\n\n### MINOR CHANGES\n\n* `Main`: Capture build, setup and\ - \ push errors and output an exit code.\n\n* `File downloading`: Add check to pre-emptively\ - \ catch file errors (e.g. 404).\n\n* `Scala`: Updated to Scala 2.13 and updated\ - \ several dependencies.\n\n* `Main`: Improve `match` completeness in some edge cases\ - \ and throw exceptions where needed.\n\n* `Changelog`: Reformat the changelog to\ - \ a more structured format.\n For every release, there is now a date, title, and\ - \ summary.\n This both improves the changelog itself but can then also be used\ - \ to postprocess the CHANGELOG programmatically.\n\n* `VDSL3`: Add a default value\ - \ for `id` when running a VDSL3 module as a standalone pipeline.\n\n* `TestBenches`:\n\ - \ - Verify switching of Viash versions\n - Prepare ConfigDeriver by copying base\ - \ resources to the targetFolder. Use cases so far showed that it's always required\ - \ and it simplifies the usage.\n - Remove some old & unmaintained IntelliJ Idea\ - \ `editor-fold` tags. Given that the testbenches were split up, these were broken\ - \ but also no longer needed.\n - Add 2 testbenches for computational requirements\ - \ when running `viash run` or `viash test`.\n - Added tests for different values\ - \ for the `--id` and `--param_list` parameters of VDSL3 modules.\n\n* `viash test`:\ - \ Use `test` as a random tag during testing, instead of `test` plus a random string.\n\ - \n### BUG FIXES\n\n* `WorkflowHelper`: fixed where passing a relative path as `--param_list`\ - \ would cause incorrect resolving of input files.\n\n* `Testbenches`: Fix GitTest\ - \ testbench to correctly increment temporary folder naming and dispose them after\ - \ the test finishes.\n\n* `viash xxx url`: Fix passing a url to viash as the config\ - \ file to process. Add a short testbench to test principle functionality.\n\n* `Testbenches`:\ - \ Simplify `testr` container.\n\n* `Main`: Improve error reporting to the user in\ - \ some cases where files or folders can't be found. Depending on the thrown exception,\ - \ more or less context was given.\n\n* `VDSL3`: Create parent directory of output\ - \ files before starting the script.\n\n" -date: '2023-02-28' -subtitle: Major code cleanup and minor improvements to VDSL3 -version: 0.7.0 -whats_new: "\n* Default namespace separator has been changed from `_` to `/`. This\ - \ means \n Docker images will be named `///`\n\ - \ by default. For example, `ghcr.io/openpipelines-bio/mapping/cellranger_count`\n\ - \ instead of `ghcr.io/openpipelines-bio/mapping_cellranger_count`.\n\n* Removed\ - \ deprecated code of unused functionality to simplify code.\n - Shorthand notation\ - \ for specitying input/output arguments\n - Shorthand notation for specifying Docker\ - \ requirements\n - Legacy Nextflow platform\n\n* Improvements in VDSL3 and the\ - \ Nextflow Workflow Helper to make behaviour\n more predictable and fixing some\ - \ bugs in the meantime. Run the following\n to get access to the updated helpers:\n\ - \n ```bash\n WF_DIR=\"src/wf_utils\"\n [[ -d $WF_DIR ]] || mkdir -p $WF_DIR\n\ - \ viash export resource platforms/nextflow/ProfilesHelper.config > $WF_DIR/ProfilesHelper.config\n\ - \ viash export resource platforms/nextflow/WorkflowHelper.nf > $WF_DIR/WorkflowHelper.nf\n\ - \ viash export resource platforms/nextflow/DataflowHelper.nf > $WF_DIR/DataflowHelper.nf\n\ - \ ```\n\n* Improvements to test benches and several bug fixes.\n\n" diff --git a/blog/posts/viash-0.7.1/_index.yaml b/blog/posts/viash-0.7.1/_index.yaml deleted file mode 100644 index 9c84cdc6..00000000 --- a/blog/posts/viash-0.7.1/_index.yaml +++ /dev/null @@ -1,19 +0,0 @@ -changes: "### MINOR CHANGES\n\n* `DataflowHelper`: Add assertions and `def`s.\n\n\ - ### BUG FIXES\n\n* `VDSL3`: Only the first two elements from an event in a channel\ - \ are now passed to a process. This avoids calculating cache entries based on arguments\ - \ that are not used by the process, causing false-negative cache misses.\n\n* `config_schema`:\n\ - \ - Correct some incorrect markdown tags.\n - Add project config.\n - Correct\ - \ documentation/markdown tags to the correct order.\n - Add summary description\ - \ and example for 'resource' and 'argument', to be used on the reference website.\n\ - \ - Add documentation for the Nextflow directives.\n\n* `cli_schema`: Correct documentation/markdown\ - \ tags to the correct order.\n\n" -date: '2023-03-08' -subtitle: Minor improvements to VDSL3 and schema functionality. -version: 0.7.1 -whats_new: ' - - This is a minor release which improves caching in VDSL3 components and changes the - formats of the schema files for the Viash config and CLI. - - - ' diff --git a/blog/posts/viash-0.7.2/_index.yaml b/blog/posts/viash-0.7.2/_index.yaml deleted file mode 100644 index 27558138..00000000 --- a/blog/posts/viash-0.7.2/_index.yaml +++ /dev/null @@ -1,28 +0,0 @@ -changes: "### NEW FUNCTIONALITY\n\n* Resolve resource and merge paths starting with\ - \ a slash (`/`) as relative to the project directory (PR #380). To define absolute\ - \ paths (which is not recommended anyway), prefix the path with the `file://` protocol.\ - \ Examples:\n\n - `/foo` is a file or directory called `foo` in the current project\ - \ directory.\n - `file:/foo` is a file or directory called `foo` in the system\ - \ root.\n\n### MINOR CHANGES\n\n* `viash config view`: Do not modify (e.g. strip\ - \ empty fields) of the `.functionality.info` and `.functionality.arguments[].info`\ - \ fields (#386).\n\n### BUG FIXES\n\n* `ConfigMods`: Fix operator precedence issues\ - \ with conditions in the config mod parsers (PR #390).\n\n### INTERNAL CHANGES\n\ - \n* Clean up unused code (PR #380).\n\n* Move circe encoders/decoders for File and\ - \ Path from `io.viash.functionality.arguments` to `io.viash.helpers.circe` (PR #380).\n\ - \n* Store the project root directory (that is, the directory of the `_viash.yaml`)\ - \ in a ViashProject object (PR #380).\n\n* Tests: Reworked language tests to be\ - \ grouped in their own subfolder and split off the bash language test from the general\ - \ `testbash` folder (PR #381).\n\n* Tests: Add additional language tests for `viash\ - \ config inject` (PR #381).\n\n* Tests: Added test for `io.viash.helpers.IO` (PR\ - \ #380).\n\n\n" -date: '2023-04-17' -subtitle: Project-relative paths and improved metadata handling -version: 0.7.2 -whats_new: ' - - This update adds functionality to resolve paths starting with a slash as relative - to the project directory, improves handling of info metadata in the config, and - fixes to the operator precedence of config mods. - - - ' diff --git a/blog/posts/viash-0.7.3/_index.yaml b/blog/posts/viash-0.7.3/_index.yaml deleted file mode 100644 index 223a7e56..00000000 --- a/blog/posts/viash-0.7.3/_index.yaml +++ /dev/null @@ -1,23 +0,0 @@ -changes: '### BUG FIXES - - - * `DockerPlatform`: Fixed example in documentation for the `namespace_separator` - parameter (PR #396). - - - * `viash config view`: Resource parent paths should be directories and not file - (PR #398). - - - - ' -date: '2023-04-19' -subtitle: Minor bug fixes in documentation and config view -version: 0.7.3 -whats_new: ' - - Fix minor issues in the documentation and with the way parent paths of resources - are printed a config view. - - - ' diff --git a/blog/posts/viash-0.7.4/_index.yaml b/blog/posts/viash-0.7.4/_index.yaml deleted file mode 100644 index cfdb436c..00000000 --- a/blog/posts/viash-0.7.4/_index.yaml +++ /dev/null @@ -1,46 +0,0 @@ -changes: "### NEW FUNCTIONALITY\n\n* Add default labels in Nextflow config files that\ - \ set default values for cpu and memory settings (PR #412). Values are more or less\ - \ logarithmically spaced (1, 2, 5, 10, ...).\n\n* `Author`: Added `info` field to\ - \ authors. Deprecated `props` field (PR #423).\n\n* `viash config view` and `viash\ - \ ns list`: Set the `.info.output` path when a platform argument is provided.\n\n\ - * `viash ns exec`: Added two more fields:\n\n - `{output}`: path to the destination\ - \ directory when building the component\n - `{abs-output}`: absolute path to the\ - \ destination directory when building the component\n\n* `DockerPlatform`: Add `entrypoint`\ - \ and `cmd` parameters to the docker platform config that allows overriding the\ - \ default docker container settings (PR #432).\n\n### MINOR CHANGES\n\n* `Nextflow\ - \ VDSL3`:\n - Add profiles to the Nextflow Config file when the main script is\ - \ a `NextflowScript` (#408).\n - Add a `script` parameter in Nextflow Config file\ - \ to add a single string or list of strings to the `nextflow.config` (PR #430).\n\ - \n* `Scripts`: Remove the `entrypoint` parameter for all script types except `NextflowScript`\ - \ (#409). All these scripts had to check individually whether the parameter was\ - \ unset, now it can be done in the `Script` apply method.\n\n* `schema export`:\n\ - \ - Restructure Nextflow-Directives, -Auto and -Config into a `nextflowParameters`\ - \ group (PR #412). Previously only NextflowDirectives was exposed.\n - Restructure\ - \ the format to group authors & computational requirements together with functionality\ - \ (PR #426).\n - Restructure the Viash Config and Project Config pages under a\ - \ `config` category (PR #426).\n - Add references in Functionality and Nextflow\ - \ VDSL3 to the new documentation pages (PR #426).\n - Add description and/or examples\ - \ for platforms and requirements (PR #428).\n\n### BUG FIXES\n\n* `viash config\ - \ inject`: Fix an empty line being added at the script start for each time `viash\ - \ config inject` was run (#377).\n\n* `WorkflowHelper`: Fixed an issue where passing\ - \ a remote file URI (for example `http://` or `s3://`) as `param_list` caused `No\ - \ such file` errors.\n\n* `BashWrapper`: Fix escaping of the included script where\ - \ a line starting with a pipe character with optional leading spaces is stripped\ - \ of the leading spaces and pipe character.\n This was quite unlikely to happen\ - \ except when `viash config inject` was called on a Nextflow Script, which lead\ - \ to no real config code being injected however workflows were getting corrupted.\ - \ (#421)\n\n* `Deprecation testbench`: Add missing classes to be checked (PR #426).\n\ - \n" -date: '2023-05-31' -subtitle: Minor bug fixes and minor improvements to VDSL3 -version: 0.7.4 -whats_new: ' - - Some small fixes and consistency improvements. - - A few Quality of Life improvements were made e.g. to override the Docker `entrypoint` - when working with Nextflow and providing default labels when building a Nextflow - workflow. - - - ' diff --git a/blog/posts/viash-0.7.5/index.qmd b/blog/posts/viash-0.7.5/index.qmd new file mode 100644 index 00000000..d33c2e05 --- /dev/null +++ b/blog/posts/viash-0.7.5/index.qmd @@ -0,0 +1,96 @@ +--- +title: "Viash 0.7.5" +subtitle: "Minor breaking changes and new features" +date: "2023-08-11" +categories: [ New Release ] +author: Viash Team +--- + +## What's new? + +This release contains minor breaking change due to deprecated or outdated functionality being removed. + +New functionality includes: + + - Export a JSON schema for the Viash config with `viash export json_schema` + + - Export a Bash or Zsh autocomplete script with `viash export cli_autocomplete` + + - Nextflow VDSL3 modules now have a `fromState` and `toState` argument to allow for better control of the data that gets passed to the module and how the state is managed in a Nextflow workflow. + +## Full changelog + +### BREAKING CHANGES + +* `viash export cli_schema`: Added `--format yaml/json` argument, default format is now a YAML (PR #448). + +* `viash export config_schema`: Added `--format yaml/json` argument, default format is now a YAML (PR #448). + +* `NextflowLegacyPlatform`: Removed deprecated code (PR #469). + +* `viash_*`: Remove legacy viash_build, viash_test and viash_push components (PR #470). + +* `ComputationalRequirements`, `Functionality`, `DockerPlatform`, `DockerRequirements`: Remove documentation of removed fields (PR #477). + +### NEW FUNCTIONALITY + +* `viash export json_schema`: Export a json schema derived from the class reflections and annotations already used by the `config_schema` (PR #446). + +* `viash export config_schema`: Output `default` values of member fields (PR #446). + +* `CI`: Test support for different Java versions on GitHub Actions (PR #456). Focussing on LTS releases starting from 11, so this is 11 and 17. Also test latest Java version, currently 20. + +* `viash test` and `viash ns test`: add `--setup` argument to determine the docker build strategy before a component is tested (PR #451). + +* `viash export cli_autocomplete`: Export a Bash or Zsh autocomplete script (PR #465 & #482). + +* `help message`: Print the relevant help message of (sub-)command when `--help` is given as an argument instead of only printing the help message when it is the leading argument and otherwise silently disregarding it (initially added in PR #472, replaced by PR #496). This is a new feature implemented in Scallop 5.0.0. + +* `Logging`: Add a Logger helper class (PR #485 & #490). Allows manually enabling or disabling colorizing TTY output by using `--colorize`. Add provisions for adding debugging or trace code which is not outputted by default. Changing logging level can be changed with `--loglevel`. These CLI arguments are currently hidden. + +* `NextflowPlatform`: Nextflow VDSL3 modules now have a `fromState` and `toState` argument to allow for better control of the data that gets passed to the module and how the state is managed in a Nextflow workflow (#479, PR #501). + +### MINOR CHANGES + +* `PythonScript`: Pass `-B` to Python to avoid creating `*.pyc` and `*.pyo` files on importing helper functions (PR #442). + +* `viash config`: Special double values now support `+.inf`, `-.inf` or `.nan` values (PR #446 and PR #450). The stringified versions `"+.inf"`, `"-.inf"` or `".nan"` are supported as well. This is in line with the yaml spec. + +* `system environment variables`: Add wrapper around `sys.env` and provide access to specific variables (PR #457). Has advantages for documentation output and testbenches. + +* `testbench`: Added some minor testbenches to tackle missing coverage (PR #459, #486, #488, #489, #492 & #494). + +* `viash export config_schema`: Simplify file structure (PR #464). + +* `helpers.Format`: Add a helper for the Format helper object (PR #466). + +* `testbench`: Use config deriver to create config variants for testing (PR #498). This reduces the amount of config files that need to be maintained. + +### BUG FIXES + +* `viash config`: Validate Viash config Yaml files better and try to give a more informative error message back to the user instead of a stack trace (PR #443). + +* `viash ns build`: Fix the error summary when a setup or push failure occurs. These conditions were not displayed and could cause confusion (PR #447). + +* `testbench`: Fix the viash version switch test bench not working for newer Java versions (PR #452). + +* `malformed input exception`: Capture MalformedInputExceptions when thrown by reading files with invalid Ascii characters when unsupported by Java (PR #458). + +* `viash project file parsing`: Give a more informative message when the viash project file fails to parse correctly (PR #475). + +* `DockerPlatform`: Fix issue when mounting an input or output folder containing spaces (PR #484). + +* `Config mod`: Fix a config mod where the filter should execute multiple deletes (PR #503). + +### DOCUMENTATION + +* `NextflowPlatform`: Add documentation for the usage and arguments of a VDSL3 module (PR #501). + +### INTERNAL CHANGES + +* `NextflowVDSL3Platform`: Renamed to `NextflowPlatform` (PR #469). + +* Rename mentions of `NextFlow` to `Nextflow` (PR #476). + +* `Reference static pages`: Move `.qmd` files from the website to a local folder here; `docs/reference` (PR #504). This way we can track behaviour changes that need to be documented locally. + diff --git a/guide/nextflow_vdsl3/create-a-pipeline.qmd b/guide/nextflow_vdsl3/create-a-pipeline.qmd index a03f9bd0..e8c2ec51 100644 --- a/guide/nextflow_vdsl3/create-a-pipeline.qmd +++ b/guide/nextflow_vdsl3/create-a-pipeline.qmd @@ -1,6 +1,6 @@ --- title: Create a pipeline -order: 30 +order: 40 --- {{< include ../../_includes/_clone_template.qmd >}} @@ -52,6 +52,40 @@ Once everything is built, a new **target** directory has been created containing tree target ``` + +## Importing a VDSL3 module + +After building a VDSL3 module from A VDSL3 module can be imported just like any other Nextflow module. + +**Example:** + +```groovy +include { mymodule } from 'target/nextflow/mymodule/main.nf' +``` + + +## VDSL3 module interface + +VDSL3 modules are actually workflows which take one channel and emit one channel. It expects the channel events to be tuples containing an 'id' and a 'state': `[id, state]`, where `id` is a unique String and `state` is a `Map[String, Object]`. The resulting channel then consists of tuples `[id, new_state]`. + +**Example:** + +```groovy +workflow { + Channel.fromList([ + ["myid", [input: file("in.txt")]] + ]) + | mymodule +} +``` + +:::{.callout-note} +If the input tuple has more than two elements, the elements after the second element are passed through to the output tuple. +That is, an input tuple `[id, input, ...]` will result in a tuple `[id, output, ...]` after running the module. +For example, an input tuple `["foo", [input: file("in.txt")], "bar"]` will result in an output tuple `["foo", [output: file("out.txt")], "bar"]`. +::: + + ## Create a pipeline Below is a first Nextflow pipeline which uses just one VDSL3 module and with hard-coded input parameters (file1 and file2). @@ -92,26 +126,17 @@ HERE main.nf ``` -## VDSL3 module interface - -It's important to note what the interface of every VDSL3 module is. A VDSL3 module expects an input to be a tuple with the following elements: + -* `id` (`String`): A unique identifier used for tracking data objects and for ensuring output filenames are unique. -* `data` (`Map[String, Any]` or `File`): A named map (or dictionary) used to pass the module's input arguments. If the module only has a - single input file, the file itself can simply be passed. -* `...` (`Any*`): Any other elements in the tuple simply pass through the module without being altered in any way. For this reason, it is often referred to as the "passthrough" objects. +## Customizing VDSL3 modules on the fly -In turn, a VDSL3 module will return a tuple with the same interface, except that the input data object has been replaced with the output data: +Usually, Nextflow processes are quite static objects. For example, changing its directives can be quite tricky. -* `id` (`String`): The identifier from the input tuple. -* `data` (`Map[String, Any]` or `File`): A named map (or dictionary) containing the module's output files. **Important**: If the module only has a single output file, the file itself will be returned. -* `...` (`Any*`): The passthrough objects from the input tuple (if any). +The `un()` function is a unique feature for every VDSL3 module which allows dynamically altering the behaviour of a module from within the pipeline. For example, we use it to set the `publishDir` directive to `"output/"` so the output of that step in the pipeline will be stored as output. -## What is `.run()`? +See the [reference documentation](/reference/nextflow_vdsl3/import_module.qmd#customizing-vdsl3-modules-on-the-fly) for a complete list of arguments of `.run()`. -Usually, Nextflow processes are quite static objects. For example, changing its directives can be quite tricky. -The `run()` function is a unique feature for every VDSL3 module which allows dynamically altering the behaviour of a module from within the pipeline. In this case, we use it to set the `publishDir` directive to `"output/"` so the output of that step in the pipeline will be stored as output. ## Run the pipeline diff --git a/guide/nextflow_vdsl3/create-a-module.qmd b/guide/nextflow_vdsl3/create-and-use-a-module.qmd similarity index 65% rename from guide/nextflow_vdsl3/create-a-module.qmd rename to guide/nextflow_vdsl3/create-and-use-a-module.qmd index 99b11bb8..f8bf35ec 100644 --- a/guide/nextflow_vdsl3/create-a-module.qmd +++ b/guide/nextflow_vdsl3/create-and-use-a-module.qmd @@ -1,6 +1,6 @@ --- -title: Create a module -order: 20 +title: Create and use a module +order: 10 --- @@ -52,45 +52,41 @@ pwalk(langs, function(id, label, example_config, ...) { ``` ::: -## Build the VDSL3 module +## Generating a VDSL3 module We will now turn the Viash component into a VDSL3 module. By default, the `viash build` command will select the first platform in the list of platforms. To select the `nextflow` platform, use the `--platform nextflow` argument, or `-p nextflow` for short. -::: {.panel-tabset} ```{r viash-build-nxf} #| echo: false #| output: asis -langs <- langs %>% filter(id == "bash") -pwalk(langs, function(id, label, config_path, script_path, ...) { - qrt( - "## {% label %} - | - |```{bash build-example} - |viash build config.vsh.yaml -o target -p nextflow - |``` - | - |This will generate a Nextflow module in the `target/` directory: - | - |```{bash view-tree} - |tree target - |``` - |", - .dir = paste0(temp_dir, "/", id) - ) -}) +id <- "bash" +qrt( + "```{bash build-example} + |viash build config.vsh.yaml -o target -p nextflow + |``` + | + |This will generate a Nextflow module in the `target/` directory: + | + |```{bash view-tree} + |tree target + |``` + |", + .dir = paste0(temp_dir, "/", id) +) ``` -::: -This `main.nf` file is both a **standalone Nextflow pipeline** and a module which can be used as **part of another pipeline**. +This `main.nf` file is both a [**standalone Nextflow pipeline**](run-a-module.qmd) and a module which can be imported as [**part of another pipeline**](import-a-module.qmd). :::{.callout-tip} -You can also use the `viash ns build` command to build all of the platforms in one go. Give it a try! More information in the following section. +In larger proejcts it's recommended to use the [`viash ns build`](/reference/cli/ns_build.qmd) command to [build all of the components](/guide/project/batch-processing.qmd) in one go. Give it a try! ::: -## Module as a standalone pipeline +## Running a module as a standalone pipeline -When VDSL3 modules are used as a standalone pipeline, you need to specify the input parameters and a `--publish_dir` parameter, -as Nextflow will automatically choose the parameter names of the output files. + +Unlike typical Nextflow modules, VDSL3 modules can actually be used as a standalone pipeline. + +To run a VDSL3 module as a standalone pipeline, you need to specify the input parameters and a `--publish_dir` parameter, as Nextflow will automatically choose the parameter names of the output files. ```{r nextflow-run, echo=FALSE, output="asis"} id <- "bash" @@ -177,33 +173,58 @@ Instead of a YAML, you can also pass a JSON or a CSV to the `--param_list` parameter. ::: + ## Module as part of a pipeline This module can also be used as part of a Nextflow pipeline. Below is a short preview of what this looks like. ```groovy -import { example_bash } from "target/main.nf" - -Channel.fromList([ - ["sample1", file("sample1.txt")], - ["sample2", file("sample2.txt")], - ["sample3", file("sample3.txt")] -]) - | view { it -> "input: $it" } - | example_bash - | view { it -> "output: $it" } +include { mymodule1 } from 'target/nextflow/mymodule1/main.nf' +include { mymodule2 } from 'target/nextflow/mymodule2/main.nf' + +workflow { + Channel.fromList([ + [ + // a unique identifier for this tuple + "myid", + // the state for this tuple + [ + input: file("in.txt"), + module1_k: 10, + module2_k: 4 + ] + ] + ]) + | mymodule1.run( + // use a hashmap to define which part of the state is used to run mymodule1 + fromState: [ + input: "input", + k: "module1_k" + ], + // use a hashmap to define how the output of mymodule1 is stored back into the state + toState: [ + module1_output: "output" + ] + ) + | mymodule2.run( + // use a closure to define which data is used to run mymodule2 + fromState: { id, state -> + [ + input: state.module1_output, + k: state.module2_k + ] + }, + // use a closure to return only the output of module2 as a new state + toState: { id, output, state -> + output + }, + auto: [ + publish: true + ] + ) +} ``` We will discuss building pipelines with VDSL3 modules in more detail in [Create a pipeline](create-a-pipeline.qmd). -## Improvements over standard Nextflow modules - -* No need to write any Nextflow Groovy code, just your script and the Viash config. -* VDSL3 module are also standalone pipelines. -* Help documentation is automatically generated. -* Standardized interface for passing parameter lists. -* Automatically uses the Docker platform's container. - - -{{< include ../../_includes/_prune_all_images.qmd >}} \ No newline at end of file diff --git a/guide/nextflow_vdsl3/index.qmd b/guide/nextflow_vdsl3/index.qmd index 3ccaf1d7..24e19bc2 100644 --- a/guide/nextflow_vdsl3/index.qmd +++ b/guide/nextflow_vdsl3/index.qmd @@ -2,4 +2,24 @@ title: Nextflow VDSL3 order: 30 hidden: true ---- \ No newline at end of file +--- + +Nextflow is a highly popular and widely-used workflow manager in computational biology, featuring outstanding portability, reproducibility and scalability. However, while Nextflow's advantages are impressive, developing a Nextflow pipeline can be challenging, requiring significant domain knowledge and verbose code that is labour-intensive. Fortunately, Viash provides a solution to the barriers of Nextflow pipeline development. + +Viash can help developers wrap their code into a state-of-the-art Nextflow script called a VDSL3 module. As we will demonstrate in the remainder of this guide, VDSL3 is effectively a separate DSL layer on top of Nextflow enabled by Viash, hence it is called Viash + Nextflow DSL 3, or VDSL3 for short. VDSL3's benefits extend beyond Nextflow pipeline development, including reusability, test-driven development, separation of concerns, and continuous testing. + +You can use Viash to speed up or replace your pipeline development processes in the following steps: + +* [Use Viash to generate VDSL3 modules](create-and-use-a-module.qmd#build-the-vdsl3-module) +* [Run a module as a standaline pipeline](create-and-use-a-module.qmd#running-a-module-as-a-standalone-pipeline) +* [Import a VDSL3 module](create-and-use-a-module.qmd#module-as-part-of-a-pipeline) +* [Create a Nextflow workflow](create-a-pipeline.qmd) using one or more modules + + +## Improvements of VDSL3 modules over standard Nextflow modules + +* No need to write any Nextflow Groovy code, just your script and the Viash config. +* VDSL3 module are also standalone pipelines. +* Help documentation is automatically generated. +* Standardized interface for passing parameter lists. +* Automatically uses the Docker platform's container. diff --git a/guide/nextflow_vdsl3/introduction.qmd b/guide/nextflow_vdsl3/introduction.qmd deleted file mode 100644 index 547b14f2..00000000 --- a/guide/nextflow_vdsl3/introduction.qmd +++ /dev/null @@ -1,11 +0,0 @@ ---- -title: Introduction -description: What is VDSL3? -order: 10 ---- - -Nextflow is a highly popular and widely-used workflow manager in computational biology, featuring outstanding portability, reproducibility and scalability. However, while Nextflow's advantages are impressive, developing a Nextflow pipeline can be challenging, requiring significant domain knowledge and verbose code that is labour-intensive. Fortunately, Viash provides a solution to the barriers of Nextflow pipeline development. - -Viash can help developers wrap their code into a state-of-the-art Nextflow script called a VDSL3 module. As we will demonstrate in the remainder of this guide, VDSL3 is effectively a separate DSL layer on top of Nextflow enabled by Viash, hence it is called Viash + Nextflow DSL 3, or VDSL3 for short. VDSL3's benefits extend beyond Nextflow pipeline development, including reusability, test-driven development, separation of concerns, and continuous testing. - -In the following sections, we'll show how to use build Nextflow modules from Viash components and how to put them together in a pipeline. \ No newline at end of file diff --git a/index.qmd b/index.qmd index 71388afb..765700ef 100644 --- a/index.qmd +++ b/index.qmd @@ -101,7 +101,7 @@ Every executable built by Viash comes with an intuitive CLI automatically. This Viash generates Nextflow modules from your component to create portable and scalable data pipelines that run platform-independent. ::: {.learn-more} -[Learn more »](guide/nextflow_vdsl3/introduction.qmd) +[Learn more »](guide/nextflow_vdsl3/index.qmd) ::: ::: diff --git a/quickstart/index.qmd b/quickstart/index.qmd index d8695d82..cce61d87 100644 --- a/quickstart/index.qmd +++ b/quickstart/index.qmd @@ -140,6 +140,6 @@ Our comprehensive [guide](/guide/) and [reference documentation](/reference/) is * [Ensuring reproducibility and designing customised Docker images](/guide/component/add-dependencies.qmd) * [Ensuring code reliability with unit testing for Viash](/guide/component/unit-testing.qmd) * [Streamlining your workflow by performing batch operations on Viash projects](/guide/project/batch-processing.qmd) -* [Building Nextflow pipelines using Viash components](/guide/nextflow_vdsl3/introduction.qmd) +* [Building Nextflow pipelines using Viash components](/guide/nextflow_vdsl3/index.qmd) So, get ready to enhance your skills and create outstanding solutions with Viash! \ No newline at end of file diff --git a/reference/cli/_build.yaml b/reference/cli/_build.yaml deleted file mode 100644 index 209241fe..00000000 --- a/reference/cli/_build.yaml +++ /dev/null @@ -1,65 +0,0 @@ -data: -- bannerCommand: viash build - bannerDescription: Build an executable from the provided viash config file. - bannerUsage: viash build config.vsh.yaml -o output [-p docker] [-m] [-s] - name: build - opts: - - argName: arg - descr: Specifies which platform amongst those specified in the config to use. - If this is not provided, the first platform will be used. If no platforms are - defined in the config, the native platform will be used. In addition, the path - to a platform yaml file can also be specified. - hidden: false - name: platform - optType: opt - required: false - short: p - type: String - - default: config.vsh.yaml - descr: 'A viash config file (example: config.vsh.yaml). This argument can also - be a script with the config as a header.' - hidden: false - name: config - optType: trailArgs - required: true - type: String - - argName: arg - default: List() - descr: Modify a viash config at runtime using [dynamic config modding](/reference/config_mods/#). - hidden: false - name: config_mod - optType: opt - required: false - short: c - type: List[String] - - argName: arg - default: output/ - descr: 'Path to directory in which the executable and any resources is built to. - Default: "output/".' - hidden: false - name: output - optType: opt - required: true - short: o - type: String - - argName: arg - descr: Which [setup strategy](/guide/component/add-dependencies.html#step-3-rebuild-docker-image) - for creating the container to use [Docker Platform only]. - hidden: false - name: setup - optType: opt - required: false - short: s - type: String - - argName: arg - default: 'false' - descr: Whether or not to push the container to a Docker registry [Docker Platform - only]. - hidden: false - name: push - optType: opt - required: false - type: Boolean - subcommands: [] -title: Viash Build -usesSubcommands: false diff --git a/reference/cli/_config_inject.yaml b/reference/cli/_config_inject.yaml deleted file mode 100644 index 46eb9c9a..00000000 --- a/reference/cli/_config_inject.yaml +++ /dev/null @@ -1,37 +0,0 @@ -data: -- bannerCommand: viash config inject - bannerDescription: Inject a Viash header into the main script of a Viash component. - bannerUsage: viash config inject config.vsh.yaml - name: inject - opts: - - argName: arg - descr: Specifies which platform amongst those specified in the config to use. - If this is not provided, the first platform will be used. If no platforms are - defined in the config, the native platform will be used. In addition, the path - to a platform yaml file can also be specified. - hidden: false - name: platform - optType: opt - required: false - short: p - type: String - - default: config.vsh.yaml - descr: 'A viash config file (example: config.vsh.yaml). This argument can also - be a script with the config as a header.' - hidden: false - name: config - optType: trailArgs - required: true - type: String - - argName: arg - default: List() - descr: Modify a viash config at runtime using [dynamic config modding](/reference/config_mods/#). - hidden: false - name: config_mod - optType: opt - required: false - short: c - type: List[String] - subcommands: [] -title: Viash Config Inject -usesSubcommands: false diff --git a/reference/cli/_config_view.yaml b/reference/cli/_config_view.yaml deleted file mode 100644 index 6e7dd169..00000000 --- a/reference/cli/_config_view.yaml +++ /dev/null @@ -1,57 +0,0 @@ -data: -- bannerCommand: viash config view - bannerDescription: View the config file after parsing. - bannerUsage: viash config view config.vsh.yaml - name: view - opts: - - argName: arg - descr: Specifies which platform amongst those specified in the config to use. - If this is not provided, the first platform will be used. If no platforms are - defined in the config, the native platform will be used. In addition, the path - to a platform yaml file can also be specified. - hidden: false - name: platform - optType: opt - required: false - short: p - type: String - - default: config.vsh.yaml - descr: 'A viash config file (example: config.vsh.yaml). This argument can also - be a script with the config as a header.' - hidden: false - name: config - optType: trailArgs - required: true - type: String - - argName: arg - default: List() - descr: Modify a viash config at runtime using [dynamic config modding](/reference/config_mods/#). - hidden: false - name: config_mod - optType: opt - required: false - short: c - type: List[String] - - argName: arg - choices: - - yaml - - json - default: yaml - descr: Which output format to use. - hidden: false - name: format - optType: choice - required: false - short: f - type: String - - argName: arg - default: 'false' - descr: Whether or not to postprocess each component's [argument groups](/reference/config/functionality/#argument_groups). - hidden: false - name: parse_argument_groups - optType: opt - required: false - type: Boolean - subcommands: [] -title: Viash Config View -usesSubcommands: false diff --git a/reference/cli/_ns_build.yaml b/reference/cli/_ns_build.yaml deleted file mode 100644 index dca37153..00000000 --- a/reference/cli/_ns_build.yaml +++ /dev/null @@ -1,108 +0,0 @@ -data: -- bannerCommand: viash ns build - bannerDescription: Build a namespace from many viash config files. - bannerUsage: viash ns build [-n nmspc] [-s src] [-t target] [-p docker] [--setup] - [--push] [--parallel] [--flatten] - name: build - opts: - - argName: arg - descr: 'Filter which components get selected by component and [namespace](/guide/project/structure.html#grouping-components-in-namespaces) - name. Can be a regex. Example: "^mynamespace/component1$".' - hidden: false - name: query - optType: opt - required: false - short: q - type: String - - argName: arg - descr: 'Filter which namespaces get selected by [namespace](/guide/project/structure.html#grouping-components-in-namespaces) - name. Can be a regex. Example: "^mynamespace$".' - hidden: false - name: query_namespace - optType: opt - required: false - short: n - type: String - - argName: arg - descr: 'Filter which components get selected by component name. Can be a regex. - Example: "^component1".' - hidden: false - name: query_name - optType: opt - required: false - type: String - - argName: arg - descr: 'A source directory containing viash config files, possibly structured - in a hierarchical folder structure. Default: src/.' - hidden: false - name: src - optType: opt - required: false - short: s - type: String - - argName: arg - descr: Acts as a regular expression to filter the platform ids specified in the - found config files. If this is not provided, all platforms will be used. If - no platforms are defined in a config, the native platform will be used. In addition, - the path to a platform yaml file can also be specified. - hidden: false - name: platform - optType: opt - required: false - short: p - type: String - - argName: arg - default: 'false' - descr: Whether or not to run the process in parallel. - hidden: false - name: parallel - optType: opt - required: false - short: l - type: Boolean - - argName: arg - default: List() - descr: Modify a viash config at runtime using [dynamic config modding](/reference/config_mods/#). - hidden: false - name: config_mod - optType: opt - required: false - short: c - type: List[String] - - argName: arg - descr: 'A target directory to build the executables into. Default: target/.' - hidden: false - name: target - optType: opt - required: false - short: t - type: String - - argName: arg - descr: Which [setup strategy](/guide/component/add-dependencies.html#step-3-rebuild-docker-image) - for creating the container to use [Docker Platform only]. - hidden: false - name: setup - optType: opt - required: false - type: String - - argName: arg - default: 'false' - descr: Whether or not to push the container to a Docker registry [Docker Platform - only]. - hidden: false - name: push - optType: opt - required: false - type: Boolean - - argName: arg - default: 'false' - descr: Flatten the target builds, handy for building one platform to a bin directory. - hidden: false - name: flatten - optType: opt - required: false - short: f - type: Boolean - subcommands: [] -title: Viash Ns Build -usesSubcommands: false diff --git a/reference/cli/_ns_exec.yaml b/reference/cli/_ns_exec.yaml deleted file mode 100644 index bd896787..00000000 --- a/reference/cli/_ns_exec.yaml +++ /dev/null @@ -1,114 +0,0 @@ -data: -- bannerCommand: viash ns exec - bannerDescription: "Execute a command for all found Viash components.\nThe syntax\ - \ of this command is inspired by `find . -exec echo {} \\;`.\n\nThe following\ - \ fields are automatically replaced: \n\n * `{}` | `{path}`: path to the config\ - \ file\n * `{abs-path}`: absolute path to the config file\n * `{dir}`: path to\ - \ the parent directory of the config file\n * `{abs-dir}`: absolute path to the\ - \ directory of the config file\n * `{main-script}`: path to the main script (if\ - \ any)\n * `{abs-main-script}`: absolute path to the main script (if any)\n *\ - \ `{functionality-name}`: name of the component\n * `{namespace}`: namespace of\ - \ the component\n * `{platform}`: selected platform id (only when --apply_platform\ - \ is used)\n * `{output}`: path to the destination directory when building the\ - \ component\n * `{abs-output}`: absolute path to the destination directory when\ - \ building the component\n\nA command suffixed by `\\;` (or nothing) will execute\ - \ one command for each\nof the Viash components, whereas a command suffixed by\ - \ `+` will execute one\ncommand for all Viash components." - bannerUsage: 'viash ns exec ''echo {path} \\;'' - - viash ns exec ''chmod +x {main-script} +''' - name: exec - opts: - - argName: arg - descr: 'Filter which components get selected by component and [namespace](/guide/project/structure.html#grouping-components-in-namespaces) - name. Can be a regex. Example: "^mynamespace/component1$".' - hidden: false - name: query - optType: opt - required: false - short: q - type: String - - argName: arg - descr: 'Filter which namespaces get selected by [namespace](/guide/project/structure.html#grouping-components-in-namespaces) - name. Can be a regex. Example: "^mynamespace$".' - hidden: false - name: query_namespace - optType: opt - required: false - short: n - type: String - - argName: arg - descr: 'Filter which components get selected by component name. Can be a regex. - Example: "^component1".' - hidden: false - name: query_name - optType: opt - required: false - type: String - - argName: arg - descr: 'A source directory containing viash config files, possibly structured - in a hierarchical folder structure. Default: src/.' - hidden: false - name: src - optType: opt - required: false - short: s - type: String - - argName: arg - descr: Acts as a regular expression to filter the platform ids specified in the - found config files. If this is not provided, all platforms will be used. If - no platforms are defined in a config, the native platform will be used. In addition, - the path to a platform yaml file can also be specified. - hidden: false - name: platform - optType: opt - required: false - short: p - type: String - - argName: arg - default: 'false' - descr: Whether or not to run the process in parallel. - hidden: false - name: parallel - optType: opt - required: false - short: l - type: Boolean - - argName: arg - default: List() - descr: Modify a viash config at runtime using [dynamic config modding](/reference/config_mods/#). - hidden: false - name: config_mod - optType: opt - required: false - short: c - type: List[String] - - argName: arg - default: 'false' - descr: "Fills in the {platform} and {output} field by applying each platform to\ - \ the \nconfig separately. Note that this results in the provided command being\ - \ applied\nonce for every platform that matches the --platform regex." - hidden: false - name: apply_platform - optType: opt - required: false - short: a - type: Boolean - - argName: arg - default: 'false' - descr: Perform a dry run. - hidden: false - name: dry_run - optType: opt - required: false - short: d - type: Boolean - - descr: The command to execute for each viash config file in the namespace. - hidden: false - name: cmd - optType: trailArgs - required: true - type: String - subcommands: [] -title: Viash Ns Exec -usesSubcommands: false diff --git a/reference/cli/_ns_list.yaml b/reference/cli/_ns_list.yaml deleted file mode 100644 index c9545187..00000000 --- a/reference/cli/_ns_list.yaml +++ /dev/null @@ -1,93 +0,0 @@ -data: -- bannerCommand: viash ns list - bannerDescription: List a namespace containing many viash config files. - bannerUsage: viash ns list [-n nmspc] [-s src] [-p docker] - name: list - opts: - - argName: arg - descr: 'Filter which components get selected by component and [namespace](/guide/project/structure.html#grouping-components-in-namespaces) - name. Can be a regex. Example: "^mynamespace/component1$".' - hidden: false - name: query - optType: opt - required: false - short: q - type: String - - argName: arg - descr: 'Filter which namespaces get selected by [namespace](/guide/project/structure.html#grouping-components-in-namespaces) - name. Can be a regex. Example: "^mynamespace$".' - hidden: false - name: query_namespace - optType: opt - required: false - short: n - type: String - - argName: arg - descr: 'Filter which components get selected by component name. Can be a regex. - Example: "^component1".' - hidden: false - name: query_name - optType: opt - required: false - type: String - - argName: arg - descr: 'A source directory containing viash config files, possibly structured - in a hierarchical folder structure. Default: src/.' - hidden: false - name: src - optType: opt - required: false - short: s - type: String - - argName: arg - descr: Acts as a regular expression to filter the platform ids specified in the - found config files. If this is not provided, all platforms will be used. If - no platforms are defined in a config, the native platform will be used. In addition, - the path to a platform yaml file can also be specified. - hidden: false - name: platform - optType: opt - required: false - short: p - type: String - - argName: arg - default: 'false' - descr: Whether or not to run the process in parallel. - hidden: false - name: parallel - optType: opt - required: false - short: l - type: Boolean - - argName: arg - default: List() - descr: Modify a viash config at runtime using [dynamic config modding](/reference/config_mods/#). - hidden: false - name: config_mod - optType: opt - required: false - short: c - type: List[String] - - argName: arg - choices: - - yaml - - json - default: yaml - descr: Which output format to use. - hidden: false - name: format - optType: choice - required: false - short: f - type: String - - argName: arg - default: 'false' - descr: Whether or not to postprocess each component's [argument groups](/reference/config/functionality/#argument_groups). - hidden: false - name: parse_argument_groups - optType: opt - required: false - type: Boolean - subcommands: [] -title: Viash Ns List -usesSubcommands: false diff --git a/reference/cli/_ns_test.yaml b/reference/cli/_ns_test.yaml deleted file mode 100644 index ea209731..00000000 --- a/reference/cli/_ns_test.yaml +++ /dev/null @@ -1,118 +0,0 @@ -data: -- bannerCommand: viash ns test - bannerDescription: Test a namespace containing many viash config files. - bannerUsage: viash ns test [-n nmspc] [-s src] [-p docker] [--parallel] [--tsv file.tsv] - [--append] - name: test - opts: - - argName: arg - descr: 'Filter which components get selected by component and [namespace](/guide/project/structure.html#grouping-components-in-namespaces) - name. Can be a regex. Example: "^mynamespace/component1$".' - hidden: false - name: query - optType: opt - required: false - short: q - type: String - - argName: arg - descr: 'Filter which namespaces get selected by [namespace](/guide/project/structure.html#grouping-components-in-namespaces) - name. Can be a regex. Example: "^mynamespace$".' - hidden: false - name: query_namespace - optType: opt - required: false - short: n - type: String - - argName: arg - descr: 'Filter which components get selected by component name. Can be a regex. - Example: "^component1".' - hidden: false - name: query_name - optType: opt - required: false - type: String - - argName: arg - descr: 'A source directory containing viash config files, possibly structured - in a hierarchical folder structure. Default: src/.' - hidden: false - name: src - optType: opt - required: false - short: s - type: String - - argName: arg - descr: Acts as a regular expression to filter the platform ids specified in the - found config files. If this is not provided, all platforms will be used. If - no platforms are defined in a config, the native platform will be used. In addition, - the path to a platform yaml file can also be specified. - hidden: false - name: platform - optType: opt - required: false - short: p - type: String - - argName: arg - default: 'false' - descr: Whether or not to run the process in parallel. - hidden: false - name: parallel - optType: opt - required: false - short: l - type: Boolean - - argName: arg - default: List() - descr: Modify a viash config at runtime using [dynamic config modding](/reference/config_mods/#). - hidden: false - name: config_mod - optType: opt - required: false - short: c - type: List[String] - - argName: arg - descr: Whether or not to keep temporary files. By default, files will be deleted - if all goes well but remain when an error occurs. By specifying --keep true, - the temporary files will always be retained, whereas --keep false will always - delete them. The temporary directory can be overwritten by setting defining - a VIASH_TEMP directory. - hidden: false - name: keep - optType: opt - required: false - short: k - type: String - - argName: arg - descr: The maximum number of (logical) cpus a component is allowed to use. - hidden: false - name: cpus - optType: opt - required: false - type: Int - - argName: arg - descr: The maximum amount of memory a component is allowed to allocate. Unit must - be one of B, KB, MB, GB, TB or PB. - hidden: false - name: memory - optType: opt - required: false - type: String - - argName: arg - descr: Path to write a summary of the test results to. - hidden: false - name: tsv - optType: opt - required: false - short: t - type: String - - argName: arg - default: 'false' - descr: Append to tsv instead of overwrite - hidden: false - name: append - optType: opt - required: false - short: a - type: Boolean - subcommands: [] -title: Viash Ns Test -usesSubcommands: false diff --git a/reference/cli/_run.yaml b/reference/cli/_run.yaml deleted file mode 100644 index b4c8ba85..00000000 --- a/reference/cli/_run.yaml +++ /dev/null @@ -1,72 +0,0 @@ -data: -- bannerCommand: viash run - bannerDescription: Executes a viash component from the provided viash config file. - viash generates a temporary executable and immediately executes it with the given - parameters. - bannerUsage: viash run config.vsh.yaml [-p docker] [-k true/false] -- [arguments - for script] - footer: " -- param1 param2 ... Extra parameters to be passed to the component\ - \ itself.\n -- is used to separate viash arguments from\ - \ the arguments\n of the component.\n\nThe temporary\ - \ directory can be altered by setting the VIASH_TEMP directory. Example:\n export\ - \ VIASH_TEMP=/home/myuser/.viash_temp\n viash run config.vsh.yaml" - name: run - opts: - - argName: arg - descr: Specifies which platform amongst those specified in the config to use. - If this is not provided, the first platform will be used. If no platforms are - defined in the config, the native platform will be used. In addition, the path - to a platform yaml file can also be specified. - hidden: false - name: platform - optType: opt - required: false - short: p - type: String - - default: config.vsh.yaml - descr: 'A viash config file (example: config.vsh.yaml). This argument can also - be a script with the config as a header.' - hidden: false - name: config - optType: trailArgs - required: true - type: String - - argName: arg - default: List() - descr: Modify a viash config at runtime using [dynamic config modding](/reference/config_mods/#). - hidden: false - name: config_mod - optType: opt - required: false - short: c - type: List[String] - - argName: arg - descr: Whether or not to keep temporary files. By default, files will be deleted - if all goes well but remain when an error occurs. By specifying --keep true, - the temporary files will always be retained, whereas --keep false will always - delete them. The temporary directory can be overwritten by setting defining - a VIASH_TEMP directory. - hidden: false - name: keep - optType: opt - required: false - short: k - type: String - - argName: arg - descr: The maximum number of (logical) cpus a component is allowed to use. - hidden: false - name: cpus - optType: opt - required: false - type: Int - - argName: arg - descr: The maximum amount of memory a component is allowed to allocate. Unit must - be one of B, KB, MB, GB, TB or PB. - hidden: false - name: memory - optType: opt - required: false - type: String - subcommands: [] -title: Viash Run -usesSubcommands: false diff --git a/reference/cli/_test.yaml b/reference/cli/_test.yaml deleted file mode 100644 index 7f14ca3b..00000000 --- a/reference/cli/_test.yaml +++ /dev/null @@ -1,67 +0,0 @@ -data: -- bannerCommand: viash test - bannerDescription: Test the component using the tests defined in the viash config - file. - bannerUsage: viash test config.vsh.yaml [-p docker] [-k true/false] - footer: "\nThe temporary directory can be altered by setting the VIASH_TEMP directory.\ - \ Example:\n export VIASH_TEMP=/home/myuser/.viash_temp\n viash run meta.vsh.yaml" - name: test - opts: - - argName: arg - descr: Specifies which platform amongst those specified in the config to use. - If this is not provided, the first platform will be used. If no platforms are - defined in the config, the native platform will be used. In addition, the path - to a platform yaml file can also be specified. - hidden: false - name: platform - optType: opt - required: false - short: p - type: String - - default: config.vsh.yaml - descr: 'A viash config file (example: config.vsh.yaml). This argument can also - be a script with the config as a header.' - hidden: false - name: config - optType: trailArgs - required: true - type: String - - argName: arg - default: List() - descr: Modify a viash config at runtime using [dynamic config modding](/reference/config_mods/#). - hidden: false - name: config_mod - optType: opt - required: false - short: c - type: List[String] - - argName: arg - descr: Whether or not to keep temporary files. By default, files will be deleted - if all goes well but remain when an error occurs. By specifying --keep true, - the temporary files will always be retained, whereas --keep false will always - delete them. The temporary directory can be overwritten by setting defining - a VIASH_TEMP directory. - hidden: false - name: keep - optType: opt - required: false - short: k - type: String - - argName: arg - descr: The maximum number of (logical) cpus a component is allowed to use. - hidden: false - name: cpus - optType: opt - required: false - type: Int - - argName: arg - descr: The maximum amount of memory a component is allowed to allocate. Unit must - be one of B, KB, MB, GB, TB or PB. - hidden: false - name: memory - optType: opt - required: false - type: String - subcommands: [] -title: Viash Test -usesSubcommands: false diff --git a/reference/cli/build.qmd b/reference/cli/build.qmd index 40dbf631..9e385bc5 100644 --- a/reference/cli/build.qmd +++ b/reference/cli/build.qmd @@ -19,4 +19,3 @@ Build an executable from the provided viash config file. | `--setup`, `-s` | Which [setup strategy](/guide/component/add-dependencies.html#step-3-rebuild-docker-image) for creating the container to use [Docker Platform only]. | `String` | | `--help`, `-h` | Show help message | | - diff --git a/reference/cli/config_inject.qmd b/reference/cli/config_inject.qmd index 56f30738..193b9c69 100644 --- a/reference/cli/config_inject.qmd +++ b/reference/cli/config_inject.qmd @@ -16,4 +16,3 @@ Inject a Viash header into the main script of a Viash component. | `--platform`, `-p` | Specifies which platform amongst those specified in the config to use. If this is not provided, the first platform will be used. If no platforms are defined in the config, the native platform will be used. In addition, the path to a platform yaml file can also be specified. | `String` | | `--help`, `-h` | Show help message | | - diff --git a/reference/cli/config_view.qmd b/reference/cli/config_view.qmd index 69d9db1d..6f14d5d5 100644 --- a/reference/cli/config_view.qmd +++ b/reference/cli/config_view.qmd @@ -18,4 +18,3 @@ View the config file after parsing. | `--platform`, `-p` | Specifies which platform amongst those specified in the config to use. If this is not provided, the first platform will be used. If no platforms are defined in the config, the native platform will be used. In addition, the path to a platform yaml file can also be specified. | `String` | | `--help`, `-h` | Show help message | | - diff --git a/reference/cli/ns_build.qmd b/reference/cli/ns_build.qmd index 22b408a0..93a3e1c7 100644 --- a/reference/cli/ns_build.qmd +++ b/reference/cli/ns_build.qmd @@ -24,4 +24,3 @@ Build a namespace from many viash config files. | `--target`, `-t` | A target directory to build the executables into. Default: target/. | `String` | | `--help`, `-h` | Show help message | | - diff --git a/reference/cli/ns_exec.qmd b/reference/cli/ns_exec.qmd index f4f9236d..413f9623 100644 --- a/reference/cli/ns_exec.qmd +++ b/reference/cli/ns_exec.qmd @@ -44,4 +44,3 @@ command for all Viash components. | `--src`, `-s` | A source directory containing viash config files, possibly structured in a hierarchical folder structure. Default: src/. | `String` | | `--help`, `-h` | Show help message | | - diff --git a/reference/cli/ns_list.qmd b/reference/cli/ns_list.qmd index f00268cc..c82a507c 100644 --- a/reference/cli/ns_list.qmd +++ b/reference/cli/ns_list.qmd @@ -22,4 +22,3 @@ List a namespace containing many viash config files. | `--src`, `-s` | A source directory containing viash config files, possibly structured in a hierarchical folder structure. Default: src/. | `String` | | `--help`, `-h` | Show help message | | - diff --git a/reference/cli/ns_test.qmd b/reference/cli/ns_test.qmd index 99cca38b..3f320d62 100644 --- a/reference/cli/ns_test.qmd +++ b/reference/cli/ns_test.qmd @@ -7,7 +7,7 @@ Test a namespace containing many viash config files. **Usage:** -`viash ns test [-n nmspc] [-s src] [-p docker] [--parallel] [--tsv file.tsv] [--append]` +`viash ns test [-n nmspc] [-s src] [-p docker] [--parallel] [--tsv file.tsv] [--setup cachedbuild] [--append]` | Argument | Description | Type | |-|:----|-: @@ -21,8 +21,8 @@ Test a namespace containing many viash config files. | `--query`, `-q` | Filter which components get selected by component and [namespace](/guide/project/structure.html#grouping-components-in-namespaces) name. Can be a regex. Example: "^mynamespace/component1\$". | `String` | | `--query_name` | Filter which components get selected by component name. Can be a regex. Example: "^component1". | `String` | | `--query_namespace`, `-n` | Filter which namespaces get selected by [namespace](/guide/project/structure.html#grouping-components-in-namespaces) name. Can be a regex. Example: "^mynamespace\$". | `String` | +| `--setup` | Which [setup strategy](/guide/component/add-dependencies.html#step-3-rebuild-docker-image) for creating the container to use [Docker Platform only]. | `String` | | `--src`, `-s` | A source directory containing viash config files, possibly structured in a hierarchical folder structure. Default: src/. | `String` | | `--tsv`, `-t` | Path to write a summary of the test results to. | `String` | | `--help`, `-h` | Show help message | | - diff --git a/reference/cli/run.qmd b/reference/cli/run.qmd index c465c918..3170a780 100644 --- a/reference/cli/run.qmd +++ b/reference/cli/run.qmd @@ -19,4 +19,3 @@ Executes a viash component from the provided viash config file. viash generates | `--platform`, `-p` | Specifies which platform amongst those specified in the config to use. If this is not provided, the first platform will be used. If no platforms are defined in the config, the native platform will be used. In addition, the path to a platform yaml file can also be specified. | `String` | | `--help`, `-h` | Show help message | | - diff --git a/reference/cli/test.qmd b/reference/cli/test.qmd index 3d0e3147..c7dd98d6 100644 --- a/reference/cli/test.qmd +++ b/reference/cli/test.qmd @@ -7,7 +7,7 @@ Test the component using the tests defined in the viash config file. **Usage:** -`viash test config.vsh.yaml [-p docker] [-k true/false]` +`viash test config.vsh.yaml [-p docker] [-k true/false] [--setup cachedbuild]` | Argument | Description | Type | |-|:----|-: @@ -17,6 +17,6 @@ Test the component using the tests defined in the viash config file. | `--keep`, `-k` | Whether or not to keep temporary files. By default, files will be deleted if all goes well but remain when an error occurs. By specifying --keep true, the temporary files will always be retained, whereas --keep false will always delete them. The temporary directory can be overwritten by setting defining a VIASH_TEMP directory. | `String` | | `--memory` | The maximum amount of memory a component is allowed to allocate. Unit must be one of B, KB, MB, GB, TB or PB. | `String` | | `--platform`, `-p` | Specifies which platform amongst those specified in the config to use. If this is not provided, the first platform will be used. If no platforms are defined in the config, the native platform will be used. In addition, the path to a platform yaml file can also be specified. | `String` | +| `--setup`, `-s` | Which [setup strategy](/guide/component/add-dependencies.html#step-3-rebuild-docker-image) for creating the container to use [Docker Platform only]. | `String` | | `--help`, `-h` | Show help message | | - diff --git a/reference/cli_schema_export.json b/reference/cli_schema_export.json index c012de40..20a4f865 100644 --- a/reference/cli_schema_export.json +++ b/reference/cli_schema_export.json @@ -65,6 +65,36 @@ "hidden" : false, "type" : "String", "optType" : "opt" + }, + { + "name" : "colorize", + "descr" : "Specify whether the console output should be colorized. If not specified, we attempt to detect this automatically.", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "true", + "false", + "auto" + ], + "type" : "String", + "optType" : "choice" + }, + { + "name" : "loglevel", + "descr" : "Specify the log level in use", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "error", + "warn", + "info", + "debug", + "trace" + ], + "type" : "String", + "optType" : "choice" } ] }, @@ -106,6 +136,36 @@ "type" : "List[String]", "optType" : "opt" }, + { + "name" : "colorize", + "descr" : "Specify whether the console output should be colorized. If not specified, we attempt to detect this automatically.", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "true", + "false", + "auto" + ], + "type" : "String", + "optType" : "choice" + }, + { + "name" : "loglevel", + "descr" : "Specify the log level in use", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "error", + "warn", + "info", + "debug", + "trace" + ], + "type" : "String", + "optType" : "choice" + }, { "name" : "output", "short" : "o", @@ -143,7 +203,7 @@ "name" : "test", "bannerCommand" : "viash test", "bannerDescription" : "Test the component using the tests defined in the viash config file.", - "bannerUsage" : "viash test config.vsh.yaml [-p docker] [-k true/false]", + "bannerUsage" : "viash test config.vsh.yaml [-p docker] [-k true/false] [--setup cachedbuild]", "footer" : "\nThe temporary directory can be altered by setting the VIASH_TEMP directory. Example:\n export VIASH_TEMP=/home/myuser/.viash_temp\n viash run meta.vsh.yaml", "subcommands" : [ ], @@ -205,6 +265,46 @@ "hidden" : false, "type" : "String", "optType" : "opt" + }, + { + "name" : "colorize", + "descr" : "Specify whether the console output should be colorized. If not specified, we attempt to detect this automatically.", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "true", + "false", + "auto" + ], + "type" : "String", + "optType" : "choice" + }, + { + "name" : "loglevel", + "descr" : "Specify the log level in use", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "error", + "warn", + "info", + "debug", + "trace" + ], + "type" : "String", + "optType" : "choice" + }, + { + "name" : "setup", + "short" : "s", + "descr" : "Which @[setup strategy](docker_setup_strategy) for creating the container to use [Docker Platform only].", + "required" : false, + "argName" : "arg", + "hidden" : false, + "type" : "String", + "optType" : "opt" } ] }, @@ -300,6 +400,36 @@ "type" : "String", "optType" : "opt" }, + { + "name" : "colorize", + "descr" : "Specify whether the console output should be colorized. If not specified, we attempt to detect this automatically.", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "true", + "false", + "auto" + ], + "type" : "String", + "optType" : "choice" + }, + { + "name" : "loglevel", + "descr" : "Specify the log level in use", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "error", + "warn", + "info", + "debug", + "trace" + ], + "type" : "String", + "optType" : "choice" + }, { "name" : "setup", "descr" : "Which @[setup strategy](docker_setup_strategy) for creating the container to use [Docker Platform only].", @@ -336,7 +466,7 @@ "name" : "test", "bannerCommand" : "viash ns test", "bannerDescription" : "Test a namespace containing many viash config files.", - "bannerUsage" : "viash ns test [-n nmspc] [-s src] [-p docker] [--parallel] [--tsv file.tsv] [--append]", + "bannerUsage" : "viash ns test [-n nmspc] [-s src] [-p docker] [--parallel] [--tsv file.tsv] [--setup cachedbuild] [--append]", "subcommands" : [ ], "opts" : [ @@ -439,6 +569,45 @@ "type" : "String", "optType" : "opt" }, + { + "name" : "colorize", + "descr" : "Specify whether the console output should be colorized. If not specified, we attempt to detect this automatically.", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "true", + "false", + "auto" + ], + "type" : "String", + "optType" : "choice" + }, + { + "name" : "loglevel", + "descr" : "Specify the log level in use", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "error", + "warn", + "info", + "debug", + "trace" + ], + "type" : "String", + "optType" : "choice" + }, + { + "name" : "setup", + "descr" : "Which @[setup strategy](docker_setup_strategy) for creating the container to use [Docker Platform only].", + "required" : false, + "argName" : "arg", + "hidden" : false, + "type" : "String", + "optType" : "opt" + }, { "name" : "tsv", "short" : "t", @@ -541,6 +710,36 @@ "type" : "List[String]", "optType" : "opt" }, + { + "name" : "colorize", + "descr" : "Specify whether the console output should be colorized. If not specified, we attempt to detect this automatically.", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "true", + "false", + "auto" + ], + "type" : "String", + "optType" : "choice" + }, + { + "name" : "loglevel", + "descr" : "Specify the log level in use", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "error", + "warn", + "info", + "debug", + "trace" + ], + "type" : "String", + "optType" : "choice" + }, { "name" : "format", "short" : "f", @@ -647,6 +846,36 @@ "type" : "List[String]", "optType" : "opt" }, + { + "name" : "colorize", + "descr" : "Specify whether the console output should be colorized. If not specified, we attempt to detect this automatically.", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "true", + "false", + "auto" + ], + "type" : "String", + "optType" : "choice" + }, + { + "name" : "loglevel", + "descr" : "Specify the log level in use", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "error", + "warn", + "info", + "debug", + "trace" + ], + "type" : "String", + "optType" : "choice" + }, { "name" : "apply_platform", "short" : "a", @@ -724,6 +953,36 @@ "type" : "List[String]", "optType" : "opt" }, + { + "name" : "colorize", + "descr" : "Specify whether the console output should be colorized. If not specified, we attempt to detect this automatically.", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "true", + "false", + "auto" + ], + "type" : "String", + "optType" : "choice" + }, + { + "name" : "loglevel", + "descr" : "Specify the log level in use", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "error", + "warn", + "info", + "debug", + "trace" + ], + "type" : "String", + "optType" : "choice" + }, { "name" : "format", "short" : "f", @@ -788,6 +1047,36 @@ "hidden" : false, "type" : "List[String]", "optType" : "opt" + }, + { + "name" : "colorize", + "descr" : "Specify whether the console output should be colorized. If not specified, we attempt to detect this automatically.", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "true", + "false", + "auto" + ], + "type" : "String", + "optType" : "choice" + }, + { + "name" : "loglevel", + "descr" : "Specify the log level in use", + "required" : false, + "argName" : "arg", + "hidden" : true, + "choices" : [ + "error", + "warn", + "info", + "debug", + "trace" + ], + "type" : "String", + "optType" : "choice" } ] } diff --git a/reference/config/_index.yaml b/reference/config/_index.yaml deleted file mode 100644 index bb5e1ceb..00000000 --- a/reference/config/_index.yaml +++ /dev/null @@ -1,45 +0,0 @@ -data: -- description: "A Viash configuration is a YAML file which contains metadata to describe\ - \ the behaviour and build target(s) of a component. \nWe commonly name this file\ - \ `config.vsh.yaml` in our examples, but you can name it however you choose. \ - \ \n" - example: - - example: "functionality:\n name: hello_world\n arguments:\n - type: string\n\ - \ name: --input\n default: \"world\"\n resources:\n - type: bash_script\n\ - \ path: script.sh\n text: echo Hello $par_input\nplatforms:\n - type:\ - \ docker\n image: \"bash:4.0\"\n" - format: yaml - hierarchy: - - io.viash.config.Config - name: __this__ - type: Config -- description: 'The [functionality](/reference/config/functionality/#) describes the - behaviour of the script in terms of arguments and resources. - - By specifying a few restrictions (e.g. mandatory arguments) and adding some descriptions, - Viash will automatically generate a stylish command-line interface for you. - - ' - name: functionality - type: Functionality -- description: "A list of platforms to generate target artifacts for.\n\n - [Native](/reference/config/platforms/native/#)\n\ - \ - [Docker](/reference/config/platforms/docker/#)\n - [Nextflow VDSL3](/reference/config/platforms/nextflow/#)\n" - name: platforms - type: List of Platform -- description: 'Config inheritance by including YAML partials. This is useful for - defining common APIs in - - separate files. `__merge__` can be used in any level of the YAML. For example, - - not just in the config but also in the functionality or any of the platforms. - - ' - example: - - example: '__merge__: ../api/common_interface.yaml' - format: yaml - name: '`__merge__' - since: Viash 0.6.3 - type: Option of File -order: 20 -title: Config -topic: config diff --git a/reference/config/environmentVariables.qmd b/reference/config/environmentVariables.qmd new file mode 100644 index 00000000..62d64773 --- /dev/null +++ b/reference/config/environmentVariables.qmd @@ -0,0 +1,31 @@ +--- +title: "Environment Variables" +search: true +--- + +Viash checks several environment variables during operation. + +## VIASH_HOME + +**Type**: `String` + +If `VIASH_HOME` is not defined, the fallback `HOME`/.viash is used. + +Location where specific downloaded versions of Viash will be cached and run from. + + +## VIASH_VERSION + +**Type**: `String` + +**Default**: `Empty` + +A specific Viash version can be set to run the commands with. If so required, the specific Viash version will be downloaded. +This is useful when replicating older results or building Viash components that use outdated code. + + +**Example:** + +```sh +VIASH_VERSION=0.7.0 viash ns build +``` diff --git a/reference/config/functionality/_author.yaml b/reference/config/functionality/_author.yaml deleted file mode 100644 index 2f16e4cd..00000000 --- a/reference/config/functionality/_author.yaml +++ /dev/null @@ -1,46 +0,0 @@ -data: -- description: Author metadata. - example: - - example: "name: Jane Doe\nrole: [author, maintainer]\nemail: jane@doe.com\ninfo:\n\ - \ github: janedoe\n twitter: janedoe\n orcid: XXAABBCCXX\n groups: [ one,\ - \ two, three ]\n" - format: yaml - hierarchy: - - io.viash.functionality.Author - name: __this__ - since: Viash 0.3.2 - type: Author -- description: Full name of the author, usually in the name of FirstName MiddleName - LastName. - name: name - type: String -- description: E-mail of the author. - name: email - type: Option of String -- description: 'Structured information. Can be any shape: a string, vector, map or - even nested map.' - name: info - since: Viash 0.7.4 - type: Json -- description: 'Role of the author. Suggested items: - - - * `"author"`: Authors who have made substantial contributions to the component. - - * `"maintainer"`: The maintainer of the component. - - * `"contributor"`: Authors who have made smaller contributions (such as code patches - etc.). - - ' - name: roles - type: OneOrMore of String -- deprecated: - deprecation: 0.7.4 - message: Use `info` instead. - removal: 0.8.0 - description: Author properties. Must be a map of strings. - name: props - type: Map of String,String -title: Author -topic: functionality diff --git a/reference/config/functionality/_computationalRequirements.yaml b/reference/config/functionality/_computationalRequirements.yaml deleted file mode 100644 index 944270fb..00000000 --- a/reference/config/functionality/_computationalRequirements.yaml +++ /dev/null @@ -1,36 +0,0 @@ -data: -- description: Computational requirements related to running the component. - hierarchy: - - io.viash.functionality.ComputationalRequirements - name: __this__ - since: Viash 0.6.0 - type: ComputationalRequirements -- name: n_proc - removed: - deprecation: 0.6.1 - message: Use `cpus` instead. - removal: 0.7.0 - type: Option of Int -- description: The maximum number of (logical) cpus a component is allowed to use. - example: - - example: 'cpus: 10' - format: yaml - name: cpus - type: Option of Int -- description: A list of commands which should be present on the system for the script - to function. - example: - - example: 'commands: [ which, bash, awk, date, grep, egrep, ps, sed, tail, tee - ]' - format: yaml - name: commands - type: List of String -- description: The maximum amount of memory a component is allowed to allocate. Unit - must be one of B, KB, MB, GB, TB or PB. - example: - - example: 'memory: 10GB' - format: yaml - name: memory - type: Option of String -title: Computational Requirements -topic: functionality diff --git a/reference/config/functionality/_index.yaml b/reference/config/functionality/_index.yaml deleted file mode 100644 index d1494fb0..00000000 --- a/reference/config/functionality/_index.yaml +++ /dev/null @@ -1,232 +0,0 @@ -data: -- description: 'The functionality-part of the config file describes the behaviour - of the script in terms of arguments and resources. - - By specifying a few restrictions (e.g. mandatory arguments) and adding some descriptions, - Viash will automatically generate a stylish command-line interface for you. - - ' - hierarchy: - - io.viash.functionality.Functionality - name: __this__ - type: Functionality -- description: Name of the component and the filename of the executable when built - with `viash build`. - example: - - example: 'name: this_is_my_component' - format: yaml - name: name - type: String -- description: Setting this to false with disable this component when using namespaces. - name: enabled - removed: - deprecation: 0.6.0 - message: Use `status` instead. - removal: 0.7.0 - since: Viash 0.5.13 - type: Boolean -- description: One or more Bash/R/Python scripts to be used to test the component - behaviour when `viash test` is invoked. Additional files of type `file` will be - made available only during testing. Each test script should expect no command-line - inputs, be platform-independent, and return an exit code >0 when unexpected behaviour - occurs during testing. - name: tests - removed: - deprecation: 0.5.13 - message: Use `test_resources` instead. No functional difference. - removal: 0.7.0 - type: List of Resource -- description: 'Structured information. Can be any shape: a string, vector, map or - even nested map.' - example: - - example: "info:\n twitter: wizzkid\n classes: [ one, two, three ]" - format: yaml - name: info - since: Viash 0.4.0 - type: Json -- description: Version of the component. This field will be used to version the executable - and the Docker container. - example: - - example: 'version: 0.8' - format: yaml - name: version - type: Option of String -- description: A list of input arguments in addition to the `arguments` list. Any - arguments specified here will have their `type` set to `file` and the `direction` - set to `input` by default. - example: - - example: "inputs:\n - name: input_file\n - name: another_input" - format: yaml - - description: 'This results in the following output when calling the component - with the `--help` argument:' - example: "component_with_inputs\n \n Inputs:\n input_file\n type:\ - \ file\n \n another_input\n type: file" - format: bash - name: inputs - removed: - deprecation: 0.6.0 - message: Use `arguments` instead. - removal: 0.7.0 - since: Viash 0.5.11 - type: List of Argument -- description: "A list of [authors](/reference/config/functionality/author.html).\ - \ An author must at least have a name, but can also have a list of roles, an e-mail\ - \ address, and a map of custom properties.\n\nSuggested values for roles are:\n\ - \ \n| Role | Abbrev. | Description |\n|------|---------|-------------|\n| maintainer\ - \ | mnt | for the maintainer of the code. Ideally, exactly one maintainer is specified.\ - \ |\n| author | aut | for persons who have made substantial contributions to the\ - \ software. |\n| contributor | ctb| for persons who have made smaller contributions\ - \ (such as code patches).\n| datacontributor | dtc | for persons or organisations\ - \ that contributed data sets for the software\n| copyrightholder | cph | for all\ - \ copyright holders. This is a legal concept so should use the legal name of an\ - \ institution or corporate body.\n| funder | fnd | for persons or organizations\ - \ that furnished financial support for the development of the software\n\nThe\ - \ [full list of roles](https://www.loc.gov/marc/relators/relaterm.html) is extremely\ - \ comprehensive.\n" - example: - - example: "authors:\n - name: Jane Doe\n role: [author, maintainer]\n email:\ - \ jane@doe.com\n info:\n github: janedoe\n twitter: janedoe\n \ - \ orcid: XXAABBCCXX\n groups: [ one, two, three ]\n - name: Tim Farbe\n\ - \ roles: [author]\n email: tim@far.be\n" - format: yaml - name: authors - since: Viash 0.3.1 - type: List of Author -- description: Allows setting a component to active, deprecated or disabled. - name: status - since: Viash 0.6.0 - type: Status -- description: "[Computational requirements](/reference/config/functionality/computationalRequirements.html)\ - \ related to running the component. \n`cpus` specifies the maximum number of (logical)\ - \ cpus a component is allowed to use., whereas\n`memory` specifies the maximum\ - \ amount of memory a component is allowed to allicate. Memory units must be\n\ - in B, KB, MB, GB, TB or PB." - example: - - example: "requirements:\n cpus: 5\n memory: 10GB\n" - format: yaml - name: requirements - since: Viash 0.6.0 - type: ComputationalRequirements -- description: "[Resources](/reference/config/functionality/resources/#) are files\ - \ that support the component. The first resource should be [a script](/guide/component/create-component.html#create-a-script)\ - \ that will be executed when the functionality is run. Additional resources will\ - \ be copied to the same directory.\n\nCommon properties:\n\n * type: `file` /\ - \ `r_script` / `python_script` / `bash_script` / `javascript_script` / `scala_script`\ - \ / `csharp_script`, specifies the type of the resource. The first resource cannot\ - \ be of type `file`. When the type is not specified, the default type is simply\ - \ `file`.\n * dest: filename, the resulting name of the resource. From within\ - \ a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`.\ - \ If unspecified, `dest` will be set to the basename of the `path` parameter.\n\ - \ * path: `path/to/file`, the path of the input file. Can be a relative or an\ - \ absolute path, or a URI. Mutually exclusive with `text`.\n * text: ...multiline\ - \ text..., the content of the resulting file specified as a string. Mutually exclusive\ - \ with `path`.\n * is_executable: `true` / `false`, whether the resulting resource\ - \ file should be made executable.\n" - example: - - example: "resources:\n - type: r_script\n path: script.R\n - type: file\n\ - \ path: resource1.txt\n" - format: yaml - name: resources - type: List of Resource -- description: One or more [scripts](/guide/component/create-component.html#create-a-script) - to be used to test the component behaviour when `viash test` is invoked. Additional - files of type `file` will be made available only during testing. Each test script - should expect no command-line inputs, be platform-independent, and return an exit - code >0 when unexpected behaviour occurs during testing. See [Unit Testing](/guide/component/unit-testing.html) - for more info. - example: - - example: "test_resources:\n - type: bash_script\n path: tests/test1.sh\n \ - \ - type: r_script\n path: tests/test2.R\n - path: resource1.txt\n" - format: yaml - name: test_resources - type: List of Resource -- description: "A grouping of the [arguments](/reference/config/functionality/arguments/#),\ - \ used to display the help message.\n\n - `name: foo`, the name of the argument\ - \ group. \n - `description: Description of foo`, a description of the argument\ - \ group. Multiline descriptions are supported.\n - `arguments: [arg1, arg2, ...]`,\ - \ list of the arguments names.\n\n" - example: - - example: "argument_groups:\n - name: \"Input\"\n arguments:\n - name:\ - \ \"--id\"\n type: string\n required: true\n - name: \"--input\"\ - \n type: file\n required: true\n - name: \"Output\"\n arguments:\n\ - \ - name: \"--output\"\n type: file\n direction: output\n\ - \ required: true\n - name: \"--output_optional\"\n type:\ - \ file\n direction: output\n" - format: yaml - - description: 'This results in the following output when calling the component - with the `--help` argument:' - example: "component_name\n\n Input:\n --id\n type: string\n\n \ - \ --input\n type: file\n\n Output:\n --output\n \ - \ type: file\n\n --optional_output\n type: file\n" - format: bash - name: argument_groups - since: Viash 0.5.14 - type: List of ArgumentGroup -- description: A description of the component. This will be displayed with `--help`. - example: - - example: "description: |\n This component performs function Y and Z.\n It is\ - \ possible to make this a multiline string.\n" - format: yaml - name: description - type: Option of String -- description: A description on how to use the component. This will be displayed with - `--help` under the 'Usage:' section. - example: - - example: 'usage: Place the executable in a directory containing TSV files and - run it' - format: yaml - name: usage - type: Option of String -- description: Adds the resources directory to the PATH variable when set to true. - This is set to false by default. - name: add_resources_to_path - removed: - deprecation: '' - message: Extending the PATH turned out to be not desirable. - removal: 0.5.11 - since: Viash 0.5.5 - type: Boolean -- description: A list of output arguments in addition to the `arguments` list. Any - arguments specified here will have their `type` set to `file` and thr `direction` - set to `output` by default. - example: - - example: "outputs:\n - name: output_file\n - name: another_output" - format: yaml - - description: 'This results in the following output when calling the component - with the `--help` argument:' - example: "component_with_outputs\n \n Outputs:\n output_file\n \ - \ type: file, output\n \n another_output\n type: file, output" - format: bash - name: outputs - removed: - deprecation: 0.6.0 - message: Use `arguments` instead. - removal: 0.7.0 - since: Viash 0.5.11 - type: List of Argument -- description: Namespace this component is a part of. See the [Namespaces guide](/guide/project/structure.html#grouping-components-in-namespaces) - for more information on namespaces. - example: - - example: 'namespace: fancy_components' - format: yaml - name: namespace - type: Option of String -- description: "A list of [arguments](/reference/config/functionality/arguments/#)\ - \ for this component. For each argument, a type and a name must be specified.\ - \ Depending on the type of argument, different properties can be set. See these\ - \ reference pages per type for more information: \n\n - [string](/reference/config/functionality/arguments/string.html)\n\ - \ - [file](/reference/config/functionality/arguments/file.html)\n - [integer](/reference/config/functionality/arguments/integer.html)\n\ - \ - [double](/reference/config/functionality/arguments/double.html)\n - [boolean](/reference/config/functionality/arguments/boolean.html)\n\ - \ - [boolean_true](/reference/config/functionality/arguments/boolean_true.html)\n\ - \ - [boolean_false](/reference/config/functionality/arguments/boolean_false.html)\n" - example: - - example: "arguments:\n - name: --foo\n type: file\n alternatives: [-f]\n\ - \ description: Description of foo\n default: \"/foo/bar\"\n must_exist:\ - \ true\n direction: output\n required: false\n multiple: true\n \ - \ multiple_sep: \",\"\n - name: --bar\n type: string\n" - format: yaml - name: arguments - type: List of Argument -order: 10 -title: Functionality -topic: functionality diff --git a/reference/config/functionality/argumentGroup.qmd b/reference/config/functionality/argumentGroup.qmd new file mode 100644 index 00000000..abe548ee --- /dev/null +++ b/reference/config/functionality/argumentGroup.qmd @@ -0,0 +1,26 @@ +--- +title: "Argument Group" +search: true +--- + +A grouping of the [arguments](/reference/config/functionality/arguments/#), used to display the help message. + +## arguments + +**Type**: `List of Argument` + +List of arguments. + +## description + +**Type**: `String` + +**Default**: `Empty` + +Description of foo`, a description of the argument group. Multiline descriptions are supported. + +## name + +**Type**: `String` + +The name of the argument group. diff --git a/reference/config/functionality/arguments/_boolean.yaml b/reference/config/functionality/arguments/_boolean.yaml deleted file mode 100644 index 8db78972..00000000 --- a/reference/config/functionality/arguments/_boolean.yaml +++ /dev/null @@ -1,83 +0,0 @@ -data: -- description: 'A `boolean` type argument has two possible values: `true` or `false`.' - example: - - example: "arguments:\n - name: --trim\n type: boolean\n default: true\n\ - \ description: Trim whitespace from the final output\n alternatives: [\"\ - -t\"]\n" - format: yaml - hierarchy: - - io.viash.functionality.arguments.BooleanArgument - - io.viash.functionality.arguments.BooleanArgumentBase - - io.viash.functionality.arguments.Argument - name: __this__ - type: BooleanArgument -- description: List of alternative format variations for this argument. - name: alternatives - type: OneOrMore of String -- description: "The name of the argument. Can be in the formats `--trim`, `-t` or\ - \ `trim`. The number of dashes determines how values can be passed: \n\n - `--trim`\ - \ is a long option, which can be passed with `executable_name --trim`\n - `-t`\ - \ is a short option, which can be passed with `executable_name -t`\n - `trim`\ - \ is an argument, which can be passed with `executable_name trim` \n" - name: name - type: String -- description: 'Structured information. Can be any shape: a string, vector, map or - even nested map.' - example: - - example: "info:\n category: cat1\n labels: [one, two, three]" - format: yaml - name: info - since: Viash 0.6.3 - type: Json -- description: The default value when no argument value is provided. This will not - work if the [`required`](#required) property is enabled. - example: - - example: "- name: --my_boolean\n type: boolean\n default: true\n" - format: yaml - name: default - type: OneOrMore of Boolean -- description: An example value for this argument. If no [`default`](#default) property - was specified, this will be used for that purpose. - example: - - example: "- name: --my_boolean\n type: boolean\n example: true\n" - format: yaml - name: example - type: OneOrMore of Boolean -- description: A description of the argument. This will be displayed with `--help`. - name: description - type: Option of String -- description: The delimiter character for providing [`multiple`](#multiple) values. - `:` by default. - example: - - example: "- name: --my_boolean\n type: boolean\n multiple: true\n multiple_sep:\ - \ \",\"\n" - format: yaml - - description: 'Here''s an example of how to use this:' - example: my_component --my_boolean=true,true,false - format: bash - name: multiple_sep - type: String -- description: Treat the argument value as an array. Arrays can be passed using the - delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo - 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) - property. `false` by default. - example: - - example: "- name: --my_boolean\n type: boolean\n multiple: true\n" - format: yaml - - description: 'Here''s an example of how to use this:' - example: my_component --my_boolean=true:true:false - format: bash - name: multiple - type: Boolean -- description: Specifies the type of the argument. - name: type - type: String -- description: Make the value for this argument required. If set to `true`, an error - will be produced if no value was provided. `false` by default. - example: - - example: "- name: --my_boolean\n type: boolean\n required: true\n" - format: yaml - name: required - type: Boolean -title: Boolean -topic: arguments diff --git a/reference/config/functionality/arguments/_boolean_false.yaml b/reference/config/functionality/arguments/_boolean_false.yaml deleted file mode 100644 index f44b5afe..00000000 --- a/reference/config/functionality/arguments/_boolean_false.yaml +++ /dev/null @@ -1,41 +0,0 @@ -data: -- description: An argument of the `boolean_false` type acts like an inverted `boolean` - flag with a default value of `true`. When called as an argument it sets the `boolean` - to `false`. - example: - - example: "arguments:\n - name: --no-log\n type: boolean_false\n description:\ - \ Disable logging\n alternatives: [\"-nl\"]\n" - format: yaml - hierarchy: - - io.viash.functionality.arguments.BooleanFalseArgument - - io.viash.functionality.arguments.BooleanArgumentBase - - io.viash.functionality.arguments.Argument - name: __this__ - type: BooleanFalseArgument -- description: List of alternative format variations for this argument. - name: alternatives - type: OneOrMore of String -- description: "The name of the argument. Can be in the formats `--no-log`, `-n` or\ - \ `no-log`. The number of dashes determines how values can be passed: \n\n -\ - \ `--no-log` is a long option, which can be passed with `executable_name --no-log`\n\ - \ - `-n` is a short option, which can be passed with `executable_name -n`\n \ - \ - `no-log` is an argument, which can be passed with `executable_name no-log`\ - \ \n" - name: name - type: String -- description: 'Structured information. Can be any shape: a string, vector, map or - even nested map.' - example: - - example: "info:\n category: cat1\n labels: [one, two, three]" - format: yaml - name: info - since: Viash 0.6.3 - type: Json -- description: A description of the argument. This will be displayed with `--help`. - name: description - type: Option of String -- description: Specifies the type of the argument. - name: type - type: String -title: Boolean_False -topic: arguments diff --git a/reference/config/functionality/arguments/_boolean_true.yaml b/reference/config/functionality/arguments/_boolean_true.yaml deleted file mode 100644 index 242086d6..00000000 --- a/reference/config/functionality/arguments/_boolean_true.yaml +++ /dev/null @@ -1,41 +0,0 @@ -data: -- description: An argument of the `boolean_true` type acts like a `boolean` flag with - a default value of `false`. When called as an argument it sets the `boolean` to - `true`. - example: - - example: "arguments:\n - name: --silent\n type: boolean_true\n description:\ - \ Ignore console output\n alternatives: [\"-s\"]\n" - format: yaml - hierarchy: - - io.viash.functionality.arguments.BooleanTrueArgument - - io.viash.functionality.arguments.BooleanArgumentBase - - io.viash.functionality.arguments.Argument - name: __this__ - type: BooleanTrueArgument -- description: List of alternative format variations for this argument. - name: alternatives - type: OneOrMore of String -- description: "The name of the argument. Can be in the formats `--silent`, `-s` or\ - \ `silent`. The number of dashes determines how values can be passed: \n\n -\ - \ `--silent` is a long option, which can be passed with `executable_name --silent`\n\ - \ - `-s` is a short option, which can be passed with `executable_name -s`\n \ - \ - `silent` is an argument, which can be passed with `executable_name silent`\ - \ \n" - name: name - type: String -- description: 'Structured information. Can be any shape: a string, vector, map or - even nested map.' - example: - - example: "info:\n category: cat1\n labels: [one, two, three]" - format: yaml - name: info - since: Viash 0.6.3 - type: Json -- description: A description of the argument. This will be displayed with `--help`. - name: description - type: Option of String -- description: Specifies the type of the argument. - name: type - type: String -title: Boolean_True -topic: arguments diff --git a/reference/config/functionality/arguments/_double.yaml b/reference/config/functionality/arguments/_double.yaml deleted file mode 100644 index 5662152f..00000000 --- a/reference/config/functionality/arguments/_double.yaml +++ /dev/null @@ -1,98 +0,0 @@ -data: -- description: A `double` type argument has a numeric value with decimal points - example: - - example: "arguments:\n - name: --litres\n type: double\n default: 1.5\n\ - \ description: Litres of fluid to process\n alternatives: [\"-l\"]\n" - format: yaml - hierarchy: - - io.viash.functionality.arguments.DoubleArgument - - io.viash.functionality.arguments.Argument - name: __this__ - type: DoubleArgument -- description: List of alternative format variations for this argument. - name: alternatives - type: OneOrMore of String -- description: "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`.\ - \ The number of dashes determines how values can be passed: \n\n - `--foo` is\ - \ a long option, which can be passed with `executable_name --foo=value` or `executable_name\ - \ --foo value`\n - `-f` is a short option, which can be passed with `executable_name\ - \ -f value`\n - `foo` is an argument, which can be passed with `executable_name\ - \ value` \n" - name: name - type: String -- description: 'Structured information. Can be any shape: a string, vector, map or - even nested map.' - example: - - example: "info:\n category: cat1\n labels: [one, two, three]" - format: yaml - name: info - since: Viash 0.6.3 - type: Json -- description: Maximum allowed value for this argument. If set and the provided value - is higher than the maximum, an error will be produced. Can be combined with [`min`](#min) - to clamp values. - example: - - example: "- name: --my_double\n type: double\n max: 80.4\n" - format: yaml - name: max - type: Option of Double -- description: The default value when no argument value is provided. This will not - work if the [`required`](#required) property is enabled. - example: - - example: "- name: --my_double\n type: double\n default: 5.8\n" - format: yaml - name: default - type: OneOrMore of Double -- description: An example value for this argument. If no [`default`](#default) property - was specified, this will be used for that purpose. - example: - - example: "- name: --my_double\n type: double\n example: 5.8\n" - format: yaml - name: example - type: OneOrMore of Double -- description: A description of the argument. This will be displayed with `--help`. - name: description - type: Option of String -- description: The delimiter character for providing [`multiple`](#multiple) values. - `:` by default. - example: - - example: "- name: --my_double\n type: double\n multiple: true\n multiple_sep:\ - \ \",\"\n" - format: yaml - - description: 'Here''s an example of how to use this:' - example: my_component --my_double=5.8,22.6,200.4 - format: bash - name: multiple_sep - type: String -- description: Minimum allowed value for this argument. If set and the provided value - is lower than the minimum, an error will be produced. Can be combined with [`max`](#max) - to clamp values. - example: - - example: "- name: --my_double\n type: double\n min: 25.5\n" - format: yaml - name: min - type: Option of Double -- description: Treat the argument value as an array. Arrays can be passed using the - delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo - 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) - property. `false` by default. - example: - - example: "- name: --my_double\n type: double\n multiple: true\n" - format: yaml - - description: 'Here''s an example of how to use this:' - example: my_component --my_double=5.8:22.6:200.4 - format: bash - name: multiple - type: Boolean -- description: Specifies the type of the argument. - name: type - type: String -- description: Make the value for this argument required. If set to `true`, an error - will be produced if no value was provided. `false` by default. - example: - - example: "- name: --my_double\n type: double\n required: true\n" - format: yaml - name: required - type: Boolean -title: Double -topic: arguments diff --git a/reference/config/functionality/arguments/_file.yaml b/reference/config/functionality/arguments/_file.yaml deleted file mode 100644 index f651590e..00000000 --- a/reference/config/functionality/arguments/_file.yaml +++ /dev/null @@ -1,107 +0,0 @@ -data: -- description: A `file` type argument has a string value that points to a file or - folder path. - example: - - example: "arguments:\n - name: --input_csv\n type: file\n must_exist: true\n\ - \ description: CSV file to read contents from\n alternatives: [\"-i\"\ - ]\n" - format: yaml - hierarchy: - - io.viash.functionality.arguments.FileArgument - - io.viash.functionality.arguments.Argument - name: __this__ - type: FileArgument -- description: List of alternative format variations for this argument. - name: alternatives - type: OneOrMore of String -- description: "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`.\ - \ The number of dashes determines how values can be passed: \n\n - `--foo` is\ - \ a long option, which can be passed with `executable_name --foo=value` or `executable_name\ - \ --foo value`\n - `-f` is a short option, which can be passed with `executable_name\ - \ -f value`\n - `foo` is an argument, which can be passed with `executable_name\ - \ value` \n" - name: name - type: String -- description: 'If the output filename is a path and it does not exist, create it - before executing the script (only for `direction: output`).' - example: - - example: "- name: --my_file\n type: file\n direction: output\n create_parent:\ - \ true\n" - format: yaml - name: create_parent - type: Boolean -- description: Makes this argument an `input` or an `output`, as in does the file/folder - needs to be read or written. `input` by default. - example: - - example: "- name: --my_output_file\n type: file\n direction: output\n" - format: yaml - name: direction - type: Direction -- description: 'Structured information. Can be any shape: a string, vector, map or - even nested map.' - example: - - example: "info:\n category: cat1\n labels: [one, two, three]" - format: yaml - name: info - since: Viash 0.6.3 - type: Json -- description: Checks whether the file or folder exists. For input files, this check - will happen before the execution of the script, while for output files the check - will happen afterwards. - example: - - example: "- name: --my_file\n type: file\n must_exist: true\n" - format: yaml - name: must_exist - type: Boolean -- description: The default value when no argument value is provided. This will not - work if the [`required`](#required) property is enabled. - example: - - example: "- name: --my_file\n type: file\n default: data.csv\n" - format: yaml - name: default - type: OneOrMore of Path -- description: An example value for this argument. If no [`default`](#default) property - was specified, this will be used for that purpose. - example: - - example: "- name: --my_file\n type: file\n example: data.csv\n" - format: yaml - name: example - type: OneOrMore of Path -- description: A description of the argument. This will be displayed with `--help`. - name: description - type: Option of String -- description: The delimiter character for providing [`multiple`](#multiple) values. - `:` by default. - example: - - example: "- name: --my_files\n type: file\n multiple: true\n multiple_sep:\ - \ \",\"\n" - format: yaml - - description: 'Here''s an example of how to use this:' - example: my_component --my_files=firstFile.csv,anotherFile.csv,yetAnother.csv - format: bash - name: multiple_sep - type: String -- description: Treat the argument value as an array. Arrays can be passed using the - delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo - 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) - property. `false` by default. - example: - - example: "- name: --my_files\n type: file\n multiple: true\n" - format: yaml - - description: 'Here''s an example of how to use this:' - example: my_component --my_files=firstFile.csv:anotherFile.csv:yetAnother.csv - format: bash - name: multiple - type: Boolean -- description: Specifies the type of the argument. - name: type - type: String -- description: Make the value for this argument required. If set to `true`, an error - will be produced if no value was provided. `false` by default. - example: - - example: "- name: --my_file\n type: file\n required: true\n" - format: yaml - name: required - type: Boolean -title: File -topic: arguments diff --git a/reference/config/functionality/arguments/_index.yaml b/reference/config/functionality/arguments/_index.yaml deleted file mode 100644 index 2c017f5d..00000000 --- a/reference/config/functionality/arguments/_index.yaml +++ /dev/null @@ -1,20 +0,0 @@ -data: -- description: "For each argument, a type and a name must be specified. Depending\ - \ on the type of argument, different properties can be set. See these reference\ - \ pages per type for more information: \n\n - [string](/reference/config/functionality/arguments/string.html)\n\ - \ - [file](/reference/config/functionality/arguments/file.html)\n - [integer](/reference/config/functionality/arguments/integer.html)\n\ - \ - [double](/reference/config/functionality/arguments/double.html)\n - [boolean](/reference/config/functionality/arguments/boolean.html)\n\ - \ - [boolean_true](/reference/config/functionality/arguments/boolean_true.html)\n\ - \ - [boolean_false](/reference/config/functionality/arguments/boolean_false.html)\n" - example: - - example: "arguments:\n - name: --foo\n type: file\n alternatives: [-f]\n\ - \ description: Description of foo\n default: \"/foo/bar\"\n must_exist:\ - \ true\n direction: output\n required: false\n multiple: true\n \ - \ multiple_sep: \",\"\n - name: --bar\n type: string\n" - format: yaml - hierarchy: - - io.viash.functionality.arguments.Argument - name: __this__ - type: Argument -title: Argument -topic: arguments diff --git a/reference/config/functionality/arguments/_integer.yaml b/reference/config/functionality/arguments/_integer.yaml deleted file mode 100644 index 72ab0748..00000000 --- a/reference/config/functionality/arguments/_integer.yaml +++ /dev/null @@ -1,107 +0,0 @@ -data: -- description: An `integer` type argument has a numeric value without decimal points. - example: - - example: "arguments:\n - name: --core_amount\n type: integer\n default:\ - \ 16\n description: Amount of CPU cores to use\n alternatives: [\"-c\"\ - ]\n" - format: yaml - hierarchy: - - io.viash.functionality.arguments.IntegerArgument - - io.viash.functionality.arguments.Argument - name: __this__ - type: IntegerArgument -- description: List of alternative format variations for this argument. - name: alternatives - type: OneOrMore of String -- description: "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`.\ - \ The number of dashes determines how values can be passed: \n\n - `--foo` is\ - \ a long option, which can be passed with `executable_name --foo=value` or `executable_name\ - \ --foo value`\n - `-f` is a short option, which can be passed with `executable_name\ - \ -f value`\n - `foo` is an argument, which can be passed with `executable_name\ - \ value` \n" - name: name - type: String -- description: Limit the amount of valid values for this argument to those set in - this list. When set and a value not present in the list is provided, an error - will be produced. - example: - - example: "- name: --values\n type: integer\n choices: [1024, 2048, 4096]\n" - format: yaml - name: choices - type: List of Int -- description: 'Structured information. Can be any shape: a string, vector, map or - even nested map.' - example: - - example: "info:\n category: cat1\n labels: [one, two, three]" - format: yaml - name: info - since: Viash 0.6.3 - type: Json -- description: Maximum allowed value for this argument. If set and the provided value - is higher than the maximum, an error will be produced. Can be combined with [`min`](#min) - to clamp values. - example: - - example: "- name: --my_integer\n type: integer\n max: 150\n" - format: yaml - name: max - type: Option of Int -- description: The default value when no argument value is provided. This will not - work if the [`required`](#required) property is enabled. - example: - - example: "- name: --my_integer\n type: integer\n default: 100\n" - format: yaml - name: default - type: OneOrMore of Int -- description: An example value for this argument. If no [`default`](#default) property - was specified, this will be used for that purpose. - example: - - example: "- name: --my_integer\n type: integer\n example: 100\n" - format: yaml - name: example - type: OneOrMore of Int -- description: A description of the argument. This will be displayed with `--help`. - name: description - type: Option of String -- description: The delimiter character for providing [`multiple`](#multiple) values. - `:` by default. - example: - - example: "- name: --my_integer\n type: integer\n multiple: true\n multiple_sep:\ - \ \",\"\n" - format: yaml - - description: 'Here''s an example of how to use this:' - example: my_component --my_integer=10:80:152 - format: bash - name: multiple_sep - type: String -- description: Minimum allowed value for this argument. If set and the provided value - is lower than the minimum, an error will be produced. Can be combined with [`max`](#max) - to clamp values. - example: - - example: "- name: --my_integer\n type: integer\n min: 50\n" - format: yaml - name: min - type: Option of Int -- description: Treat the argument value as an array. Arrays can be passed using the - delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo - 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) - property. `false` by default. - example: - - example: "- name: --my_integer\n type: integer\n multiple: true\n" - format: yaml - - description: 'Here''s an example of how to use this:' - example: my_component --my_integer=10:80:152 - format: bash - name: multiple - type: Boolean -- description: Specifies the type of the argument. - name: type - type: String -- description: Make the value for this argument required. If set to `true`, an error - will be produced if no value was provided. `false` by default. - example: - - example: "- name: --my_integer\n type: integer\n required: true\n" - format: yaml - name: required - type: Boolean -title: Integer -topic: arguments diff --git a/reference/config/functionality/arguments/_long.yaml b/reference/config/functionality/arguments/_long.yaml deleted file mode 100644 index 91c8c007..00000000 --- a/reference/config/functionality/arguments/_long.yaml +++ /dev/null @@ -1,107 +0,0 @@ -data: -- description: An `long` type argument has a numeric value without decimal points. - example: - - example: "arguments:\n - name: --core_amount\n type: long\n default: 16\n\ - \ description: Amount of CPU cores to use\n alternatives: [\"-c\"]\n" - format: yaml - hierarchy: - - io.viash.functionality.arguments.LongArgument - - io.viash.functionality.arguments.Argument - name: __this__ - since: Viash 0.6.1 - type: LongArgument -- description: List of alternative format variations for this argument. - name: alternatives - type: OneOrMore of String -- description: "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`.\ - \ The number of dashes determines how values can be passed: \n\n - `--foo` is\ - \ a long option, which can be passed with `executable_name --foo=value` or `executable_name\ - \ --foo value`\n - `-f` is a short option, which can be passed with `executable_name\ - \ -f value`\n - `foo` is an argument, which can be passed with `executable_name\ - \ value` \n" - name: name - type: String -- description: Limit the amount of valid values for this argument to those set in - this list. When set and a value not present in the list is provided, an error - will be produced. - example: - - example: "- name: --values\n type: long\n choices: [1024, 2048, 4096]\n" - format: yaml - name: choices - type: List of Long -- description: 'Structured information. Can be any shape: a string, vector, map or - even nested map.' - example: - - example: "info:\n category: cat1\n labels: [one, two, three]" - format: yaml - name: info - since: Viash 0.6.3 - type: Json -- description: Maximum allowed value for this argument. If set and the provided value - is higher than the maximum, an error will be produced. Can be combined with [`min`](#min) - to clamp values. - example: - - example: "- name: --my_long\n type: long\n max: 150\n" - format: yaml - name: max - type: Option of Long -- description: The default value when no argument value is provided. This will not - work if the [`required`](#required) property is enabled. - example: - - example: "- name: --my_long\n type: long\n default: 100\n" - format: yaml - name: default - type: OneOrMore of Long -- description: An example value for this argument. If no [`default`](#default) property - was specified, this will be used for that purpose. - example: - - example: "- name: --my_long\n type: long\n example: 100\n" - format: yaml - name: example - type: OneOrMore of Long -- description: A description of the argument. This will be displayed with `--help`. - name: description - type: Option of String -- description: The delimiter character for providing [`multiple`](#multiple) values. - `:` by default. - example: - - example: "- name: --my_long\n type: long\n multiple: true\n multiple_sep: \"\ - ,\"\n" - format: yaml - - description: 'Here''s an example of how to use this:' - example: my_component --my_long=10:80:152 - format: bash - name: multiple_sep - type: String -- description: Minimum allowed value for this argument. If set and the provided value - is lower than the minimum, an error will be produced. Can be combined with [`max`](#max) - to clamp values. - example: - - example: "- name: --my_long\n type: long\n min: 50\n" - format: yaml - name: min - type: Option of Long -- description: Treat the argument value as an array. Arrays can be passed using the - delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo - 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) - property. `false` by default. - example: - - example: "- name: --my_long\n type: long\n multiple: true\n" - format: yaml - - description: 'Here''s an example of how to use this:' - example: my_component --my_long=10:80:152 - format: bash - name: multiple - type: Boolean -- description: Specifies the type of the argument. - name: type - type: String -- description: Make the value for this argument required. If set to `true`, an error - will be produced if no value was provided. `false` by default. - example: - - example: "- name: --my_long\n type: long\n required: true\n" - format: yaml - name: required - type: Boolean -title: Long -topic: arguments diff --git a/reference/config/functionality/arguments/_string.yaml b/reference/config/functionality/arguments/_string.yaml deleted file mode 100644 index 5b86fd2d..00000000 --- a/reference/config/functionality/arguments/_string.yaml +++ /dev/null @@ -1,94 +0,0 @@ -data: -- description: A `string` type argument has a value made up of an ordered sequences - of characters, like "Hello" or "I'm a string". - example: - - example: "arguments:\n - name: --search_query\n type: string\n default:\ - \ \"meaning of life\"\n description: The term to search for\n alternatives:\ - \ [\"-q\"]\n" - format: yaml - hierarchy: - - io.viash.functionality.arguments.StringArgument - - io.viash.functionality.arguments.Argument - name: __this__ - type: StringArgument -- description: List of alternative format variations for this argument. - name: alternatives - type: OneOrMore of String -- description: "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`.\ - \ The number of dashes determines how values can be passed: \n\n - `--foo` is\ - \ a long option, which can be passed with `executable_name --foo=value` or `executable_name\ - \ --foo value`\n - `-f` is a short option, which can be passed with `executable_name\ - \ -f value`\n - `foo` is an argument, which can be passed with `executable_name\ - \ value` \n" - name: name - type: String -- description: Limit the amount of valid values for this argument to those set in - this list. When set and a value not present in the list is provided, an error - will be produced. - example: - - example: "- name: --language\n type: string\n choices: [\"python\", \"r\", \"\ - javascript\"]\n" - format: yaml - name: choices - type: List of String -- description: 'Structured information. Can be any shape: a string, vector, map or - even nested map.' - example: - - example: "info:\n category: cat1\n labels: [one, two, three]" - format: yaml - name: info - since: Viash 0.6.3 - type: Json -- description: The default value when no argument value is provided. This will not - work if the [`required`](#required) property is enabled. - example: - - example: "- name: --my_string\n type: string\n default: \"The answer is 42\"\ - \n" - format: yaml - name: default - type: OneOrMore of String -- description: An example value for this argument. If no [`default`](#default) property - was specified, this will be used for that purpose. - example: - - example: "- name: --my_string\n type: string\n example: \"Hello World\"\n" - format: yaml - name: example - type: OneOrMore of String -- description: A description of the argument. This will be displayed with `--help`. - name: description - type: Option of String -- description: The delimiter character for providing [`multiple`](#multiple) values. - `:` by default. - example: - - example: "- name: --my_string\n type: string\n multiple: true\n multiple_sep:\ - \ \",\"\n" - format: yaml - - description: 'Here''s an example of how to use this:' - example: my_component --my_string=Marc,Susan,Paul - format: bash - name: multiple_sep - type: String -- description: Treat the argument value as an array. Arrays can be passed using the - delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo - 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) - property. `false` by default. - example: - - example: "- name: --my_string\n type: string\n multiple: true\n" - format: yaml - - description: 'Here''s an example of how to use this:' - example: my_component --my_string=Marc:Susan:Paul - format: bash - name: multiple - type: Boolean -- description: Specifies the type of the argument. - name: type - type: String -- description: Make the value for this argument required. If set to `true`, an error - will be produced if no value was provided. `false` by default. - example: - - example: "- name: --my_string\n type: string\n required: true\n" - format: yaml - name: required - type: Boolean -title: String -topic: arguments diff --git a/reference/config/functionality/arguments/boolean.qmd b/reference/config/functionality/arguments/boolean.qmd index 874120ce..f914ff61 100644 --- a/reference/config/functionality/arguments/boolean.qmd +++ b/reference/config/functionality/arguments/boolean.qmd @@ -21,12 +21,16 @@ arguments: **Type**: `String` / `List of String` +**Default**: `Empty` + List of alternative format variations for this argument. ## default **Type**: `Boolean` / `List of Boolean` +**Default**: `Empty` + The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled. **Example:** @@ -42,12 +46,16 @@ The default value when no argument value is provided. This will not work if the **Type**: `String` +**Default**: `Empty` + A description of the argument. This will be displayed with `--help`. ## example **Type**: `Boolean` / `List of Boolean` +**Default**: `Empty` + An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose. **Example:** @@ -63,6 +71,8 @@ An example value for this argument. If no [`default`](#default) property was spe **Type**: `Json` +**Default**: `Empty` + Structured information. Can be any shape: a string, vector, map or even nested map. **Example:** @@ -77,6 +87,8 @@ info: **Type**: `Boolean` +**Default**: `False` + Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default. **Examples:** @@ -98,6 +110,8 @@ my_component --my_boolean=true:true:false **Type**: `String` +**Default**: `:` + The delimiter character for providing [`multiple`](#multiple) values. `:` by default. **Examples:** @@ -131,6 +145,8 @@ The name of the argument. Can be in the formats `--trim`, `-t` or `trim`. The nu **Type**: `Boolean` +**Default**: `False` + Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default. **Example:** @@ -147,4 +163,3 @@ Make the value for this argument required. If set to `true`, an error will be pr **Type**: `String` Specifies the type of the argument. - diff --git a/reference/config/functionality/arguments/boolean_false.qmd b/reference/config/functionality/arguments/boolean_false.qmd index 8c85f99d..eb1c0fd5 100644 --- a/reference/config/functionality/arguments/boolean_false.qmd +++ b/reference/config/functionality/arguments/boolean_false.qmd @@ -1,5 +1,5 @@ --- -title: "Boolean_False" +title: "Boolean False" search: true --- @@ -20,18 +20,24 @@ arguments: **Type**: `String` / `List of String` +**Default**: `Empty` + List of alternative format variations for this argument. ## description **Type**: `String` +**Default**: `Empty` + A description of the argument. This will be displayed with `--help`. ## info **Type**: `Json` +**Default**: `Empty` + Structured information. Can be any shape: a string, vector, map or even nested map. **Example:** @@ -58,4 +64,3 @@ The name of the argument. Can be in the formats `--no-log`, `-n` or `no-log`. Th **Type**: `String` Specifies the type of the argument. - diff --git a/reference/config/functionality/arguments/boolean_true.qmd b/reference/config/functionality/arguments/boolean_true.qmd index 79525873..ff2c2d6e 100644 --- a/reference/config/functionality/arguments/boolean_true.qmd +++ b/reference/config/functionality/arguments/boolean_true.qmd @@ -1,5 +1,5 @@ --- -title: "Boolean_True" +title: "Boolean True" search: true --- @@ -20,18 +20,24 @@ arguments: **Type**: `String` / `List of String` +**Default**: `Empty` + List of alternative format variations for this argument. ## description **Type**: `String` +**Default**: `Empty` + A description of the argument. This will be displayed with `--help`. ## info **Type**: `Json` +**Default**: `Empty` + Structured information. Can be any shape: a string, vector, map or even nested map. **Example:** @@ -58,4 +64,3 @@ The name of the argument. Can be in the formats `--silent`, `-s` or `silent`. Th **Type**: `String` Specifies the type of the argument. - diff --git a/reference/config/functionality/arguments/double.qmd b/reference/config/functionality/arguments/double.qmd index 67573641..a2c7b16a 100644 --- a/reference/config/functionality/arguments/double.qmd +++ b/reference/config/functionality/arguments/double.qmd @@ -21,12 +21,16 @@ arguments: **Type**: `String` / `List of String` +**Default**: `Empty` + List of alternative format variations for this argument. ## default **Type**: `Double` / `List of Double` +**Default**: `Empty` + The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled. **Example:** @@ -42,12 +46,16 @@ The default value when no argument value is provided. This will not work if the **Type**: `String` +**Default**: `Empty` + A description of the argument. This will be displayed with `--help`. ## example **Type**: `Double` / `List of Double` +**Default**: `Empty` + An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose. **Example:** @@ -63,6 +71,8 @@ An example value for this argument. If no [`default`](#default) property was spe **Type**: `Json` +**Default**: `Empty` + Structured information. Can be any shape: a string, vector, map or even nested map. **Example:** @@ -77,6 +87,8 @@ info: **Type**: `Double` +**Default**: `Empty` + Maximum allowed value for this argument. If set and the provided value is higher than the maximum, an error will be produced. Can be combined with [`min`](#min) to clamp values. **Example:** @@ -92,6 +104,8 @@ Maximum allowed value for this argument. If set and the provided value is higher **Type**: `Double` +**Default**: `Empty` + Minimum allowed value for this argument. If set and the provided value is lower than the minimum, an error will be produced. Can be combined with [`max`](#max) to clamp values. **Example:** @@ -107,6 +121,8 @@ Minimum allowed value for this argument. If set and the provided value is lower **Type**: `Boolean` +**Default**: `False` + Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default. **Examples:** @@ -128,6 +144,8 @@ my_component --my_double=5.8:22.6:200.4 **Type**: `String` +**Default**: `:` + The delimiter character for providing [`multiple`](#multiple) values. `:` by default. **Examples:** @@ -161,6 +179,8 @@ The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The numb **Type**: `Boolean` +**Default**: `False` + Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default. **Example:** @@ -177,4 +197,3 @@ Make the value for this argument required. If set to `true`, an error will be pr **Type**: `String` Specifies the type of the argument. - diff --git a/reference/config/functionality/arguments/file.qmd b/reference/config/functionality/arguments/file.qmd index 2f2edce1..c1e5a704 100644 --- a/reference/config/functionality/arguments/file.qmd +++ b/reference/config/functionality/arguments/file.qmd @@ -21,12 +21,16 @@ arguments: **Type**: `String` / `List of String` +**Default**: `Empty` + List of alternative format variations for this argument. ## create_parent **Type**: `Boolean` +**Default**: `True` + If the output filename is a path and it does not exist, create it before executing the script (only for `direction: output`). **Example:** @@ -43,6 +47,8 @@ If the output filename is a path and it does not exist, create it before executi **Type**: `Path` / `List of Path` +**Default**: `Empty` + The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled. **Example:** @@ -58,12 +64,16 @@ The default value when no argument value is provided. This will not work if the **Type**: `String` +**Default**: `Empty` + A description of the argument. This will be displayed with `--help`. ## direction **Type**: `Direction` +**Default**: `Input` + Makes this argument an `input` or an `output`, as in does the file/folder needs to be read or written. `input` by default. **Example:** @@ -79,6 +89,8 @@ Makes this argument an `input` or an `output`, as in does the file/folder needs **Type**: `Path` / `List of Path` +**Default**: `Empty` + An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose. **Example:** @@ -94,6 +106,8 @@ An example value for this argument. If no [`default`](#default) property was spe **Type**: `Json` +**Default**: `Empty` + Structured information. Can be any shape: a string, vector, map or even nested map. **Example:** @@ -108,6 +122,8 @@ info: **Type**: `Boolean` +**Default**: `False` + Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default. **Examples:** @@ -129,6 +145,8 @@ my_component --my_files=firstFile.csv:anotherFile.csv:yetAnother.csv **Type**: `String` +**Default**: `:` + The delimiter character for providing [`multiple`](#multiple) values. `:` by default. **Examples:** @@ -151,6 +169,8 @@ my_component --my_files=firstFile.csv,anotherFile.csv,yetAnother.csv **Type**: `Boolean` +**Default**: `True` + Checks whether the file or folder exists. For input files, this check will happen before the execution of the script, while for output files the check will happen afterwards. **Example:** @@ -177,6 +197,8 @@ The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The numb **Type**: `Boolean` +**Default**: `False` + Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default. **Example:** @@ -193,4 +215,3 @@ Make the value for this argument required. If set to `true`, an error will be pr **Type**: `String` Specifies the type of the argument. - diff --git a/reference/config/functionality/arguments/index.qmd b/reference/config/functionality/arguments/index.qmd index b2a660e9..a6f1dbe2 100644 --- a/reference/config/functionality/arguments/index.qmd +++ b/reference/config/functionality/arguments/index.qmd @@ -32,4 +32,3 @@ arguments: type: string ``` - diff --git a/reference/config/functionality/arguments/integer.qmd b/reference/config/functionality/arguments/integer.qmd index cb83bf0e..6b2c6b47 100644 --- a/reference/config/functionality/arguments/integer.qmd +++ b/reference/config/functionality/arguments/integer.qmd @@ -21,12 +21,16 @@ arguments: **Type**: `String` / `List of String` +**Default**: `Empty` + List of alternative format variations for this argument. ## choices **Type**: `List of Int` +**Default**: `Empty` + Limit the amount of valid values for this argument to those set in this list. When set and a value not present in the list is provided, an error will be produced. **Example:** @@ -42,6 +46,8 @@ Limit the amount of valid values for this argument to those set in this list. Wh **Type**: `Int` / `List of Int` +**Default**: `Empty` + The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled. **Example:** @@ -57,12 +63,16 @@ The default value when no argument value is provided. This will not work if the **Type**: `String` +**Default**: `Empty` + A description of the argument. This will be displayed with `--help`. ## example **Type**: `Int` / `List of Int` +**Default**: `Empty` + An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose. **Example:** @@ -78,6 +88,8 @@ An example value for this argument. If no [`default`](#default) property was spe **Type**: `Json` +**Default**: `Empty` + Structured information. Can be any shape: a string, vector, map or even nested map. **Example:** @@ -92,6 +104,8 @@ info: **Type**: `Int` +**Default**: `Empty` + Maximum allowed value for this argument. If set and the provided value is higher than the maximum, an error will be produced. Can be combined with [`min`](#min) to clamp values. **Example:** @@ -107,6 +121,8 @@ Maximum allowed value for this argument. If set and the provided value is higher **Type**: `Int` +**Default**: `Empty` + Minimum allowed value for this argument. If set and the provided value is lower than the minimum, an error will be produced. Can be combined with [`max`](#max) to clamp values. **Example:** @@ -122,6 +138,8 @@ Minimum allowed value for this argument. If set and the provided value is lower **Type**: `Boolean` +**Default**: `False` + Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default. **Examples:** @@ -143,6 +161,8 @@ my_component --my_integer=10:80:152 **Type**: `String` +**Default**: `:` + The delimiter character for providing [`multiple`](#multiple) values. `:` by default. **Examples:** @@ -176,6 +196,8 @@ The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The numb **Type**: `Boolean` +**Default**: `False` + Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default. **Example:** @@ -192,4 +214,3 @@ Make the value for this argument required. If set to `true`, an error will be pr **Type**: `String` Specifies the type of the argument. - diff --git a/reference/config/functionality/arguments/long.qmd b/reference/config/functionality/arguments/long.qmd index 6071ef01..b4e33d4b 100644 --- a/reference/config/functionality/arguments/long.qmd +++ b/reference/config/functionality/arguments/long.qmd @@ -21,12 +21,16 @@ arguments: **Type**: `String` / `List of String` +**Default**: `Empty` + List of alternative format variations for this argument. ## choices **Type**: `List of Long` +**Default**: `Empty` + Limit the amount of valid values for this argument to those set in this list. When set and a value not present in the list is provided, an error will be produced. **Example:** @@ -42,6 +46,8 @@ Limit the amount of valid values for this argument to those set in this list. Wh **Type**: `Long` / `List of Long` +**Default**: `Empty` + The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled. **Example:** @@ -57,12 +63,16 @@ The default value when no argument value is provided. This will not work if the **Type**: `String` +**Default**: `Empty` + A description of the argument. This will be displayed with `--help`. ## example **Type**: `Long` / `List of Long` +**Default**: `Empty` + An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose. **Example:** @@ -78,6 +88,8 @@ An example value for this argument. If no [`default`](#default) property was spe **Type**: `Json` +**Default**: `Empty` + Structured information. Can be any shape: a string, vector, map or even nested map. **Example:** @@ -92,6 +104,8 @@ info: **Type**: `Long` +**Default**: `Empty` + Maximum allowed value for this argument. If set and the provided value is higher than the maximum, an error will be produced. Can be combined with [`min`](#min) to clamp values. **Example:** @@ -107,6 +121,8 @@ Maximum allowed value for this argument. If set and the provided value is higher **Type**: `Long` +**Default**: `Empty` + Minimum allowed value for this argument. If set and the provided value is lower than the minimum, an error will be produced. Can be combined with [`max`](#max) to clamp values. **Example:** @@ -122,6 +138,8 @@ Minimum allowed value for this argument. If set and the provided value is lower **Type**: `Boolean` +**Default**: `False` + Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default. **Examples:** @@ -143,6 +161,8 @@ my_component --my_long=10:80:152 **Type**: `String` +**Default**: `:` + The delimiter character for providing [`multiple`](#multiple) values. `:` by default. **Examples:** @@ -176,6 +196,8 @@ The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The numb **Type**: `Boolean` +**Default**: `False` + Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default. **Example:** @@ -192,4 +214,3 @@ Make the value for this argument required. If set to `true`, an error will be pr **Type**: `String` Specifies the type of the argument. - diff --git a/reference/config/functionality/arguments/string.qmd b/reference/config/functionality/arguments/string.qmd index 857764be..84758496 100644 --- a/reference/config/functionality/arguments/string.qmd +++ b/reference/config/functionality/arguments/string.qmd @@ -21,12 +21,16 @@ arguments: **Type**: `String` / `List of String` +**Default**: `Empty` + List of alternative format variations for this argument. ## choices **Type**: `List of String` +**Default**: `Empty` + Limit the amount of valid values for this argument to those set in this list. When set and a value not present in the list is provided, an error will be produced. **Example:** @@ -42,6 +46,8 @@ Limit the amount of valid values for this argument to those set in this list. Wh **Type**: `String` / `List of String` +**Default**: `Empty` + The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled. **Example:** @@ -57,12 +63,16 @@ The default value when no argument value is provided. This will not work if the **Type**: `String` +**Default**: `Empty` + A description of the argument. This will be displayed with `--help`. ## example **Type**: `String` / `List of String` +**Default**: `Empty` + An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose. **Example:** @@ -78,6 +88,8 @@ An example value for this argument. If no [`default`](#default) property was spe **Type**: `Json` +**Default**: `Empty` + Structured information. Can be any shape: a string, vector, map or even nested map. **Example:** @@ -92,6 +104,8 @@ info: **Type**: `Boolean` +**Default**: `False` + Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default. **Examples:** @@ -113,6 +127,8 @@ my_component --my_string=Marc:Susan:Paul **Type**: `String` +**Default**: `:` + The delimiter character for providing [`multiple`](#multiple) values. `:` by default. **Examples:** @@ -146,6 +162,8 @@ The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The numb **Type**: `Boolean` +**Default**: `Empty` + Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default. **Example:** @@ -162,4 +180,3 @@ Make the value for this argument required. If set to `true`, an error will be pr **Type**: `String` Specifies the type of the argument. - diff --git a/reference/config/functionality/author.qmd b/reference/config/functionality/author.qmd index 91f20386..d20e6570 100644 --- a/reference/config/functionality/author.qmd +++ b/reference/config/functionality/author.qmd @@ -23,12 +23,16 @@ info: **Type**: `String` +**Default**: `Empty` + E-mail of the author. ## info **Type**: `Json` +**Default**: `Empty` + Structured information. Can be any shape: a string, vector, map or even nested map. ## name @@ -39,7 +43,9 @@ Full name of the author, usually in the name of FirstName MiddleName LastName. ## props -**Type**: `Map of String,String` +**Type**: `Map of String to String` + +**Default**: `Empty` ::: {.callout-warning} Deprecated since 0.7.4. Planned removal at 0.8.0. Use `info` instead. @@ -50,10 +56,11 @@ Author properties. Must be a map of strings. **Type**: `String` / `List of String` +**Default**: `Empty` + Role of the author. Suggested items: * `"author"`: Authors who have made substantial contributions to the component. * `"maintainer"`: The maintainer of the component. * `"contributor"`: Authors who have made smaller contributions (such as code patches etc.). - diff --git a/reference/config/functionality/computationalRequirements.qmd b/reference/config/functionality/computationalRequirements.qmd index cae6ff8f..5af86064 100644 --- a/reference/config/functionality/computationalRequirements.qmd +++ b/reference/config/functionality/computationalRequirements.qmd @@ -9,6 +9,8 @@ Computational requirements related to running the component. **Type**: `List of String` +**Default**: `Empty` + A list of commands which should be present on the system for the script to function. **Example:** @@ -21,6 +23,8 @@ commands: [ which, bash, awk, date, grep, egrep, ps, sed, tail, tee ] **Type**: `Int` +**Default**: `Empty` + The maximum number of (logical) cpus a component is allowed to use. **Example:** @@ -33,6 +37,8 @@ cpus: 10 **Type**: `String` +**Default**: `Empty` + The maximum amount of memory a component is allowed to allocate. Unit must be one of B, KB, MB, GB, TB or PB. **Example:** @@ -40,13 +46,3 @@ The maximum amount of memory a component is allowed to allocate. Unit must be on ```yaml memory: 10GB ``` - -## n_proc - -**Type**: `Int` - -::: {.callout-warning} -Deprecated since 0.6.1. -Removed since 0.7.0. Use `cpus` instead. -::: - diff --git a/reference/config/functionality/index.qmd b/reference/config/functionality/index.qmd index eecc5c22..6faa3d23 100644 --- a/reference/config/functionality/index.qmd +++ b/reference/config/functionality/index.qmd @@ -8,25 +8,17 @@ The functionality-part of the config file describes the behaviour of the script By specifying a few restrictions (e.g. mandatory arguments) and adding some descriptions, Viash will automatically generate a stylish command-line interface for you. -## add_resources_to_path - -**Type**: `Boolean` - -::: {.callout-warning} - -Removed since 0.5.11. Extending the PATH turned out to be not desirable. -::: -Adds the resources directory to the PATH variable when set to true. This is set to false by default. - ## argument_groups **Type**: `List of ArgumentGroup` +**Default**: `Empty` + A grouping of the [arguments](/reference/config/functionality/arguments/#), used to display the help message. - `name: foo`, the name of the argument group. - `description: Description of foo`, a description of the argument group. Multiline descriptions are supported. - - `arguments: [arg1, arg2, ...]`, list of the arguments names. + - `arguments: [arg1, arg2, ...]`, list of the arguments. @@ -79,6 +71,8 @@ component_name **Type**: `List of Argument` +**Default**: `Empty` + A list of [arguments](/reference/config/functionality/arguments/#) for this component. For each argument, a type and a name must be specified. Depending on the type of argument, different properties can be set. See these reference pages per type for more information: - [string](/reference/config/functionality/arguments/string.html) @@ -113,6 +107,8 @@ arguments: **Type**: `List of Author` +**Default**: `Empty` + A list of [authors](/reference/config/functionality/author.html). An author must at least have a name, but can also have a list of roles, an e-mail address, and a map of custom properties. Suggested values for roles are: @@ -151,6 +147,8 @@ authors: **Type**: `String` +**Default**: `Empty` + A description of the component. This will be displayed with `--help`. **Example:** @@ -162,20 +160,12 @@ description: | ``` -## enabled - -**Type**: `Boolean` - -::: {.callout-warning} -Deprecated since 0.6.0. -Removed since 0.7.0. Use `status` instead. -::: -Setting this to false with disable this component when using namespaces. - ## info **Type**: `Json` +**Default**: `Empty` + Structured information. Can be any shape: a string, vector, map or even nested map. **Example:** @@ -186,37 +176,6 @@ info: classes: [ one, two, three ] ``` -## inputs - -**Type**: `List of Argument` - -::: {.callout-warning} -Deprecated since 0.6.0. -Removed since 0.7.0. Use `arguments` instead. -::: -A list of input arguments in addition to the `arguments` list. Any arguments specified here will have their `type` set to `file` and the `direction` set to `input` by default. - -**Examples:** - -```yaml -inputs: - - name: input_file - - name: another_input -``` - -This results in the following output when calling the component with the `--help` argument: - -```bash -component_with_inputs - - Inputs: - input_file - type: file - - another_input - type: file -``` - ## name **Type**: `String` @@ -233,6 +192,8 @@ name: this_is_my_component **Type**: `String` +**Default**: `Empty` + Namespace this component is a part of. See the [Namespaces guide](/guide/project/structure.html#grouping-components-in-namespaces) for more information on namespaces. **Example:** @@ -241,41 +202,12 @@ Namespace this component is a part of. See the [Namespaces guide](/guide/project namespace: fancy_components ``` -## outputs - -**Type**: `List of Argument` - -::: {.callout-warning} -Deprecated since 0.6.0. -Removed since 0.7.0. Use `arguments` instead. -::: -A list of output arguments in addition to the `arguments` list. Any arguments specified here will have their `type` set to `file` and thr `direction` set to `output` by default. - -**Examples:** - -```yaml -outputs: - - name: output_file - - name: another_output -``` - -This results in the following output when calling the component with the `--help` argument: - -```bash -component_with_outputs - - Outputs: - output_file - type: file, output - - another_output - type: file, output -``` - ## requirements **Type**: `ComputationalRequirements` +**Default**: `Empty` + [Computational requirements](/reference/config/functionality/computationalRequirements.html) related to running the component. `cpus` specifies the maximum number of (logical) cpus a component is allowed to use., whereas `memory` specifies the maximum amount of memory a component is allowed to allicate. Memory units must be @@ -294,6 +226,8 @@ requirements: **Type**: `List of Resource` +**Default**: `Empty` + [Resources](/reference/config/functionality/resources/#) are files that support the component. The first resource should be [a script](/guide/component/create-component.html#create-a-script) that will be executed when the functionality is run. Additional resources will be copied to the same directory. Common properties: @@ -320,12 +254,16 @@ resources: **Type**: `Status` +**Default**: `Enabled` + Allows setting a component to active, deprecated or disabled. ## test_resources **Type**: `List of Resource` +**Default**: `Empty` + One or more [scripts](/guide/component/create-component.html#create-a-script) to be used to test the component behaviour when `viash test` is invoked. Additional files of type `file` will be made available only during testing. Each test script should expect no command-line inputs, be platform-independent, and return an exit code >0 when unexpected behaviour occurs during testing. See [Unit Testing](/guide/component/unit-testing.html) for more info. **Example:** @@ -340,20 +278,12 @@ test_resources: ``` -## tests - -**Type**: `List of Resource` - -::: {.callout-warning} -Deprecated since 0.5.13. -Removed since 0.7.0. Use `test_resources` instead. No functional difference. -::: -One or more Bash/R/Python scripts to be used to test the component behaviour when `viash test` is invoked. Additional files of type `file` will be made available only during testing. Each test script should expect no command-line inputs, be platform-independent, and return an exit code >0 when unexpected behaviour occurs during testing. - ## usage **Type**: `String` +**Default**: `Empty` + A description on how to use the component. This will be displayed with `--help` under the 'Usage:' section. **Example:** @@ -366,6 +296,8 @@ usage: Place the executable in a directory containing TSV files and run it **Type**: `String` +**Default**: `Empty` + Version of the component. This field will be used to version the executable and the Docker container. **Example:** @@ -373,4 +305,3 @@ Version of the component. This field will be used to version the executable and ```yaml version: 0.8 ``` - diff --git a/reference/config/functionality/resources/_bashScript.yaml b/reference/config/functionality/resources/_bashScript.yaml deleted file mode 100644 index 6b2f012b..00000000 --- a/reference/config/functionality/resources/_bashScript.yaml +++ /dev/null @@ -1,35 +0,0 @@ -data: -- description: 'An executable Bash script. - - When defined in functionality.resources, only the first entry will be executed - when running the built component or when running `viash run`. - - When defined in functionality.test_resources, all entries will be executed during - `viash test`.' - hierarchy: - - io.viash.functionality.resources.BashScript - - io.viash.functionality.resources.Script - - io.viash.functionality.resources.Resource - name: __this__ - type: BashScript -- description: The path of the input file. Can be a relative or an absolute path, - or a URI. Mutually exclusive with `text`. - name: path - type: Option of String -- description: The content of the resulting file specified as a string. Mutually exclusive - with `path`. - name: text - type: Option of String -- description: Whether the resulting resource file should be made executable. - name: is_executable - type: Option of Boolean -- description: Specifies the resource as a Bash script. - name: type - type: String -- description: Resulting filename of the resource. From within a script, the file - can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` - will be set to the basename of the `path` parameter. - name: dest - type: Option of String -title: Bash Script -topic: resources diff --git a/reference/config/functionality/resources/_cSharpScript.yaml b/reference/config/functionality/resources/_cSharpScript.yaml deleted file mode 100644 index 4ef188fc..00000000 --- a/reference/config/functionality/resources/_cSharpScript.yaml +++ /dev/null @@ -1,35 +0,0 @@ -data: -- description: 'An executable C# script. - - When defined in functionality.resources, only the first entry will be executed - when running the built component or when running `viash run`. - - When defined in functionality.test_resources, all entries will be executed during - `viash test`.' - hierarchy: - - io.viash.functionality.resources.CSharpScript - - io.viash.functionality.resources.Script - - io.viash.functionality.resources.Resource - name: __this__ - type: CSharpScript -- description: The path of the input file. Can be a relative or an absolute path, - or a URI. Mutually exclusive with `text`. - name: path - type: Option of String -- description: The content of the resulting file specified as a string. Mutually exclusive - with `path`. - name: text - type: Option of String -- description: Whether the resulting resource file should be made executable. - name: is_executable - type: Option of Boolean -- description: Specifies the resource as a C# script. - name: type - type: String -- description: Resulting filename of the resource. From within a script, the file - can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` - will be set to the basename of the `path` parameter. - name: dest - type: Option of String -title: C Sharp Script -topic: resources diff --git a/reference/config/functionality/resources/_executable.yaml b/reference/config/functionality/resources/_executable.yaml deleted file mode 100644 index dd136b5a..00000000 --- a/reference/config/functionality/resources/_executable.yaml +++ /dev/null @@ -1,29 +0,0 @@ -data: -- description: An executable file. - hierarchy: - - io.viash.functionality.resources.Executable - - io.viash.functionality.resources.Script - - io.viash.functionality.resources.Resource - name: __this__ - type: Executable -- description: The path of the input file. Can be a relative or an absolute path, - or a URI. Mutually exclusive with `text`. - name: path - type: Option of String -- description: The content of the resulting file specified as a string. Mutually exclusive - with `path`. - name: text - type: Option of String -- description: Whether the resulting resource file should be made executable. - name: is_executable - type: Option of Boolean -- description: Specifies the resource as an executable. - name: type - type: String -- description: Resulting filename of the resource. From within a script, the file - can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` - will be set to the basename of the `path` parameter. - name: dest - type: Option of String -title: Executable -topic: resources diff --git a/reference/config/functionality/resources/_index.yaml b/reference/config/functionality/resources/_index.yaml deleted file mode 100644 index 257894c4..00000000 --- a/reference/config/functionality/resources/_index.yaml +++ /dev/null @@ -1,26 +0,0 @@ -data: -- description: "Resources are files that support the component. The first resource\ - \ should be [a script](/guide/component/create-component.html#create-a-script)\ - \ that will be executed when the functionality is run. Additional resources will\ - \ be copied to the same directory.\n\nCommon properties:\n\n * type: `file` /\ - \ `r_script` / `python_script` / `bash_script` / `javascript_script` / `scala_script`\ - \ / `csharp_script`, specifies the type of the resource. The first resource cannot\ - \ be of type `file`. When the type is not specified, the default type is simply\ - \ `file`.\n * dest: filename, the resulting name of the resource. From within\ - \ a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`.\ - \ If unspecified, `dest` will be set to the basename of the `path` parameter.\n\ - \ * path: `path/to/file`, the path of the input file. Can be a relative or an\ - \ absolute path, or a URI. Mutually exclusive with `text`.\n * text: ...multiline\ - \ text..., the content of the resulting file specified as a string. Mutually exclusive\ - \ with `path`.\n * is_executable: `true` / `false`, whether the resulting resource\ - \ file should be made executable.\n" - example: - - example: "resources:\n - type: r_script\n path: script.R\n - type: file\n\ - \ path: resource1.txt\n" - format: yaml - hierarchy: - - io.viash.functionality.resources.Resource - name: __this__ - type: Resource -title: Resource -topic: resources diff --git a/reference/config/functionality/resources/_javaScriptScript.yaml b/reference/config/functionality/resources/_javaScriptScript.yaml deleted file mode 100644 index 7c9c2299..00000000 --- a/reference/config/functionality/resources/_javaScriptScript.yaml +++ /dev/null @@ -1,35 +0,0 @@ -data: -- description: 'An executable JavaScript script. - - When defined in functionality.resources, only the first entry will be executed - when running the built component or when running `viash run`. - - When defined in functionality.test_resources, all entries will be executed during - `viash test`.' - hierarchy: - - io.viash.functionality.resources.JavaScriptScript - - io.viash.functionality.resources.Script - - io.viash.functionality.resources.Resource - name: __this__ - type: JavaScriptScript -- description: The path of the input file. Can be a relative or an absolute path, - or a URI. Mutually exclusive with `text`. - name: path - type: Option of String -- description: The content of the resulting file specified as a string. Mutually exclusive - with `path`. - name: text - type: Option of String -- description: Whether the resulting resource file should be made executable. - name: is_executable - type: Option of Boolean -- description: Specifies the resource as a JavaScript script. - name: type - type: String -- description: Resulting filename of the resource. From within a script, the file - can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` - will be set to the basename of the `path` parameter. - name: dest - type: Option of String -title: Java Script Script -topic: resources diff --git a/reference/config/functionality/resources/_nextflowScript.yaml b/reference/config/functionality/resources/_nextflowScript.yaml deleted file mode 100644 index 6ca1f420..00000000 --- a/reference/config/functionality/resources/_nextflowScript.yaml +++ /dev/null @@ -1,33 +0,0 @@ -data: -- description: A Nextflow script. Work in progress; added mainly for annotation at - the moment. - hierarchy: - - io.viash.functionality.resources.NextflowScript - - io.viash.functionality.resources.Script - - io.viash.functionality.resources.Resource - name: __this__ - type: NextflowScript -- description: The path of the input file. Can be a relative or an absolute path, - or a URI. Mutually exclusive with `text`. - name: path - type: Option of String -- description: The content of the resulting file specified as a string. Mutually exclusive - with `path`. - name: text - type: Option of String -- description: The name of the workflow to be executed. - name: entrypoint - type: Option of String -- description: Whether the resulting resource file should be made executable. - name: is_executable - type: Option of Boolean -- description: Specifies the resource as a Nextflow script. - name: type - type: String -- description: Resulting filename of the resource. From within a script, the file - can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` - will be set to the basename of the `path` parameter. - name: dest - type: Option of String -title: Nextflow Script -topic: resources diff --git a/reference/config/functionality/resources/_plainFile.yaml b/reference/config/functionality/resources/_plainFile.yaml deleted file mode 100644 index d928d6bc..00000000 --- a/reference/config/functionality/resources/_plainFile.yaml +++ /dev/null @@ -1,29 +0,0 @@ -data: -- description: A plain file. This can only be used as a supporting resource for the - main script or unit tests. - hierarchy: - - io.viash.functionality.resources.PlainFile - - io.viash.functionality.resources.Resource - name: __this__ - type: PlainFile -- description: The path of the input file. Can be a relative or an absolute path, - or a URI. Mutually exclusive with `text`. - name: path - type: Option of String -- description: The content of the resulting file specified as a string. Mutually exclusive - with `path`. - name: text - type: Option of String -- description: Whether the resulting resource file should be made executable. - name: is_executable - type: Option of Boolean -- description: Specifies the resource as a plain file. - name: type - type: String -- description: Resulting filename of the resource. From within a script, the file - can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` - will be set to the basename of the `path` parameter. - name: dest - type: Option of String -title: Plain File -topic: resources diff --git a/reference/config/functionality/resources/_pythonScript.yaml b/reference/config/functionality/resources/_pythonScript.yaml deleted file mode 100644 index a3f41f98..00000000 --- a/reference/config/functionality/resources/_pythonScript.yaml +++ /dev/null @@ -1,35 +0,0 @@ -data: -- description: 'An executable Python script. - - When defined in functionality.resources, only the first entry will be executed - when running the built component or when running `viash run`. - - When defined in functionality.test_resources, all entries will be executed during - `viash test`.' - hierarchy: - - io.viash.functionality.resources.PythonScript - - io.viash.functionality.resources.Script - - io.viash.functionality.resources.Resource - name: __this__ - type: PythonScript -- description: The path of the input file. Can be a relative or an absolute path, - or a URI. Mutually exclusive with `text`. - name: path - type: Option of String -- description: The content of the resulting file specified as a string. Mutually exclusive - with `path`. - name: text - type: Option of String -- description: Whether the resulting resource file should be made executable. - name: is_executable - type: Option of Boolean -- description: Specifies the resource as a Python script. - name: type - type: String -- description: Resulting filename of the resource. From within a script, the file - can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` - will be set to the basename of the `path` parameter. - name: dest - type: Option of String -title: Python Script -topic: resources diff --git a/reference/config/functionality/resources/_rScript.yaml b/reference/config/functionality/resources/_rScript.yaml deleted file mode 100644 index a20a956e..00000000 --- a/reference/config/functionality/resources/_rScript.yaml +++ /dev/null @@ -1,35 +0,0 @@ -data: -- description: 'An executable R script. - - When defined in functionality.resources, only the first entry will be executed - when running the built component or when running `viash run`. - - When defined in functionality.test_resources, all entries will be executed during - `viash test`.' - hierarchy: - - io.viash.functionality.resources.RScript - - io.viash.functionality.resources.Script - - io.viash.functionality.resources.Resource - name: __this__ - type: RScript -- description: The path of the input file. Can be a relative or an absolute path, - or a URI. Mutually exclusive with `text`. - name: path - type: Option of String -- description: The content of the resulting file specified as a string. Mutually exclusive - with `path`. - name: text - type: Option of String -- description: Whether the resulting resource file should be made executable. - name: is_executable - type: Option of Boolean -- description: Specifies the resource as a R script. - name: type - type: String -- description: Resulting filename of the resource. From within a script, the file - can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` - will be set to the basename of the `path` parameter. - name: dest - type: Option of String -title: R Script -topic: resources diff --git a/reference/config/functionality/resources/_scalaScript.yaml b/reference/config/functionality/resources/_scalaScript.yaml deleted file mode 100644 index c14e05ea..00000000 --- a/reference/config/functionality/resources/_scalaScript.yaml +++ /dev/null @@ -1,35 +0,0 @@ -data: -- description: 'An executable Scala script. - - When defined in functionality.resources, only the first entry will be executed - when running the built component or when running `viash run`. - - When defined in functionality.test_resources, all entries will be executed during - `viash test`.' - hierarchy: - - io.viash.functionality.resources.ScalaScript - - io.viash.functionality.resources.Script - - io.viash.functionality.resources.Resource - name: __this__ - type: ScalaScript -- description: The path of the input file. Can be a relative or an absolute path, - or a URI. Mutually exclusive with `text`. - name: path - type: Option of String -- description: The content of the resulting file specified as a string. Mutually exclusive - with `path`. - name: text - type: Option of String -- description: Whether the resulting resource file should be made executable. - name: is_executable - type: Option of Boolean -- description: Specifies the resource as a Scala script. - name: type - type: String -- description: Resulting filename of the resource. From within a script, the file - can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` - will be set to the basename of the `path` parameter. - name: dest - type: Option of String -title: Scala Script -topic: resources diff --git a/reference/config/functionality/resources/bashScript.qmd b/reference/config/functionality/resources/bashScript.qmd index 425aa6ea..583084b5 100644 --- a/reference/config/functionality/resources/bashScript.qmd +++ b/reference/config/functionality/resources/bashScript.qmd @@ -11,24 +11,32 @@ When defined in functionality.test_resources, all entries will be executed durin **Type**: `String` +**Default**: `Empty` + Resulting filename of the resource. From within a script, the file can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter. ## is_executable **Type**: `Boolean` +**Default**: `Empty` + Whether the resulting resource file should be made executable. ## path **Type**: `String` +**Default**: `Empty` + The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`. ## text **Type**: `String` +**Default**: `Empty` + The content of the resulting file specified as a string. Mutually exclusive with `path`. ## type @@ -36,4 +44,3 @@ The content of the resulting file specified as a string. Mutually exclusive with **Type**: `String` Specifies the resource as a Bash script. - diff --git a/reference/config/functionality/resources/cSharpScript.qmd b/reference/config/functionality/resources/cSharpScript.qmd index d1de7ac9..45610f64 100644 --- a/reference/config/functionality/resources/cSharpScript.qmd +++ b/reference/config/functionality/resources/cSharpScript.qmd @@ -1,5 +1,5 @@ --- -title: "C Sharp Script" +title: "C# Script" search: true --- @@ -11,24 +11,32 @@ When defined in functionality.test_resources, all entries will be executed durin **Type**: `String` +**Default**: `Empty` + Resulting filename of the resource. From within a script, the file can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter. ## is_executable **Type**: `Boolean` +**Default**: `Empty` + Whether the resulting resource file should be made executable. ## path **Type**: `String` +**Default**: `Empty` + The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`. ## text **Type**: `String` +**Default**: `Empty` + The content of the resulting file specified as a string. Mutually exclusive with `path`. ## type @@ -36,4 +44,3 @@ The content of the resulting file specified as a string. Mutually exclusive with **Type**: `String` Specifies the resource as a C# script. - diff --git a/reference/config/functionality/resources/executable.qmd b/reference/config/functionality/resources/executable.qmd index dabb2520..05a0d5ee 100644 --- a/reference/config/functionality/resources/executable.qmd +++ b/reference/config/functionality/resources/executable.qmd @@ -9,24 +9,32 @@ An executable file. **Type**: `String` +**Default**: `Empty` + Resulting filename of the resource. From within a script, the file can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter. ## is_executable **Type**: `Boolean` +**Default**: `Empty` + Whether the resulting resource file should be made executable. ## path **Type**: `String` +**Default**: `Empty` + The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`. ## text **Type**: `String` +**Default**: `Empty` + The content of the resulting file specified as a string. Mutually exclusive with `path`. ## type @@ -34,4 +42,3 @@ The content of the resulting file specified as a string. Mutually exclusive with **Type**: `String` Specifies the resource as an executable. - diff --git a/reference/config/functionality/resources/index.qmd b/reference/config/functionality/resources/index.qmd index 7416555a..37de5621 100644 --- a/reference/config/functionality/resources/index.qmd +++ b/reference/config/functionality/resources/index.qmd @@ -24,4 +24,3 @@ resources: path: resource1.txt ``` - diff --git a/reference/config/functionality/resources/javaScriptScript.qmd b/reference/config/functionality/resources/javaScriptScript.qmd index bfdf362d..cdc2ca1a 100644 --- a/reference/config/functionality/resources/javaScriptScript.qmd +++ b/reference/config/functionality/resources/javaScriptScript.qmd @@ -1,5 +1,5 @@ --- -title: "Java Script Script" +title: "JavaScript Script" search: true --- @@ -11,24 +11,32 @@ When defined in functionality.test_resources, all entries will be executed durin **Type**: `String` +**Default**: `Empty` + Resulting filename of the resource. From within a script, the file can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter. ## is_executable **Type**: `Boolean` +**Default**: `Empty` + Whether the resulting resource file should be made executable. ## path **Type**: `String` +**Default**: `Empty` + The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`. ## text **Type**: `String` +**Default**: `Empty` + The content of the resulting file specified as a string. Mutually exclusive with `path`. ## type @@ -36,4 +44,3 @@ The content of the resulting file specified as a string. Mutually exclusive with **Type**: `String` Specifies the resource as a JavaScript script. - diff --git a/reference/config/functionality/resources/nextflowScript.qmd b/reference/config/functionality/resources/nextflowScript.qmd index b2ce3e8b..ed65f8cb 100644 --- a/reference/config/functionality/resources/nextflowScript.qmd +++ b/reference/config/functionality/resources/nextflowScript.qmd @@ -9,30 +9,40 @@ A Nextflow script. Work in progress; added mainly for annotation at the moment. **Type**: `String` +**Default**: `Empty` + Resulting filename of the resource. From within a script, the file can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter. ## entrypoint **Type**: `String` +**Default**: `Empty` + The name of the workflow to be executed. ## is_executable **Type**: `Boolean` +**Default**: `Empty` + Whether the resulting resource file should be made executable. ## path **Type**: `String` +**Default**: `Empty` + The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`. ## text **Type**: `String` +**Default**: `Empty` + The content of the resulting file specified as a string. Mutually exclusive with `path`. ## type @@ -40,4 +50,3 @@ The content of the resulting file specified as a string. Mutually exclusive with **Type**: `String` Specifies the resource as a Nextflow script. - diff --git a/reference/config/functionality/resources/plainFile.qmd b/reference/config/functionality/resources/plainFile.qmd index 58060af3..63a5513e 100644 --- a/reference/config/functionality/resources/plainFile.qmd +++ b/reference/config/functionality/resources/plainFile.qmd @@ -9,24 +9,32 @@ A plain file. This can only be used as a supporting resource for the main script **Type**: `String` +**Default**: `Empty` + Resulting filename of the resource. From within a script, the file can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter. ## is_executable **Type**: `Boolean` +**Default**: `Empty` + Whether the resulting resource file should be made executable. ## path **Type**: `String` +**Default**: `Empty` + The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`. ## text **Type**: `String` +**Default**: `Empty` + The content of the resulting file specified as a string. Mutually exclusive with `path`. ## type @@ -34,4 +42,3 @@ The content of the resulting file specified as a string. Mutually exclusive with **Type**: `String` Specifies the resource as a plain file. - diff --git a/reference/config/functionality/resources/pythonScript.qmd b/reference/config/functionality/resources/pythonScript.qmd index 8a7a6786..61f5aba0 100644 --- a/reference/config/functionality/resources/pythonScript.qmd +++ b/reference/config/functionality/resources/pythonScript.qmd @@ -11,24 +11,32 @@ When defined in functionality.test_resources, all entries will be executed durin **Type**: `String` +**Default**: `Empty` + Resulting filename of the resource. From within a script, the file can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter. ## is_executable **Type**: `Boolean` +**Default**: `Empty` + Whether the resulting resource file should be made executable. ## path **Type**: `String` +**Default**: `Empty` + The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`. ## text **Type**: `String` +**Default**: `Empty` + The content of the resulting file specified as a string. Mutually exclusive with `path`. ## type @@ -36,4 +44,3 @@ The content of the resulting file specified as a string. Mutually exclusive with **Type**: `String` Specifies the resource as a Python script. - diff --git a/reference/config/functionality/resources/rScript.qmd b/reference/config/functionality/resources/rScript.qmd index 5bc8b800..dce56a13 100644 --- a/reference/config/functionality/resources/rScript.qmd +++ b/reference/config/functionality/resources/rScript.qmd @@ -11,24 +11,32 @@ When defined in functionality.test_resources, all entries will be executed durin **Type**: `String` +**Default**: `Empty` + Resulting filename of the resource. From within a script, the file can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter. ## is_executable **Type**: `Boolean` +**Default**: `Empty` + Whether the resulting resource file should be made executable. ## path **Type**: `String` +**Default**: `Empty` + The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`. ## text **Type**: `String` +**Default**: `Empty` + The content of the resulting file specified as a string. Mutually exclusive with `path`. ## type @@ -36,4 +44,3 @@ The content of the resulting file specified as a string. Mutually exclusive with **Type**: `String` Specifies the resource as a R script. - diff --git a/reference/config/functionality/resources/scalaScript.qmd b/reference/config/functionality/resources/scalaScript.qmd index 204420b4..c6a58d83 100644 --- a/reference/config/functionality/resources/scalaScript.qmd +++ b/reference/config/functionality/resources/scalaScript.qmd @@ -11,24 +11,32 @@ When defined in functionality.test_resources, all entries will be executed durin **Type**: `String` +**Default**: `Empty` + Resulting filename of the resource. From within a script, the file can be accessed at `meta["resources_dir"] + "/" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter. ## is_executable **Type**: `Boolean` +**Default**: `Empty` + Whether the resulting resource file should be made executable. ## path **Type**: `String` +**Default**: `Empty` + The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`. ## text **Type**: `String` +**Default**: `Empty` + The content of the resulting file specified as a string. Mutually exclusive with `path`. ## type @@ -36,4 +44,3 @@ The content of the resulting file specified as a string. Mutually exclusive with **Type**: `String` Specifies the resource as a Scala script. - diff --git a/reference/config/index.qmd b/reference/config/index.qmd index 27305a19..d988b42b 100644 --- a/reference/config/index.qmd +++ b/reference/config/index.qmd @@ -31,6 +31,8 @@ platforms: **Type**: `File` +**Default**: `Empty` + Config inheritance by including YAML partials. This is useful for defining common APIs in separate files. `__merge__` can be used in any level of the YAML. For example, not just in the config but also in the functionality or any of the platforms. @@ -58,6 +60,5 @@ A list of platforms to generate target artifacts for. - [Native](/reference/config/platforms/native/#) - [Docker](/reference/config/platforms/docker/#) - - [Nextflow VDSL3](/reference/config/platforms/nextflow/#) - + - [Nextflow](/reference/config/platforms/nextflow/#) diff --git a/reference/config/info.qmd b/reference/config/info.qmd new file mode 100644 index 00000000..45539e89 --- /dev/null +++ b/reference/config/info.qmd @@ -0,0 +1,69 @@ +--- +title: "Info" +search: true +order: 30 +--- + +Meta information fields filled in by Viash during build. + +## config + +**Type**: `String` + +Path to the config used during build. + +## executable + +**Type**: `String` + +**Default**: `Empty` + +Output folder with main executable path. + +## git_commit + +**Type**: `String` + +**Default**: `Empty` + +Git commit hash. + +## git_remote + +**Type**: `String` + +**Default**: `Empty` + +Git remote name. + +## git_tag + +**Type**: `String` + +**Default**: `Empty` + +Git tag. + +## output + +**Type**: `String` + +**Default**: `Empty` + +Folder path to the build artifacts. + +## platform + +**Type**: `String` + +**Default**: `Empty` + +The platform id used during build. + +## viash_version + +**Type**: `String` + +**Default**: `Empty` + +The Viash version that was used to build the component. diff --git a/reference/config/platforms/_index.yaml b/reference/config/platforms/_index.yaml deleted file mode 100644 index 110dc29a..00000000 --- a/reference/config/platforms/_index.yaml +++ /dev/null @@ -1,14 +0,0 @@ -data: -- description: "A list of platforms to generate target artifacts for.\n\n * [Native](/reference/config/platforms/native/#)\n\ - \ * [Docker](/reference/config/platforms/docker/#)\n * [Nextflow VDSL3](/reference/config/platforms/nextflow/#)\n" - example: - - example: "platforms:\n - type: docker\n image: \"bash:4.0\"\n - type: native\n\ - \ - type: nextflow\n directives:\n label: [lowcpu, midmem]\n" - format: yaml - hierarchy: - - io.viash.platforms.Platform - name: __this__ - type: Platform -order: 20 -title: Platform -topic: platforms diff --git a/reference/config/platforms/docker/_index.yaml b/reference/config/platforms/docker/_index.yaml deleted file mode 100644 index e3af1a21..00000000 --- a/reference/config/platforms/docker/_index.yaml +++ /dev/null @@ -1,306 +0,0 @@ -data: -- description: 'Run a Viash component on a Docker backend platform. - - By specifying which dependencies your component needs, users will be able to build - a docker container from scratch using the setup flag, or pull it from a docker - repository. - - ' - example: - - example: "platforms:\n - type: docker\n image: \"bash:4.0\"\n setup:\n\ - \ - type: apt\n packages: [ curl ]\n" - format: yaml - hierarchy: - - io.viash.platforms.DockerPlatform - - io.viash.platforms.Platform - name: __this__ - type: DockerPlatform -- description: Name of a container's [organization](https://docs.docker.com/docker-hub/orgs/). - name: organization - type: Option of String -- description: The URL to the a [custom Docker registry](https://docs.docker.com/registry/) - example: - - example: 'registry: https://my-docker-registry.org' - format: yaml - name: registry - type: Option of String -- description: The base container to start from. You can also add the tag here if - you wish. - example: - - example: 'image: "bash:4.0"' - format: yaml - name: image - type: String -- description: Specify a Docker image based on its tag. - example: - - example: 'tag: 4.0' - format: yaml - name: tag - type: Option of String -- description: The tag the resulting image gets. Advanced usage only. - example: - - example: 'target_tag: 0.5.0' - format: yaml - name: target_tag - type: Option of String -- description: Add [docker run](https://docs.docker.com/engine/reference/run/) arguments. - name: run_args - type: OneOrMore of String -- description: 'The separator between the namespace and the name of the component, - used for determining the image name. Default: `"/"`.' - example: - - example: 'namespace_separator: "_"' - format: yaml - name: namespace_separator - type: String -- description: 'Enables or disables automatic volume mapping. Enabled when set to - `Automatic` or disabled when set to `Manual`. Default: `Automatic`.' - name: resolve_volume - type: DockerResolveVolume -- description: A list of enabled ports. This doesn't change the Dockerfile but gets - added as a command-line argument at runtime. - example: - - example: "port:\n - 80\n - 8080\n" - format: yaml - name: port - type: OneOrMore of String -- description: Specify which Python packages should be available in order to run the - component. - example: - - example: "setup:\n - type: python\n pip: [ numpy ]\n git: [ https://some.git.repository/org/repo\ - \ ]\n github: [ jkbr/httpie ]\n gitlab: [ foo/bar ]\n mercurial: [\ - \ http://... ]\n svn: [ http://...]\n bazaar: [ http://... ]\n url:\ - \ [ http://... ]\n" - format: yaml - name: python - removed: - deprecation: 0.5.15 - message: 'Use `setup` instead, e.g. `{type: docker, setup: [{ type: python, ... - }]}`. Will be removed.' - removal: 0.7.0 - type: Option of PythonRequirements -- description: "A list of requirements for installing the following types of packages:\n\ - \n - [apt](/reference/config/platforms/docker/setup/aptRequirements.html)\n -\ - \ [apk](/reference/config/platforms/docker/setup/apkRequirements.html)\n - [Docker\ - \ setup instructions](/reference/config/platforms/docker/setup/dockerRequirements.html)\n\ - \ - [JavaScript](/reference/config/platforms/docker/setup/javascriptRequirements.html)\n\ - \ - [Python](/reference/config/platforms/docker/setup/pythonRequirements.html)\n\ - \ - [R](/reference/config/platforms/docker/setup/rRequirements.html)\n - [Ruby](/reference/config/platforms/docker/setup/rubyRequirements.html)\n\ - \ - [yum](/reference/config/platforms/docker/setup/yumRequirements.html)\n\nThe\ - \ order in which these dependencies are specified determines the order in which\ - \ they will be installed.\n" - name: setup - type: List of Requirements -- description: The working directory when starting the container. This doesn't change - the Dockerfile but gets added as a command-line argument at runtime. - example: - - example: 'workdir: /home/user' - format: yaml - name: workdir - type: Option of String -- description: Specify which apk packages should be available in order to run the - component. - example: - - example: "setup:\n - type: apk\n packages: [ sl ]\n" - format: yaml - name: apk - removed: - deprecation: 0.5.15 - message: 'Use `setup` instead, e.g. `{type: docker, setup: [{ type: apk, ... }]}`. - Will be removed.' - removal: 0.7.0 - type: Option of ApkRequirements -- description: If anything is specified in the setup section, running the `---setup` - will result in an image with the name of `:`. If nothing - is specified in the `setup` section, simply `image` will be used. Advanced usage - only. - example: - - example: 'target_image: myfoo' - format: yaml - name: target_image - type: Option of String -- description: Set the default command being executed when running the Docker container. - example: - - description: Set CMD using the exec format, which is the prefered form. - example: 'cmd: ["echo", "$HOME"]' - format: yaml - - description: Set CMD using the shell format. - example: 'cmd: "echo $HOME"' - format: yaml - name: cmd - since: Viash 0.7.4 - type: Option[Either[String,List of String]] -- description: Specify which yum packages should be available in order to run the - component. - example: - - example: "setup:\n - type: yum\n packages: [ sl ]\n" - format: yaml - name: yum - removed: - deprecation: 0.5.15 - message: 'Use `setup` instead, e.g. `{type: docker, setup: [{ type: yum, ... }]}`. - Will be removed.' - removal: 0.7.0 - type: Option of YumRequirements -- description: The source of the target image. This is used for defining labels in - the dockerfile. - example: - - example: 'target_image_source: https://github.com/foo/bar' - format: yaml - name: target_image_source - type: Option of String -- description: Additional requirements specific for running unit tests. - name: test_setup - since: Viash 0.5.13 - type: List of Requirements -- description: Override the entrypoint of the base container. Default set `ENTRYPOINT - []`. - example: - - description: Disable the default override. - example: 'entrypoint: ' - format: yaml - - description: Entrypoint of the container in the exec format, which is the prefered - form. - example: 'entrypoint: ["top", "-b"]' - format: yaml - - description: Entrypoint of the container in the shell format. - example: 'entrypoint: "top -b"' - format: yaml - name: entrypoint - since: Viash 0.7.4 - type: Option[Either[String,List of String]] -- description: Specify which Docker commands should be run during setup. - example: - - example: "setup:\n - type: docker\n build_args: [ GITHUB_PAT=hello_world ]\n\ - \ run: [ git clone ... ]\n add: [ \"http://foo.bar .\" ]\n copy: [\ - \ \"http://foo.bar .\" ]\n resources: \n - resource.txt /path/to/resource.txt\n" - format: yaml - name: docker - removed: - deprecation: 0.5.15 - message: 'Use `setup` instead, e.g. `{type: docker, setup: [{ type: docker, ... - }]}`. Will be removed.' - removal: 0.7.0 - type: Option of DockerRequirements -- description: 'As with all platforms, you can give a platform a different name. By - specifying `id: foo`, you can target this platform (only) by specifying `-p foo` - in any of the Viash commands.' - example: - - example: 'id: foo' - format: yaml - name: id - type: String -- description: Specify which apt packages should be available in order to run the - component. - example: - - example: "setup:\n - type: apt\n packages: [ sl ]\n" - format: yaml - name: apt - removed: - deprecation: 0.5.15 - message: 'Use `setup` instead, e.g. `{type: docker, setup: [{ type: apt, ... }]}`. - Will be removed.' - removal: 0.7.0 - type: Option of AptRequirements -- description: The URL where the resulting image will be pushed to. Advanced usage - only. - example: - - example: 'target_registry: https://my-docker-registry.org' - format: yaml - name: target_registry - type: Option of String -- description: Adds a `privileged` flag to the docker run. - name: privileged - removed: - deprecation: 0.6.3 - message: 'Add a `privileged` flag in `run_args` instead, e.g. `{type: docker, - run_args: "--privileged"}`.' - removal: 0.7.0 - type: Option of Boolean -- description: 'The Docker setup strategy to use when building a container. - - - | Strategy | Description | - - |-----|----------| - - | `alwaysbuild` / `build` / `b` | Always build the image from the dockerfile. - This is the default setup strategy. - - | `alwayscachedbuild` / `cachedbuild` / `cb` | Always build the image from the - dockerfile, with caching enabled. - - | `ifneedbebuild` | Build the image if it does not exist locally. - - | `ifneedbecachedbuild` | Build the image with caching enabled if it does not - exist locally, with caching enabled. - - | `alwayspull` / `pull` / `p` | Try to pull the container from [Docker Hub](https://hub.docker.com) - or the [specified docker registry](/reference/config/platforms/docker/#registry). - - | `alwayspullelsebuild` / `pullelsebuild` | Try to pull the image from a registry - and build it if it doesn''t exist. - - | `alwayspullelsecachedbuild` / `pullelsecachedbuild` | Try to pull the image - from a registry and build it with caching if it doesn''t exist. - - | `ifneedbepull` | If the image does not exist locally, pull the image. - - | `ifneedbepullelsebuild` | If the image does not exist locally, pull the image. - If the image does exist, build it. - - | `ifneedbepullelsecachedbuild` | If the image does not exist locally, pull the - image. If the image does exist, build it with caching enabled. - - | `push` | Push the container to [Docker Hub](https://hub.docker.com) or the - [specified docker registry](/reference/config/platforms/docker/#registry). - - | `pushifnotpresent` | Push the container to [Docker Hub](https://hub.docker.com) - or the [specified docker registry](/reference/config/platforms/docker/#registry) - if the [tag](/reference/config/platforms/docker/#tag) does not exist yet. - - | `donothing` / `meh` | Do not build or pull anything. - - - ' - example: - - example: 'setup_strategy: alwaysbuild' - format: yaml - name: setup_strategy - type: DockerSetupStrategy -- description: Specify which R packages should be available in order to run the component. - example: - - example: "setup: \n - type: r\n cran: [ dynutils ]\n bioc: [ AnnotationDbi\ - \ ]\n git: [ https://some.git.repository/org/repo ]\n github: [ rcannood/SCORPIUS\ - \ ]\n gitlab: [ org/package ]\n svn: [ https://path.to.svn/group/repo\ - \ ]\n url: [ https://github.com/hadley/stringr/archive/HEAD.zip ]\n script:\ - \ [ 'devtools::install(\".\")' ]\n" - format: yaml - name: r - removed: - deprecation: 0.5.15 - message: 'Use `setup` instead, e.g. `{type: docker, setup: [{ type: r, ... }]}`. - Will be removed.' - removal: 0.7.0 - type: Option of RRequirements -- description: Specifies the type of the platform. - name: type - type: String -- description: The organization set in the resulting image. Advanced usage only. - example: - - example: 'target_organization: viash-io' - format: yaml - name: target_organization - type: Option of String -- description: 'In Linux, files created by a Docker container will be owned by `root`. - With `chown: true`, Viash will automatically change the ownership of output files - (arguments with `type: file` and `direction: output`) to the user running the - Viash command after execution of the component. Default value: `true`.' - example: - - example: 'chown: false' - format: yaml - name: chown - type: Boolean -order: 20 -title: Docker Platform -topic: platforms diff --git a/reference/config/platforms/docker/index.qmd b/reference/config/platforms/docker/index.qmd index 576de7a5..438d224c 100644 --- a/reference/config/platforms/docker/index.qmd +++ b/reference/config/platforms/docker/index.qmd @@ -20,48 +20,12 @@ platforms: ``` -## apk - -**Type**: `ApkRequirements` - -::: {.callout-warning} -Deprecated since 0.5.15. -Removed since 0.7.0. Use `setup` instead, e.g. `{type: docker, setup: [{ type: apk, ... }]}`. Will be removed. -::: -Specify which apk packages should be available in order to run the component. - -**Example:** - -```yaml -setup: - - type: apk - packages: [ sl ] - -``` - -## apt - -**Type**: `AptRequirements` - -::: {.callout-warning} -Deprecated since 0.5.15. -Removed since 0.7.0. Use `setup` instead, e.g. `{type: docker, setup: [{ type: apt, ... }]}`. Will be removed. -::: -Specify which apt packages should be available in order to run the component. - -**Example:** - -```yaml -setup: - - type: apt - packages: [ sl ] - -``` - ## chown **Type**: `Boolean` +**Default**: `True` + In Linux, files created by a Docker container will be owned by `root`. With `chown: true`, Viash will automatically change the ownership of output files (arguments with `type: file` and `direction: output`) to the user running the Viash command after execution of the component. Default value: `true`. **Example:** @@ -72,7 +36,11 @@ chown: false ## cmd -**Type**: `Option[Either[String,List of String]]` +**Type**: `Either + - String + - List of String` + +**Default**: `Empty` Set the default command being executed when running the Docker container. @@ -90,33 +58,13 @@ Set CMD using the shell format. cmd: "echo $HOME" ``` -## docker - -**Type**: `DockerRequirements` - -::: {.callout-warning} -Deprecated since 0.5.15. -Removed since 0.7.0. Use `setup` instead, e.g. `{type: docker, setup: [{ type: docker, ... }]}`. Will be removed. -::: -Specify which Docker commands should be run during setup. - -**Example:** - -```yaml -setup: - - type: docker - build_args: [ GITHUB_PAT=hello_world ] - run: [ git clone ... ] - add: [ "http://foo.bar ." ] - copy: [ "http://foo.bar ." ] - resources: - - resource.txt /path/to/resource.txt - -``` - ## entrypoint -**Type**: `Option[Either[String,List of String]]` +**Type**: `Either + - String + - List of String` + +**Default**: `[]` Override the entrypoint of the base container. Default set `ENTRYPOINT []`. @@ -144,6 +92,8 @@ entrypoint: "top -b" **Type**: `String` +**Default**: `docker` + As with all platforms, you can give a platform a different name. By specifying `id: foo`, you can target this platform (only) by specifying `-p foo` in any of the Viash commands. **Example:** @@ -168,6 +118,8 @@ image: "bash:4.0" **Type**: `String` +**Default**: `/` + The separator between the namespace and the name of the component, used for determining the image name. Default: `"/"`. **Example:** @@ -180,12 +132,16 @@ namespace_separator: "_" **Type**: `String` +**Default**: `Empty` + Name of a container's [organization](https://docs.docker.com/docker-hub/orgs/). ## port **Type**: `String` / `List of String` +**Default**: `Empty` + A list of enabled ports. This doesn't change the Dockerfile but gets added as a command-line argument at runtime. **Example:** @@ -197,72 +153,12 @@ port: ``` -## privileged - -**Type**: `Boolean` - -::: {.callout-warning} -Deprecated since 0.6.3. -Removed since 0.7.0. Add a `privileged` flag in `run_args` instead, e.g. `{type: docker, run_args: "--privileged"}`. -::: -Adds a `privileged` flag to the docker run. - -## python - -**Type**: `PythonRequirements` - -::: {.callout-warning} -Deprecated since 0.5.15. -Removed since 0.7.0. Use `setup` instead, e.g. `{type: docker, setup: [{ type: python, ... }]}`. Will be removed. -::: -Specify which Python packages should be available in order to run the component. - -**Example:** - -```yaml -setup: - - type: python - pip: [ numpy ] - git: [ https://some.git.repository/org/repo ] - github: [ jkbr/httpie ] - gitlab: [ foo/bar ] - mercurial: [ http://... ] - svn: [ http://...] - bazaar: [ http://... ] - url: [ http://... ] - -``` - -## r - -**Type**: `RRequirements` - -::: {.callout-warning} -Deprecated since 0.5.15. -Removed since 0.7.0. Use `setup` instead, e.g. `{type: docker, setup: [{ type: r, ... }]}`. Will be removed. -::: -Specify which R packages should be available in order to run the component. - -**Example:** - -```yaml -setup: - - type: r - cran: [ dynutils ] - bioc: [ AnnotationDbi ] - git: [ https://some.git.repository/org/repo ] - github: [ rcannood/SCORPIUS ] - gitlab: [ org/package ] - svn: [ https://path.to.svn/group/repo ] - url: [ https://github.com/hadley/stringr/archive/HEAD.zip ] - script: [ 'devtools::install(".")' ] - -``` - ## registry **Type**: `String` +**Default**: `Empty` + The URL to the a [custom Docker registry](https://docs.docker.com/registry/) **Example:** @@ -275,18 +171,24 @@ registry: https://my-docker-registry.org **Type**: `DockerResolveVolume` +**Default**: `Automatic` + Enables or disables automatic volume mapping. Enabled when set to `Automatic` or disabled when set to `Manual`. Default: `Automatic`. ## run_args **Type**: `String` / `List of String` +**Default**: `Empty` + Add [docker run](https://docs.docker.com/engine/reference/run/) arguments. ## setup **Type**: `List of Requirements` +**Default**: `Empty` + A list of requirements for installing the following types of packages: - [apt](/reference/config/platforms/docker/setup/aptRequirements.html) @@ -305,6 +207,8 @@ The order in which these dependencies are specified determines the order in whic **Type**: `DockerSetupStrategy` +**Default**: `ifneedbepullelsecachedbuild` + The Docker setup strategy to use when building a container. | Strategy | Description | @@ -335,6 +239,8 @@ setup_strategy: alwaysbuild **Type**: `String` +**Default**: `Empty` + Specify a Docker image based on its tag. **Example:** @@ -347,6 +253,8 @@ tag: 4.0 **Type**: `String` +**Default**: `Empty` + If anything is specified in the setup section, running the `---setup` will result in an image with the name of `:`. If nothing is specified in the `setup` section, simply `image` will be used. Advanced usage only. **Example:** @@ -359,6 +267,8 @@ target_image: myfoo **Type**: `String` +**Default**: `Empty` + The source of the target image. This is used for defining labels in the dockerfile. **Example:** @@ -371,6 +281,8 @@ target_image_source: https://github.com/foo/bar **Type**: `String` +**Default**: `Empty` + The organization set in the resulting image. Advanced usage only. **Example:** @@ -383,6 +295,8 @@ target_organization: viash-io **Type**: `String` +**Default**: `Empty` + The URL where the resulting image will be pushed to. Advanced usage only. **Example:** @@ -395,6 +309,8 @@ target_registry: https://my-docker-registry.org **Type**: `String` +**Default**: `Empty` + The tag the resulting image gets. Advanced usage only. **Example:** @@ -407,6 +323,8 @@ target_tag: 0.5.0 **Type**: `List of Requirements` +**Default**: `Empty` + Additional requirements specific for running unit tests. ## type @@ -419,6 +337,8 @@ Specifies the type of the platform. **Type**: `String` +**Default**: `Empty` + The working directory when starting the container. This doesn't change the Dockerfile but gets added as a command-line argument at runtime. **Example:** @@ -426,23 +346,3 @@ The working directory when starting the container. This doesn't change the Docke ```yaml workdir: /home/user ``` - -## yum - -**Type**: `YumRequirements` - -::: {.callout-warning} -Deprecated since 0.5.15. -Removed since 0.7.0. Use `setup` instead, e.g. `{type: docker, setup: [{ type: yum, ... }]}`. Will be removed. -::: -Specify which yum packages should be available in order to run the component. - -**Example:** - -```yaml -setup: - - type: yum - packages: [ sl ] - -``` - diff --git a/reference/config/platforms/docker/setup/_apkRequirements.yaml b/reference/config/platforms/docker/setup/_apkRequirements.yaml deleted file mode 100644 index 020a4c19..00000000 --- a/reference/config/platforms/docker/setup/_apkRequirements.yaml +++ /dev/null @@ -1,22 +0,0 @@ -data: -- description: Specify which apk packages should be available in order to run the - component. - example: - - example: "setup:\n - type: apk\n packages: [ sl ]\n" - format: yaml - hierarchy: - - io.viash.platforms.requirements.ApkRequirements - - io.viash.platforms.requirements.Requirements - name: __this__ - type: ApkRequirements -- description: Specifies the type of the requirement specification. - name: type - type: String -- description: Specifies which packages to install. - example: - - example: 'packages: [ sl ]' - format: yaml - name: packages - type: OneOrMore of String -title: Apk Requirements -topic: requirements diff --git a/reference/config/platforms/docker/setup/_aptRequirements.yaml b/reference/config/platforms/docker/setup/_aptRequirements.yaml deleted file mode 100644 index 533c2783..00000000 --- a/reference/config/platforms/docker/setup/_aptRequirements.yaml +++ /dev/null @@ -1,26 +0,0 @@ -data: -- description: Specify which apt packages should be available in order to run the - component. - example: - - example: "setup:\n - type: apt\n packages: [ sl ]\n" - format: yaml - hierarchy: - - io.viash.platforms.requirements.AptRequirements - - io.viash.platforms.requirements.Requirements - name: __this__ - type: AptRequirements -- description: 'If `false`, the Debian frontend is set to non-interactive (recommended). - Default: false.' - name: interactive - type: Boolean -- description: Specifies the type of the requirement specification. - name: type - type: String -- description: Specifies which packages to install. - example: - - example: 'packages: [ sl ]' - format: yaml - name: packages - type: OneOrMore of String -title: Apt Requirements -topic: requirements diff --git a/reference/config/platforms/docker/setup/_dockerRequirements.yaml b/reference/config/platforms/docker/setup/_dockerRequirements.yaml deleted file mode 100644 index bf08aabc..00000000 --- a/reference/config/platforms/docker/setup/_dockerRequirements.yaml +++ /dev/null @@ -1,70 +0,0 @@ -data: -- description: Specify which Docker commands should be run during setup. - example: - - example: "setup:\n - type: docker\n build_args: \"R_VERSION=hello_world\"\n\ - \ run: |\n echo 'Run a custom command'\n echo 'Foo' > /path/to/file.txt" - format: yaml - hierarchy: - - io.viash.platforms.requirements.DockerRequirements - - io.viash.platforms.requirements.Requirements - name: __this__ - type: DockerRequirements -- description: Specifies which `RUN` entries to add to the Dockerfile while building - it. - example: - - example: "run: |\n echo 'Run a custom command'\n echo 'Foo' > /path/to/file.txt" - format: yaml - name: run - type: OneOrMore of String -- description: Specifies which `LABEL` entries to add to the Dockerfile while building - it. - example: - - example: 'label: [ component="foo" ]' - format: yaml - name: label - type: OneOrMore of String -- description: Specifies which `ARG` entries to add to the Dockerfile while building - it. - example: - - example: 'build_args: [ "R_VERSION=4.2" ]' - format: yaml - name: build_args - type: OneOrMore of String -- description: Specifies the type of the requirement specification. - name: type - type: String -- description: Specifies which `ADD` entries to add to the Dockerfile while building - it. - example: - - example: 'add: [ "http://foo/bar ." ]' - format: yaml - name: add - type: OneOrMore of String -- description: Specifies which `ENV` entries to add to the Dockerfile while building - it. Unlike `ARG`, `ENV` entries are also accessible from inside the container. - example: - - example: 'env: [ "R_VERSION=4.2" ]' - format: yaml - name: env - type: OneOrMore of String -- description: Specifies which `COPY` entries to add to the Dockerfile while building - it. - example: - - example: 'resources: [ "resource.txt /path/to/resource.txt" ]' - format: yaml - name: resources - removed: - deprecation: 0.6.3 - message: '`resources` in `setup: {type: docker, resources: ...}` was removed. - Please use `copy` instead.' - removal: 0.7.0 - type: OneOrMore of String -- description: Specifies which `COPY` entries to add to the Dockerfile while building - it. - example: - - example: 'copy: [ "resource.txt /path/to/resource.txt" ]' - format: yaml - name: copy - type: OneOrMore of String -title: Docker Requirements -topic: requirements diff --git a/reference/config/platforms/docker/setup/_index.yaml b/reference/config/platforms/docker/setup/_index.yaml deleted file mode 100644 index 490b7d5e..00000000 --- a/reference/config/platforms/docker/setup/_index.yaml +++ /dev/null @@ -1,14 +0,0 @@ -data: -- description: "Requirements for installing the following types of packages:\n\n -\ - \ [apt](/reference/config/platforms/docker/setup/aptRequirements.html)\n - [apk](/reference/config/platforms/docker/setup/apkRequirements.html)\n\ - \ - [Docker setup instructions](/reference/config/platforms/docker/setup/dockerRequirements.html)\n\ - \ - [JavaScript](/reference/config/platforms/docker/setup/javascriptRequirements.html)\n\ - \ - [Python](/reference/config/platforms/docker/setup/pythonRequirements.html)\n\ - \ - [R](/reference/config/platforms/docker/setup/rRequirements.html)\n - [Ruby](/reference/config/platforms/docker/setup/rubyRequirements.html)\n\ - \ - [yum](/reference/config/platforms/docker/setup/yumRequirements.html)\n" - hierarchy: - - io.viash.platforms.requirements.Requirements - name: __this__ - type: Requirements -title: Requirements -topic: requirements diff --git a/reference/config/platforms/docker/setup/_javascriptRequirements.yaml b/reference/config/platforms/docker/setup/_javascriptRequirements.yaml deleted file mode 100644 index 9e6d75ad..00000000 --- a/reference/config/platforms/docker/setup/_javascriptRequirements.yaml +++ /dev/null @@ -1,48 +0,0 @@ -data: -- description: Specify which JavaScript packages should be available in order to run - the component. - example: - - example: "setup:\n - type: javascript\n npm: packagename\n git: \"https://some.git.repository/org/repo\"\ - \n github: \"owner/repository\"\n url: \"https://github.com/org/repo/archive/HEAD.zip\"\ - \n" - format: yaml - hierarchy: - - io.viash.platforms.requirements.JavaScriptRequirements - - io.viash.platforms.requirements.Requirements - name: __this__ - type: JavaScriptRequirements -- description: Specifies which packages to install from GitHub. - example: - - example: 'github: [ owner/repository ]' - format: yaml - name: github - type: OneOrMore of String -- description: Specifies which packages to install using a generic URI. - example: - - example: 'url: [ https://github.com/org/repo/archive/HEAD.zip ]' - format: yaml - name: url - type: OneOrMore of String -- description: Specifies which packages to install using a Git URI. - example: - - example: 'git: [ https://some.git.repository/org/repo ]' - format: yaml - name: git - type: OneOrMore of String -- description: Specifies which packages to install from npm. - example: - - example: 'npm: [ packagename ]' - format: yaml - name: npm - type: OneOrMore of String -- description: Specifies the type of the requirement specification. - name: type - type: String -- description: Specifies which packages to install from npm. - example: - - example: 'packages: [ packagename ]' - format: yaml - name: packages - type: OneOrMore of String -title: Javascript Requirements -topic: requirements diff --git a/reference/config/platforms/docker/setup/_pythonRequirements.yaml b/reference/config/platforms/docker/setup/_pythonRequirements.yaml deleted file mode 100644 index cb83afbc..00000000 --- a/reference/config/platforms/docker/setup/_pythonRequirements.yaml +++ /dev/null @@ -1,91 +0,0 @@ -data: -- description: Specify which Python packages should be available in order to run the - component. - example: - - example: "setup:\n - type: python\n pip: numpy\n github: [ jkbr/httpie,\ - \ foo/bar ]\n url: \"https://github.com/some_org/some_pkg/zipball/master\"\ - \n" - format: yaml - hierarchy: - - io.viash.platforms.requirements.PythonRequirements - - io.viash.platforms.requirements.Requirements - name: __this__ - type: PythonRequirements -- description: Specifies which packages to install from GitHub. - example: - - example: 'github: [ jkbr/httpie ]' - format: yaml - name: github - type: OneOrMore of String -- description: Specifies which packages to install from GitLab. - example: - - example: 'gitlab: [ foo/bar ]' - format: yaml - name: gitlab - type: OneOrMore of String -- description: Specifies which packages to install from pip. - example: - - example: 'pip: [ numpy ]' - format: yaml - name: pip - type: OneOrMore of String -- description: Specifies which packages to install from PyPI using pip. - example: - - example: 'pypi: [ numpy ]' - format: yaml - name: pypi - type: OneOrMore of String -- description: Specifies which packages to install using a Git URI. - example: - - example: 'git: [ https://some.git.repository/org/repo ]' - format: yaml - name: git - type: OneOrMore of String -- description: 'Sets the `--upgrade` flag when set to true. Default: true.' - name: upgrade - type: Boolean -- description: Specifies which packages to install from pip. - example: - - example: 'packages: [ numpy ]' - format: yaml - name: packages - type: OneOrMore of String -- description: Specifies which packages to install using a generic URI. - example: - - example: 'url: [ https://github.com/some_org/some_pkg/zipball/master ]' - format: yaml - name: url - type: OneOrMore of String -- description: Specifies which packages to install using an SVN URI. - example: - - example: 'svn: [ http://svn.repo/some_pkg/trunk/#egg=SomePackage ]' - format: yaml - name: svn - type: OneOrMore of String -- description: Specifies which packages to install using a Bazaar URI. - example: - - example: 'bazaar: [ http://bazaar.launchpad.net/some_pkg/some_pkg/release-0.1 - ]' - format: yaml - name: bazaar - type: OneOrMore of String -- description: Specifies a code block to run as part of the build. - example: - - example: "script: |\n print(\"Running custom code\")\n x = 1 + 1 == 2" - format: yaml - name: script - type: OneOrMore of String -- description: Specifies the type of the requirement specification. - name: type - type: String -- description: Specifies which packages to install using a Mercurial URI. - example: - - example: 'mercurial: [ https://hg.myproject.org/MyProject/#egg=MyProject ]' - format: yaml - name: mercurial - type: OneOrMore of String -- description: 'Sets the `--user` flag when set to true. Default: false.' - name: user - type: Boolean -title: Python Requirements -topic: requirements diff --git a/reference/config/platforms/docker/setup/_rRequirements.yaml b/reference/config/platforms/docker/setup/_rRequirements.yaml deleted file mode 100644 index 10287167..00000000 --- a/reference/config/platforms/docker/setup/_rRequirements.yaml +++ /dev/null @@ -1,84 +0,0 @@ -data: -- description: Specify which R packages should be available in order to run the component. - example: - - example: "setup: \n - type: r\n cran: anndata\n bioc: [ AnnotationDbi,\ - \ SingleCellExperiment ]\n github: rcannood/SCORPIUS\n" - format: yaml - hierarchy: - - io.viash.platforms.requirements.RRequirements - - io.viash.platforms.requirements.Requirements - name: __this__ - type: RRequirements -- description: Specifies which packages to install from BioConductor. - example: - - example: 'bioc: [ AnnotationDbi ]' - format: yaml - name: bioc - type: OneOrMore of String -- description: Specifies which packages to install from GitHub. - example: - - example: 'github: [ rcannood/SCORPIUS ]' - format: yaml - name: github - type: OneOrMore of String -- description: Specifies which packages to install from GitLab. - example: - - example: 'gitlab: [ org/package ]' - format: yaml - name: gitlab - type: OneOrMore of String -- description: Specifies which packages to install using a generic URI. - example: - - example: 'url: [ https://github.com/hadley/stringr/archive/HEAD.zip ]' - format: yaml - name: url - type: OneOrMore of String -- description: 'Forces packages specified in `bioc` to be reinstalled, even if they - are already present in the container. Default: false.' - example: - - example: 'bioc_force_install: false' - format: yaml - name: bioc_force_install - type: Boolean -- description: Specifies which packages to install using a Git URI. - example: - - example: 'git: [ https://some.git.repository/org/repo ]' - format: yaml - name: git - type: OneOrMore of String -- description: Specifies which packages to install from CRAN. - example: - - example: 'cran: [ anndata, ggplot2 ]' - format: yaml - name: cran - type: OneOrMore of String -- description: Specifies which packages to install from Bitbucket. - example: - - example: 'bitbucket: [ org/package ]' - format: yaml - name: bitbucket - type: OneOrMore of String -- description: Specifies which packages to install using an SVN URI. - example: - - example: 'svn: [ https://path.to.svn/group/repo ]' - format: yaml - name: svn - type: OneOrMore of String -- description: Specifies which packages to install from CRAN. - example: - - example: 'packages: [ anndata, ggplot2 ]' - format: yaml - name: packages - type: OneOrMore of String -- description: Specifies a code block to run as part of the build. - example: - - example: "script: |\n cat(\"Running custom code\n\")\n install.packages(\"anndata\"\ - )" - format: yaml - name: script - type: OneOrMore of String -- description: Specifies the type of the requirement specification. - name: type - type: String -title: R Requirements -topic: requirements diff --git a/reference/config/platforms/docker/setup/_rubyRequirements.yaml b/reference/config/platforms/docker/setup/_rubyRequirements.yaml deleted file mode 100644 index b216593b..00000000 --- a/reference/config/platforms/docker/setup/_rubyRequirements.yaml +++ /dev/null @@ -1,22 +0,0 @@ -data: -- description: Specify which Ruby packages should be available in order to run the - component. - example: - - example: "setup:\n - type: ruby\n packages: [ rspec ]\n" - format: yaml - hierarchy: - - io.viash.platforms.requirements.RubyRequirements - - io.viash.platforms.requirements.Requirements - name: __this__ - type: RubyRequirements -- description: Specifies the type of the requirement specification. - name: type - type: String -- description: Specifies which packages to install. - example: - - example: 'packages: [ rspec ]' - format: yaml - name: packages - type: OneOrMore of String -title: Ruby Requirements -topic: requirements diff --git a/reference/config/platforms/docker/setup/_yumRequirements.yaml b/reference/config/platforms/docker/setup/_yumRequirements.yaml deleted file mode 100644 index 7d6e9731..00000000 --- a/reference/config/platforms/docker/setup/_yumRequirements.yaml +++ /dev/null @@ -1,22 +0,0 @@ -data: -- description: Specify which yum packages should be available in order to run the - component. - example: - - example: "setup:\n - type: yum\n packages: [ sl ]\n" - format: yaml - hierarchy: - - io.viash.platforms.requirements.YumRequirements - - io.viash.platforms.requirements.Requirements - name: __this__ - type: YumRequirements -- description: Specifies the type of the requirement specification. - name: type - type: String -- description: Specifies which packages to install. - example: - - example: 'packages: [ sl ]' - format: yaml - name: packages - type: OneOrMore of String -title: Yum Requirements -topic: requirements diff --git a/reference/config/platforms/docker/setup/apkRequirements.qmd b/reference/config/platforms/docker/setup/apkRequirements.qmd index c6c53ed0..4e2d8539 100644 --- a/reference/config/platforms/docker/setup/apkRequirements.qmd +++ b/reference/config/platforms/docker/setup/apkRequirements.qmd @@ -18,6 +18,8 @@ setup: **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install. **Example:** @@ -31,4 +33,3 @@ packages: [ sl ] **Type**: `String` Specifies the type of the requirement specification. - diff --git a/reference/config/platforms/docker/setup/aptRequirements.qmd b/reference/config/platforms/docker/setup/aptRequirements.qmd index c061725f..dc46b723 100644 --- a/reference/config/platforms/docker/setup/aptRequirements.qmd +++ b/reference/config/platforms/docker/setup/aptRequirements.qmd @@ -18,12 +18,16 @@ setup: **Type**: `Boolean` +**Default**: `False` + If `false`, the Debian frontend is set to non-interactive (recommended). Default: false. ## packages **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install. **Example:** @@ -37,4 +41,3 @@ packages: [ sl ] **Type**: `String` Specifies the type of the requirement specification. - diff --git a/reference/config/platforms/docker/setup/dockerRequirements.qmd b/reference/config/platforms/docker/setup/dockerRequirements.qmd index e6c76f9a..d423605b 100644 --- a/reference/config/platforms/docker/setup/dockerRequirements.qmd +++ b/reference/config/platforms/docker/setup/dockerRequirements.qmd @@ -20,6 +20,8 @@ setup: **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which `ADD` entries to add to the Dockerfile while building it. **Example:** @@ -32,6 +34,8 @@ add: [ "http://foo/bar ." ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which `ARG` entries to add to the Dockerfile while building it. **Example:** @@ -44,6 +48,8 @@ build_args: [ "R_VERSION=4.2" ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which `COPY` entries to add to the Dockerfile while building it. **Example:** @@ -56,6 +62,8 @@ copy: [ "resource.txt /path/to/resource.txt" ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which `ENV` entries to add to the Dockerfile while building it. Unlike `ARG`, `ENV` entries are also accessible from inside the container. **Example:** @@ -68,6 +76,8 @@ env: [ "R_VERSION=4.2" ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which `LABEL` entries to add to the Dockerfile while building it. **Example:** @@ -76,26 +86,12 @@ Specifies which `LABEL` entries to add to the Dockerfile while building it. label: [ component="foo" ] ``` -## resources - -**Type**: `String` / `List of String` - -::: {.callout-warning} -Deprecated since 0.6.3. -Removed since 0.7.0. `resources` in `setup: {type: docker, resources: ...}` was removed. Please use `copy` instead. -::: -Specifies which `COPY` entries to add to the Dockerfile while building it. - -**Example:** - -```yaml -resources: [ "resource.txt /path/to/resource.txt" ] -``` - ## run **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which `RUN` entries to add to the Dockerfile while building it. **Example:** @@ -111,4 +107,3 @@ run: | **Type**: `String` Specifies the type of the requirement specification. - diff --git a/reference/config/platforms/docker/setup/index.qmd b/reference/config/platforms/docker/setup/index.qmd index ab3b5ac0..352b598b 100644 --- a/reference/config/platforms/docker/setup/index.qmd +++ b/reference/config/platforms/docker/setup/index.qmd @@ -14,4 +14,3 @@ Requirements for installing the following types of packages: - [Ruby](/reference/config/platforms/docker/setup/rubyRequirements.html) - [yum](/reference/config/platforms/docker/setup/yumRequirements.html) - diff --git a/reference/config/platforms/docker/setup/javascriptRequirements.qmd b/reference/config/platforms/docker/setup/javascriptRequirements.qmd index a8b2ba47..347dc5e4 100644 --- a/reference/config/platforms/docker/setup/javascriptRequirements.qmd +++ b/reference/config/platforms/docker/setup/javascriptRequirements.qmd @@ -1,5 +1,5 @@ --- -title: "Javascript Requirements" +title: "JavaScript Requirements" search: true --- @@ -21,6 +21,8 @@ setup: **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install using a Git URI. **Example:** @@ -33,6 +35,8 @@ git: [ https://some.git.repository/org/repo ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install from GitHub. **Example:** @@ -45,6 +49,8 @@ github: [ owner/repository ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install from npm. **Example:** @@ -57,6 +63,8 @@ npm: [ packagename ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install from npm. **Example:** @@ -75,6 +83,8 @@ Specifies the type of the requirement specification. **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install using a generic URI. **Example:** @@ -82,4 +92,3 @@ Specifies which packages to install using a generic URI. ```yaml url: [ https://github.com/org/repo/archive/HEAD.zip ] ``` - diff --git a/reference/config/platforms/docker/setup/pythonRequirements.qmd b/reference/config/platforms/docker/setup/pythonRequirements.qmd index c9375de9..8e6dff2e 100644 --- a/reference/config/platforms/docker/setup/pythonRequirements.qmd +++ b/reference/config/platforms/docker/setup/pythonRequirements.qmd @@ -20,6 +20,8 @@ setup: **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install using a Bazaar URI. **Example:** @@ -32,6 +34,8 @@ bazaar: [ http://bazaar.launchpad.net/some_pkg/some_pkg/release-0.1 ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install using a Git URI. **Example:** @@ -44,6 +48,8 @@ git: [ https://some.git.repository/org/repo ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install from GitHub. **Example:** @@ -56,6 +62,8 @@ github: [ jkbr/httpie ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install from GitLab. **Example:** @@ -68,6 +76,8 @@ gitlab: [ foo/bar ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install using a Mercurial URI. **Example:** @@ -80,6 +90,8 @@ mercurial: [ https://hg.myproject.org/MyProject/#egg=MyProject ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install from pip. **Example:** @@ -92,6 +104,8 @@ packages: [ numpy ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install from pip. **Example:** @@ -104,6 +118,8 @@ pip: [ numpy ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install from PyPI using pip. **Example:** @@ -116,6 +132,8 @@ pypi: [ numpy ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies a code block to run as part of the build. **Example:** @@ -130,6 +148,8 @@ script: | **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install using an SVN URI. **Example:** @@ -148,12 +168,16 @@ Specifies the type of the requirement specification. **Type**: `Boolean` +**Default**: `True` + Sets the `--upgrade` flag when set to true. Default: true. ## url **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install using a generic URI. **Example:** @@ -166,5 +190,6 @@ url: [ https://github.com/some_org/some_pkg/zipball/master ] **Type**: `Boolean` -Sets the `--user` flag when set to true. Default: false. +**Default**: `False` +Sets the `--user` flag when set to true. Default: false. diff --git a/reference/config/platforms/docker/setup/rRequirements.qmd b/reference/config/platforms/docker/setup/rRequirements.qmd index f19d64a1..c77e6c2e 100644 --- a/reference/config/platforms/docker/setup/rRequirements.qmd +++ b/reference/config/platforms/docker/setup/rRequirements.qmd @@ -20,6 +20,8 @@ setup: **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install from BioConductor. **Example:** @@ -32,6 +34,8 @@ bioc: [ AnnotationDbi ] **Type**: `Boolean` +**Default**: `False` + Forces packages specified in `bioc` to be reinstalled, even if they are already present in the container. Default: false. **Example:** @@ -44,6 +48,8 @@ bioc_force_install: false **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install from Bitbucket. **Example:** @@ -56,6 +62,8 @@ bitbucket: [ org/package ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install from CRAN. **Example:** @@ -68,6 +76,8 @@ cran: [ anndata, ggplot2 ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install using a Git URI. **Example:** @@ -80,6 +90,8 @@ git: [ https://some.git.repository/org/repo ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install from GitHub. **Example:** @@ -92,6 +104,8 @@ github: [ rcannood/SCORPIUS ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install from GitLab. **Example:** @@ -104,6 +118,8 @@ gitlab: [ org/package ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install from CRAN. **Example:** @@ -116,6 +132,8 @@ packages: [ anndata, ggplot2 ] **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies a code block to run as part of the build. **Example:** @@ -131,6 +149,8 @@ script: | **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install using an SVN URI. **Example:** @@ -149,6 +169,8 @@ Specifies the type of the requirement specification. **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install using a generic URI. **Example:** @@ -156,4 +178,3 @@ Specifies which packages to install using a generic URI. ```yaml url: [ https://github.com/hadley/stringr/archive/HEAD.zip ] ``` - diff --git a/reference/config/platforms/docker/setup/rubyRequirements.qmd b/reference/config/platforms/docker/setup/rubyRequirements.qmd index 5412f9f3..d7890559 100644 --- a/reference/config/platforms/docker/setup/rubyRequirements.qmd +++ b/reference/config/platforms/docker/setup/rubyRequirements.qmd @@ -18,6 +18,8 @@ setup: **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install. **Example:** @@ -31,4 +33,3 @@ packages: [ rspec ] **Type**: `String` Specifies the type of the requirement specification. - diff --git a/reference/config/platforms/docker/setup/yumRequirements.qmd b/reference/config/platforms/docker/setup/yumRequirements.qmd index fb28a47a..f875faa9 100644 --- a/reference/config/platforms/docker/setup/yumRequirements.qmd +++ b/reference/config/platforms/docker/setup/yumRequirements.qmd @@ -18,6 +18,8 @@ setup: **Type**: `String` / `List of String` +**Default**: `Empty` + Specifies which packages to install. **Example:** @@ -31,4 +33,3 @@ packages: [ sl ] **Type**: `String` Specifies the type of the requirement specification. - diff --git a/reference/config/platforms/index.qmd b/reference/config/platforms/index.qmd index 6e7d5c94..edcdaf2b 100644 --- a/reference/config/platforms/index.qmd +++ b/reference/config/platforms/index.qmd @@ -8,7 +8,7 @@ A list of platforms to generate target artifacts for. * [Native](/reference/config/platforms/native/#) * [Docker](/reference/config/platforms/docker/#) - * [Nextflow VDSL3](/reference/config/platforms/nextflow/#) + * [Nextflow](/reference/config/platforms/nextflow/#) **Example:** @@ -23,4 +23,3 @@ platforms: label: [lowcpu, midmem] ``` - diff --git a/reference/config/platforms/native/_index.yaml b/reference/config/platforms/native/_index.yaml deleted file mode 100644 index 444afd55..00000000 --- a/reference/config/platforms/native/_index.yaml +++ /dev/null @@ -1,31 +0,0 @@ -data: -- description: 'Running a Viash component on a native platform means that the script - will be executed in your current environment. - - Any dependencies are assumed to have been installed by the user, so the native - platform is meant for developers (who know what they''re doing) or for simple - bash scripts (which have no extra dependencies). - - ' - example: - - example: "platforms:\n - type: native\n" - format: yaml - hierarchy: - - io.viash.platforms.NativePlatform - - io.viash.platforms.Platform - name: __this__ - type: NativePlatform -- description: 'As with all platforms, you can give a platform a different name. By - specifying `id: foo`, you can target this platform (only) by specifying `-p foo` - in any of the Viash commands.' - example: - - example: 'id: foo' - format: yaml - name: id - type: String -- description: Specifies the type of the platform. - name: type - type: String -order: 10 -title: Native Platform -topic: platforms diff --git a/reference/config/platforms/native/index.qmd b/reference/config/platforms/native/index.qmd index bff8963c..64cf7a91 100644 --- a/reference/config/platforms/native/index.qmd +++ b/reference/config/platforms/native/index.qmd @@ -20,6 +20,8 @@ platforms: **Type**: `String` +**Default**: `native` + As with all platforms, you can give a platform a different name. By specifying `id: foo`, you can target this platform (only) by specifying `-p foo` in any of the Viash commands. **Example:** @@ -33,4 +35,3 @@ id: foo **Type**: `String` Specifies the type of the platform. - diff --git a/reference/config/platforms/nextflow/_auto.yaml b/reference/config/platforms/nextflow/_auto.yaml deleted file mode 100644 index bea45bcb..00000000 --- a/reference/config/platforms/nextflow/_auto.yaml +++ /dev/null @@ -1,51 +0,0 @@ -data: -- description: Automated processing flags which can be toggled on or off. - hierarchy: - - io.viash.platforms.nextflow.NextflowAuto - name: __this__ - type: NextflowAuto -- description: 'If `true`, an input tuple only containing only a single File (e.g. - `["foo", file("in.h5ad")]`) is automatically transformed to a map (i.e. `["foo", - [ input: file("in.h5ad") ] ]`). - - - Default: `true`. - - ' - name: simplifyInput - type: Boolean -- description: 'If `true`, an output tuple containing a map with a File (e.g. `["foo", - [ output: file("out.h5ad") ] ]`) is automatically transformed to a map (i.e. `["foo", - file("out.h5ad")]`). - - - Default: `true`. - - ' - name: simplifyOutput - type: Boolean -- description: 'If `true`, the module''s outputs are automatically published to `params.publishDir`. - - Will throw an error if `params.publishDir` is not defined. - - - Default: `false`. - - ' - name: publish - type: Boolean -- description: 'If `true`, the module''s transcripts from `work/` are automatically - published to `params.transcriptDir`. - - If not defined, `params.publishDir + "/_transcripts"` will be used. - - Will throw an error if neither are defined. - - - Default: `false`. - - ' - name: transcript - type: Boolean -title: Nextflow Auto -topic: nextflowParameters diff --git a/reference/config/platforms/nextflow/_config.yaml b/reference/config/platforms/nextflow/_config.yaml deleted file mode 100644 index a4b69d87..00000000 --- a/reference/config/platforms/nextflow/_config.yaml +++ /dev/null @@ -1,65 +0,0 @@ -data: -- description: Allows tweaking how the Nextflow Config file is generated. - hierarchy: - - io.viash.platforms.nextflow.NextflowConfig - name: __this__ - since: Viash 0.7.4 - type: NextflowConfig -- description: 'A series of default labels to specify memory and cpu constraints. - - - The default memory labels are defined as "mem1gb", "mem2gb", "mem4gb", ... upto - "mem512tb" and follows powers of 2. - - The default cpu labels are defined as "cpu1", "cpu2", "cpu5", "cpu10", ... upto - "cpu1000" and follows a semi logarithmic scale (1, 2, 5 per decade). - - - Conceptually it is possible for a Viash Config to overwrite the full labels parameter, - however likely it is more efficient to add additional labels - - in the Viash Project with a config mod. - - ' - example: - - description: Replace the default labels with a different set of labels - example: "labels:\n lowmem: \"memory = 4.GB\"\n lowcpu: \"cpus = 4\"\n midmem:\ - \ \"memory = 25.GB\"\n midcpu: \"cpus = 10\"\n highmem: \"memory = 50.GB\"\ - \n highcpu: \"cpus = 20\"\n vhighmem: \"memory = 100.GB\"\n vhighcpu: \"\ - cpus = 40\"\n" - format: yaml - - description: Add 'lowmem' and 'lowcpu' to the default labels by using a config - mod - example: -c '.platforms[.type == "nextflow"].config.labels.lowmem := "memory = - 4.GB";.platforms[.type == "nextflow"].config.labels.lowcpu := "cpus = 4"' - format: viash_config_mod - - description: Add 'lowmem' and 'lowcpu' to the default labels by using the Viash - Project file - example: "config_mods: |\n .platforms[.type == \"nextflow\"].config.labels.lowmem\ - \ := \"memory = 4.GB\"\n .platforms[.type == \"nextflow\"].config.labels.lowcpu\ - \ := \"cpus = 4\"\n" - format: viash_project_file - - description: Replace the default labels with a different set of labels by using - the Viash Project file - example: "config_mods: |\n .platforms[.type == \"nextflow\"].config.labels :=\ - \ { lowmem: \"memory = 4.GB\", lowcpu: \"cpus = 4\", midmem: \"memory = 25.GB\"\ - , midcpu: \"cpus = 10\", highmem: \"memory = 50.GB\", highcpu: \"cpus = 20\"\ - , vhighmem: \"memory = 100.GB\", vhighcpu: \"cpus = 40\" }\n" - format: viash_project_file - name: labels - type: ListMap of String,String -- description: 'Includes a single string or list of strings into the nextflow.config - file. - - This can be used to add custom profiles or include an additional config file. - - ' - example: - - example: "script:\n - |\n profiles {\n ...\n }\n" - format: yaml - - example: 'script: includeConfig("config.config")' - format: yaml - name: script - type: OneOrMore of String -title: Nextflow Config -topic: nextflowParameters diff --git a/reference/config/platforms/nextflow/_directives.yaml b/reference/config/platforms/nextflow/_directives.yaml deleted file mode 100644 index 9c407d48..00000000 --- a/reference/config/platforms/nextflow/_directives.yaml +++ /dev/null @@ -1,544 +0,0 @@ -data: -- description: 'Directives are optional settings that affect the execution of the - process. - - ' - example: - - example: "directives:\n container: rocker/r-ver:4.1\n label: highcpu\n \ - \ cpus: 4\n memory: 16 GB" - format: yaml - hierarchy: - - io.viash.platforms.nextflow.NextflowDirectives - name: __this__ - type: NextflowDirectives -- description: 'The `beforeScript` directive allows you to execute a custom (Bash) - snippet before the main process script is run. This may be useful to initialise - the underlying cluster environment or for other custom initialisation. - - - See [`beforeScript`](https://www.nextflow.io/docs/latest/process.html#beforeScript). - - ' - example: - - example: source /cluster/bin/setup - format: yaml - name: beforeScript - type: Option of String -- description: 'Environment Modules is a package manager that allows you to dynamically - configure your execution environment and easily switch between multiple versions - of the same software tool. - - - If it is available in your system you can use it with Nextflow in order to configure - the processes execution environment in your pipeline. - - - In a process definition you can use the `module` directive to load a specific - module version to be used in the process execution environment. - - - See [`module`](https://www.nextflow.io/docs/latest/process.html#module). - - ' - example: - - example: '"ncbi-blast/2.2.27"' - format: yaml - - example: '"ncbi-blast/2.2.27:t_coffee/10.0"' - format: yaml - - example: '["ncbi-blast/2.2.27", "t_coffee/10.0"]' - format: yaml - name: module - type: OneOrMore of String -- description: 'The `queue` directory allows you to set the queue where jobs are scheduled - when using a grid based executor in your pipeline. - - - See [`queue`](https://www.nextflow.io/docs/latest/process.html#queue). - - ' - example: - - example: '"long"' - format: yaml - - example: '"short,long"' - format: yaml - - example: '["short", "long"]' - format: yaml - name: queue - type: OneOrMore of String -- description: 'The `label` directive allows the annotation of processes with mnemonic - identifier of your choice. - - - See [`label`](https://www.nextflow.io/docs/latest/process.html#label). - - ' - example: - - example: '"big_mem"' - format: yaml - - example: '"big_cpu"' - format: yaml - - example: '["big_mem", "big_cpu"]' - format: yaml - name: label - type: OneOrMore of String -- description: 'The `container` directive allows you to execute the process script - in a Docker container. - - - It requires the Docker daemon to be running in machine where the pipeline is executed, - i.e. the local machine when using the local executor or the cluster nodes when - the pipeline is deployed through a grid executor. - - - Viash implements allows either a string value or a map. In case a map is used, - the allowed keys are: `registry`, `image`, and `tag`. The `image` value must be - specified. - - - See [`container`](https://www.nextflow.io/docs/latest/process.html#container). - - ' - example: - - example: '"foo/bar:tag"' - format: yaml - - description: 'This is transformed to `"reg/im:ta"`:' - example: '[ registry: "reg", image: "im", tag: "ta" ]' - format: yaml - - description: 'This is transformed to `"im:latest"`:' - example: '[ image: "im" ]' - format: yaml - name: container - type: Option[Either[Map of String,String,String]] -- description: 'The `publishDir` directive allows you to publish the process output - files to a specified folder. - - - Viash implements this directive as a plain string or a map. The allowed keywords - for the map are: `path`, `mode`, `overwrite`, `pattern`, `saveAs`, `enabled`. - The `path` key and value are required. - - The allowed values for `mode` are: `symlink`, `rellink`, `link`, `copy`, `copyNoFollow`, - `move`. - - - See [`publishDir`](https://www.nextflow.io/docs/latest/process.html#publishdir). - - ' - example: - - example: '[]' - format: yaml - - example: '[ [ path: "foo", enabled: true ], [ path: "bar", enabled: false ] ]' - format: yaml - - description: 'This is transformed to `[[ path: "/path/to/dir" ]]`:' - example: '"/path/to/dir"' - format: yaml - - description: 'This is transformed to `[[ path: "/path/to/dir", mode: "cache" ]]`:' - example: '[ path: "/path/to/dir", mode: "cache" ]' - format: yaml - name: publishDir - type: OneOrMore[Either[String,Map of String,String]] -- description: 'The `maxForks` directive allows you to define the maximum number of - process instances that can be executed in parallel. By default this value is equals - to the number of CPU cores available minus 1. - - - If you want to execute a process in a sequential manner, set this directive to - one. - - - See [`maxForks`](https://www.nextflow.io/docs/latest/process.html#maxforks). - - ' - example: - - example: '1' - format: yaml - - example: '3' - format: yaml - name: maxForks - type: Option[Either of String,Int] -- description: 'The `maxErrors` directive allows you to specify the maximum number - of times a process can fail when using the `retry` error strategy. By default - this directive is disabled. - - - See [`maxErrors`](https://www.nextflow.io/docs/latest/process.html#maxerrors). - - ' - example: - - example: '1' - format: yaml - - example: '3' - format: yaml - name: maxErrors - type: Option[Either of String,Int] -- description: 'The `cpus` directive allows you to define the number of (logical) - CPU required by the process'' task. - - - See [`cpus`](https://www.nextflow.io/docs/latest/process.html#cpus). - - ' - example: - - example: '1' - format: yaml - - example: '10' - format: yaml - name: cpus - type: Option[Either of Int,String] -- description: 'The `accelerator` directive allows you to specify the hardware accelerator - requirement for the task execution e.g. GPU processor. - - - Viash implements this directive as a map with accepted keywords: `type`, `limit`, - `request`, and `runtime`. - - - See [`accelerator`](https://www.nextflow.io/docs/latest/process.html#accelerator). - - ' - example: - - example: '[ limit: 4, type: "nvidia-tesla-k80" ]' - format: yaml - name: accelerator - type: Map of String,String -- description: 'The `time` directive allows you to define how long a process is allowed - to run. - - - See [`time`](https://www.nextflow.io/docs/latest/process.html#time). - - ' - example: - - example: '"1h"' - format: yaml - - example: '"2days"' - format: yaml - - example: '"1day 6hours 3minutes 30seconds"' - format: yaml - name: time - type: Option of String -- description: 'The `afterScript` directive allows you to execute a custom (Bash) - snippet immediately after the main process has run. This may be useful to clean - up your staging area. - - - See [`afterScript`](https://www.nextflow.io/docs/latest/process.html#afterscript). - - ' - example: - - example: source /cluster/bin/cleanup - format: yaml - name: afterScript - type: Option of String -- description: "The `executor` defines the underlying system where processes are executed.\ - \ By default a process uses the executor defined globally in the nextflow.config\ - \ file.\n\nThe `executor` directive allows you to configure what executor has\ - \ to be used by the process, overriding the default configuration. The following\ - \ values can be used:\n\n| Name | Executor |\n|------|----------|\n| awsbatch\ - \ | The process is executed using the AWS Batch service. | \n| azurebatch | The\ - \ process is executed using the Azure Batch service. | \n| condor | The process\ - \ is executed using the HTCondor job scheduler. | \n| google-lifesciences | The\ - \ process is executed using the Google Genomics Pipelines service. | \n| ignite\ - \ | The process is executed using the Apache Ignite cluster. | \n| k8s | The process\ - \ is executed using the Kubernetes cluster. | \n| local | The process is executed\ - \ in the computer where Nextflow is launched. | \n| lsf | The process is executed\ - \ using the Platform LSF job scheduler. | \n| moab | The process is executed using\ - \ the Moab job scheduler. | \n| nqsii | The process is executed using the NQSII\ - \ job scheduler. | \n| oge | Alias for the sge executor. | \n| pbs | The process\ - \ is executed using the PBS/Torque job scheduler. | \n| pbspro | The process is\ - \ executed using the PBS Pro job scheduler. | \n| sge | The process is executed\ - \ using the Sun Grid Engine / Open Grid Engine. | \n| slurm | The process is executed\ - \ using the SLURM job scheduler. | \n| tes | The process is executed using the\ - \ GA4GH TES service. | \n| uge | Alias for the sge executor. |\n\nSee [`executor`](https://www.nextflow.io/docs/latest/process.html#executor).\n" - example: - - example: '"local"' - format: yaml - - example: '"sge"' - format: yaml - name: executor - type: Option of String -- description: 'The `containerOptions` directive allows you to specify any container - execution option supported by the underlying container engine (ie. Docker, Singularity, - etc). This can be useful to provide container settings only for a specific process - e.g. mount a custom path. - - - See [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#containeroptions). - - ' - example: - - example: '"--foo bar"' - format: yaml - - example: '["--foo bar", "-f b"]' - format: yaml - name: containerOptions - type: OneOrMore of String -- description: 'The `disk` directive allows you to define how much local disk storage - the process is allowed to use. - - - See [`disk`](https://www.nextflow.io/docs/latest/process.html#disk). - - ' - example: - - example: '"1 GB"' - format: yaml - - example: '"2TB"' - format: yaml - - example: '"3.2KB"' - format: yaml - - example: '"10.B"' - format: yaml - name: disk - type: Option of String -- description: 'The `tag` directive allows you to associate each process execution - with a custom label, so that it will be easier to identify them in the log file - or in the trace execution report. - - - See [`tag`](https://www.nextflow.io/docs/latest/process.html#tag). - - ' - example: - - example: '"foo"' - format: yaml - - example: '''$id''' - format: yaml - name: tag - type: Option of String -- description: 'The `conda` directive allows for the definition of the process dependencies - using the Conda package manager. - - - Nextflow automatically sets up an environment for the given package names listed - by in the `conda` directive. - - - See [`conda`](https://www.nextflow.io/docs/latest/process.html#conda). - - ' - example: - - example: '"bwa=0.7.15"' - format: yaml - - example: '"bwa=0.7.15 fastqc=0.11.5"' - format: yaml - - example: '["bwa=0.7.15", "fastqc=0.11.5"]' - format: yaml - name: conda - type: OneOrMore of String -- description: ' The `machineType` can be used to specify a predefined Google Compute - Platform machine type when running using the Google Life Sciences executor. - - - See [`machineType`](https://www.nextflow.io/docs/latest/process.html#machinetype). - - ' - example: - - example: '"n1-highmem-8"' - format: yaml - name: machineType - type: Option of String -- description: "The `stageInMode` directive defines how input files are staged-in\ - \ to the process work directory. The following values are allowed:\n\n| Value\ - \ | Description |\n|-------|-------------| \n| copy | Input files are staged in\ - \ the process work directory by creating a copy. | \n| link | Input files are\ - \ staged in the process work directory by creating an (hard) link for each of\ - \ them. | \n| symlink | Input files are staged in the process work directory by\ - \ creating a symbolic link with an absolute path for each of them (default). |\ - \ \n| rellink | Input files are staged in the process work directory by creating\ - \ a symbolic link with a relative path for each of them. | \n\nSee [`stageInMode`](https://www.nextflow.io/docs/latest/process.html#stageinmode).\n" - example: - - example: '"copy"' - format: yaml - - example: '"link"' - format: yaml - name: stageInMode - type: Option of String -- description: 'The `cache` directive allows you to store the process results to a - local cache. When the cache is enabled and the pipeline is launched with the resume - option, any following attempt to execute the process, along with the same inputs, - will cause the process execution to be skipped, producing the stored data as the - actual results. - - - The caching feature generates a unique key by indexing the process script and - inputs. This key is used to identify univocally the outputs produced by the process - execution. - - - The `cache` is enabled by default, you can disable it for a specific process by - setting the cache directive to `false`. - - - Accepted values are: `true`, `false`, `"deep"`, and `"lenient"`. - - - See [`cache`](https://www.nextflow.io/docs/latest/process.html#cache). - - ' - example: - - example: 'true' - format: yaml - - example: 'false' - format: yaml - - example: '"deep"' - format: yaml - - example: '"lenient"' - format: yaml - name: cache - type: Option[Either of Boolean,String] -- description: 'The `pod` directive allows the definition of pods specific settings, - such as environment variables, secrets and config maps when using the Kubernetes - executor. - - - See [`pod`](https://www.nextflow.io/docs/latest/process.html#pod). - - ' - example: - - example: '[ label: "key", value: "val" ]' - format: yaml - - example: '[ annotation: "key", value: "val" ]' - format: yaml - - example: '[ env: "key", value: "val" ]' - format: yaml - - example: '[ [label: "l", value: "v"], [env: "e", value: "v"]]' - format: yaml - name: pod - type: OneOrMore[Map of String,String] -- description: 'The `penv` directive allows you to define the parallel environment - to be used when submitting a parallel task to the SGE resource manager. - - - See [`penv`](https://www.nextflow.io/docs/latest/process.html#penv). - - ' - example: - - example: '"smp"' - format: yaml - name: penv - type: Option of String -- description: 'The `scratch` directive allows you to execute the process in a temporary - folder that is local to the execution node. - - - See [`scratch`](https://www.nextflow.io/docs/latest/process.html#scratch). - - ' - example: - - example: 'true' - format: yaml - - example: '"/path/to/scratch"' - format: yaml - - example: '''$MY_PATH_TO_SCRATCH''' - format: yaml - - example: '"ram-disk"' - format: yaml - name: scratch - type: Option[Either of Boolean,String] -- description: 'The `storeDir` directive allows you to define a directory that is - used as a permanent cache for your process results. - - - See [`storeDir`](https://www.nextflow.io/docs/latest/process.html#storeDir). - - ' - example: - - example: '"/path/to/storeDir"' - format: yaml - name: storeDir - type: Option of String -- description: 'The `maxRetries` directive allows you to define the maximum number - of times a process instance can be re-submitted in case of failure. This value - is applied only when using the retry error strategy. By default only one retry - is allowed. - - - See [`maxRetries`](https://www.nextflow.io/docs/latest/process.html#maxretries). - - ' - example: - - example: '1' - format: yaml - - example: '3' - format: yaml - name: maxRetries - type: Option[Either of String,Int] -- description: "By default the stdout produced by the commands executed in all processes\ - \ is ignored. By setting the `echo` directive to true, you can forward the process\ - \ stdout to the current top running process stdout file, showing it in the shell\ - \ terminal.\n \nSee [`echo`](https://www.nextflow.io/docs/latest/process.html#echo).\n" - example: - - example: 'true' - format: yaml - - example: 'false' - format: yaml - name: echo - type: Option[Either of Boolean,String] -- description: 'The `errorStrategy` directive allows you to define how an error condition - is managed by the process. By default when an error status is returned by the - executed script, the process stops immediately. This in turn forces the entire - pipeline to terminate. - - - Table of available error strategies: - - | Name | Executor | - - |------|----------| - - | `terminate` | Terminates the execution as soon as an error condition is reported. - Pending jobs are killed (default) | - - | `finish` | Initiates an orderly pipeline shutdown when an error condition is - raised, waiting the completion of any submitted job. | - - | `ignore` | Ignores processes execution errors. | - - | `retry` | Re-submit for execution a process returning an error condition. | - - - See [`errorStrategy`](https://www.nextflow.io/docs/latest/process.html#errorstrategy). - - ' - example: - - example: '"terminate"' - format: yaml - - example: '"finish"' - format: yaml - name: errorStrategy - type: Option of String -- description: 'The `memory` directive allows you to define how much memory the process - is allowed to use. - - - See [`memory`](https://www.nextflow.io/docs/latest/process.html#memory). - - ' - example: - - example: '"1 GB"' - format: yaml - - example: '"2TB"' - format: yaml - - example: '"3.2KB"' - format: yaml - - example: '"10.B"' - format: yaml - name: memory - type: Option of String -- description: "The `stageOutMode` directive defines how output files are staged-out\ - \ from the scratch directory to the process work directory. The following values\ - \ are allowed:\n\n| Value | Description |\n|-------|-------------| \n| copy |\ - \ Output files are copied from the scratch directory to the work directory. |\ - \ \n| move | Output files are moved from the scratch directory to the work directory.\ - \ | \n| rsync | Output files are copied from the scratch directory to the work\ - \ directory by using the rsync utility. |\n\nSee [`stageOutMode`](https://www.nextflow.io/docs/latest/process.html#stageoutmode).\n" - example: - - example: '"copy"' - format: yaml - - example: '"link"' - format: yaml - name: stageOutMode - type: Option of String -title: Nextflow Directives -topic: nextflowParameters diff --git a/reference/config/platforms/nextflow/_index.yaml b/reference/config/platforms/nextflow/_index.yaml deleted file mode 100644 index 8a66ae08..00000000 --- a/reference/config/platforms/nextflow/_index.yaml +++ /dev/null @@ -1,76 +0,0 @@ -data: -- description: Next-gen platform for generating NextFlow VDSL3 modules. - example: - - example: "platforms:\n - type: nextflow\n directives:\n label: [lowcpu,\ - \ midmem]\n" - format: yaml - hierarchy: - - io.viash.platforms.NextflowVdsl3Platform - - io.viash.platforms.NextflowPlatform - - io.viash.platforms.Platform - name: __this__ - type: NextflowVdsl3Platform -- description: '[Automated processing flags](/reference/config/platforms/nextflow/auto.html) - which can be toggled on or off: - - - | Flag | Description | Default | - - |---|---------|----| - - | `simplifyInput` | If `true`, an input tuple only containing only a single File - (e.g. `["foo", file("in.h5ad")]`) is automatically transformed to a map (i.e. - `["foo", [ input: file("in.h5ad") ] ]`). | `true` | - - | `simplifyOutput` | If `true`, an output tuple containing a map with a File (e.g. - `["foo", [ output: file("out.h5ad") ] ]`) is automatically transformed to a map - (i.e. `["foo", file("out.h5ad")]`). | `true` | - - | `transcript` | If `true`, the module''s transcripts from `work/` are automatically - published to `params.transcriptDir`. If not defined, `params.publishDir + "/_transcripts"` - will be used. Will throw an error if neither are defined. | `false` | - - | `publish` | If `true`, the module''s outputs are automatically published to - `params.publishDir`. Will throw an error if `params.publishDir` is not defined. - | `false` | - - - ' - example: - - example: "auto:\n publish: true" - format: yaml - name: auto - type: NextflowAuto -- description: "[Directives](/reference/config/platforms/nextflow/directives.html)\ - \ are optional settings that affect the execution of the process. These mostly\ - \ match up with the Nextflow counterparts. \n" - example: - - example: "directives:\n container: rocker/r-ver:4.1\n label: highcpu\n cpus:\ - \ 4\n memory: 16 GB" - format: yaml - name: directives - type: NextflowDirectives -- description: Specifies the Docker platform id to be used to run Nextflow. - name: container - type: String -- description: Whether or not to print debug messages. - name: debug - type: Boolean -- description: Every platform can be given a specific id that can later be referred - to explicitly when running or building the Viash component. - example: - - example: 'id: foo' - format: yaml - name: id - type: String -- description: Specifies the type of the platform. - name: type - type: String -- description: Allows tweaking how the [Nextflow Config](/reference/config/platforms/nextflow/config.html) - file is generated. - name: config - since: Viash 0.7.4 - type: NextflowConfig -order: 30 -title: Nextflow Vdsl3 Platform -topic: platforms diff --git a/reference/config/platforms/nextflow/auto.qmd b/reference/config/platforms/nextflow/auto.qmd index 079bde2f..49c7cafe 100644 --- a/reference/config/platforms/nextflow/auto.qmd +++ b/reference/config/platforms/nextflow/auto.qmd @@ -9,6 +9,8 @@ Automated processing flags which can be toggled on or off. **Type**: `Boolean` +**Default**: `False` + If `true`, the module's outputs are automatically published to `params.publishDir`. Will throw an error if `params.publishDir` is not defined. @@ -19,6 +21,8 @@ Default: `false`. **Type**: `Boolean` +**Default**: `True` + If `true`, an input tuple only containing only a single File (e.g. `["foo", file("in.h5ad")]`) is automatically transformed to a map (i.e. `["foo", [ input: file("in.h5ad") ] ]`). Default: `true`. @@ -28,6 +32,8 @@ Default: `true`. **Type**: `Boolean` +**Default**: `True` + If `true`, an output tuple containing a map with a File (e.g. `["foo", [ output: file("out.h5ad") ] ]`) is automatically transformed to a map (i.e. `["foo", file("out.h5ad")]`). Default: `true`. @@ -37,10 +43,11 @@ Default: `true`. **Type**: `Boolean` +**Default**: `False` + If `true`, the module's transcripts from `work/` are automatically published to `params.transcriptDir`. If not defined, `params.publishDir + "/_transcripts"` will be used. Will throw an error if neither are defined. Default: `false`. - diff --git a/reference/config/platforms/nextflow/config.qmd b/reference/config/platforms/nextflow/config.qmd index f25bed5f..e16b8fc7 100644 --- a/reference/config/platforms/nextflow/config.qmd +++ b/reference/config/platforms/nextflow/config.qmd @@ -7,7 +7,9 @@ Allows tweaking how the Nextflow Config file is generated. ## labels -**Type**: `ListMap of String,String` +**Type**: `Map of String to String` + +**Default**: `A series of default labels to specify memory and cpu constraints` A series of default labels to specify memory and cpu constraints. @@ -62,6 +64,8 @@ config_mods: | **Type**: `String` / `List of String` +**Default**: `Empty` + Includes a single string or list of strings into the nextflow.config file. This can be used to add custom profiles or include an additional config file. @@ -80,4 +84,3 @@ script: ```yaml script: includeConfig("config.config") ``` - diff --git a/reference/config/platforms/nextflow/directives.qmd b/reference/config/platforms/nextflow/directives.qmd index 8ca94f28..2231baea 100644 --- a/reference/config/platforms/nextflow/directives.qmd +++ b/reference/config/platforms/nextflow/directives.qmd @@ -18,7 +18,9 @@ directives: ## accelerator -**Type**: `Map of String,String` +**Type**: `Map of String to String` + +**Default**: `Empty` The `accelerator` directive allows you to specify the hardware accelerator requirement for the task execution e.g. GPU processor. @@ -37,6 +39,8 @@ See [`accelerator`](https://www.nextflow.io/docs/latest/process.html#accelerator **Type**: `String` +**Default**: `Empty` + The `afterScript` directive allows you to execute a custom (Bash) snippet immediately after the main process has run. This may be useful to clean up your staging area. See [`afterScript`](https://www.nextflow.io/docs/latest/process.html#afterscript). @@ -52,6 +56,8 @@ source /cluster/bin/cleanup **Type**: `String` +**Default**: `Empty` + The `beforeScript` directive allows you to execute a custom (Bash) snippet before the main process script is run. This may be useful to initialise the underlying cluster environment or for other custom initialisation. See [`beforeScript`](https://www.nextflow.io/docs/latest/process.html#beforeScript). @@ -65,7 +71,11 @@ source /cluster/bin/setup ## cache -**Type**: `Option[Either of Boolean,String]` +**Type**: `Either + - Boolean + - String` + +**Default**: `Empty` The `cache` directive allows you to store the process results to a local cache. When the cache is enabled and the pipeline is launched with the resume option, any following attempt to execute the process, along with the same inputs, will cause the process execution to be skipped, producing the stored data as the actual results. @@ -100,6 +110,8 @@ false **Type**: `String` / `List of String` +**Default**: `Empty` + The `conda` directive allows for the definition of the process dependencies using the Conda package manager. Nextflow automatically sets up an environment for the given package names listed by in the `conda` directive. @@ -123,7 +135,11 @@ See [`conda`](https://www.nextflow.io/docs/latest/process.html#conda). ## container -**Type**: `Option[Either[Map of String,String,String]]` +**Type**: `Either + - Map of String to String + - String` + +**Default**: `Empty` The `container` directive allows you to execute the process script in a Docker container. @@ -156,6 +172,8 @@ This is transformed to `"im:latest"`: **Type**: `String` / `List of String` +**Default**: `Empty` + The `containerOptions` directive allows you to specify any container execution option supported by the underlying container engine (ie. Docker, Singularity, etc). This can be useful to provide container settings only for a specific process e.g. mount a custom path. See [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#containeroptions). @@ -173,7 +191,11 @@ See [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#contai ## cpus -**Type**: `Option[Either of Int,String]` +**Type**: `Either + - Int + - String` + +**Default**: `Empty` The `cpus` directive allows you to define the number of (logical) CPU required by the process' task. @@ -194,6 +216,8 @@ See [`cpus`](https://www.nextflow.io/docs/latest/process.html#cpus). **Type**: `String` +**Default**: `Empty` + The `disk` directive allows you to define how much local disk storage the process is allowed to use. See [`disk`](https://www.nextflow.io/docs/latest/process.html#disk). @@ -219,7 +243,11 @@ See [`disk`](https://www.nextflow.io/docs/latest/process.html#disk). ## echo -**Type**: `Option[Either of Boolean,String]` +**Type**: `Either + - Boolean + - String` + +**Default**: `Empty` By default the stdout produced by the commands executed in all processes is ignored. By setting the `echo` directive to true, you can forward the process stdout to the current top running process stdout file, showing it in the shell terminal. @@ -240,6 +268,8 @@ false **Type**: `String` +**Default**: `Empty` + The `errorStrategy` directive allows you to define how an error condition is managed by the process. By default when an error status is returned by the executed script, the process stops immediately. This in turn forces the entire pipeline to terminate. Table of available error strategies: @@ -267,6 +297,8 @@ See [`errorStrategy`](https://www.nextflow.io/docs/latest/process.html#errorstra **Type**: `String` +**Default**: `Empty` + The `executor` defines the underlying system where processes are executed. By default a process uses the executor defined globally in the nextflow.config file. The `executor` directive allows you to configure what executor has to be used by the process, overriding the default configuration. The following values can be used: @@ -308,6 +340,8 @@ See [`executor`](https://www.nextflow.io/docs/latest/process.html#executor). **Type**: `String` / `List of String` +**Default**: `Empty` + The `label` directive allows the annotation of processes with mnemonic identifier of your choice. See [`label`](https://www.nextflow.io/docs/latest/process.html#label). @@ -331,6 +365,8 @@ See [`label`](https://www.nextflow.io/docs/latest/process.html#label). **Type**: `String` +**Default**: `Empty` + The `machineType` can be used to specify a predefined Google Compute Platform machine type when running using the Google Life Sciences executor. See [`machineType`](https://www.nextflow.io/docs/latest/process.html#machinetype). @@ -344,7 +380,11 @@ See [`machineType`](https://www.nextflow.io/docs/latest/process.html#machinetype ## maxErrors -**Type**: `Option[Either of String,Int]` +**Type**: `Either + - String + - Int` + +**Default**: `Empty` The `maxErrors` directive allows you to specify the maximum number of times a process can fail when using the `retry` error strategy. By default this directive is disabled. @@ -363,7 +403,11 @@ See [`maxErrors`](https://www.nextflow.io/docs/latest/process.html#maxerrors). ## maxForks -**Type**: `Option[Either of String,Int]` +**Type**: `Either + - String + - Int` + +**Default**: `Empty` The `maxForks` directive allows you to define the maximum number of process instances that can be executed in parallel. By default this value is equals to the number of CPU cores available minus 1. @@ -384,7 +428,11 @@ See [`maxForks`](https://www.nextflow.io/docs/latest/process.html#maxforks). ## maxRetries -**Type**: `Option[Either of String,Int]` +**Type**: `Either + - String + - Int` + +**Default**: `Empty` The `maxRetries` directive allows you to define the maximum number of times a process instance can be re-submitted in case of failure. This value is applied only when using the retry error strategy. By default only one retry is allowed. @@ -405,6 +453,8 @@ See [`maxRetries`](https://www.nextflow.io/docs/latest/process.html#maxretries). **Type**: `String` +**Default**: `Empty` + The `memory` directive allows you to define how much memory the process is allowed to use. See [`memory`](https://www.nextflow.io/docs/latest/process.html#memory). @@ -432,6 +482,8 @@ See [`memory`](https://www.nextflow.io/docs/latest/process.html#memory). **Type**: `String` / `List of String` +**Default**: `Empty` + Environment Modules is a package manager that allows you to dynamically configure your execution environment and easily switch between multiple versions of the same software tool. If it is available in your system you can use it with Nextflow in order to configure the processes execution environment in your pipeline. @@ -459,6 +511,8 @@ See [`module`](https://www.nextflow.io/docs/latest/process.html#module). **Type**: `String` +**Default**: `Empty` + The `penv` directive allows you to define the parallel environment to be used when submitting a parallel task to the SGE resource manager. See [`penv`](https://www.nextflow.io/docs/latest/process.html#penv). @@ -472,7 +526,9 @@ See [`penv`](https://www.nextflow.io/docs/latest/process.html#penv). ## pod -**Type**: `OneOrMore[Map of String,String]` +**Type**: `Map of String to String` / `List of Map of String to String` + +**Default**: `Empty` The `pod` directive allows the definition of pods specific settings, such as environment variables, secrets and config maps when using the Kubernetes executor. @@ -499,7 +555,13 @@ See [`pod`](https://www.nextflow.io/docs/latest/process.html#pod). ## publishDir -**Type**: `OneOrMore[Either[String,Map of String,String]]` +**Type**: `Either + - String + - Map of String to String` / `List of Either + - String + - Map of String to String` + +**Default**: `Empty` The `publishDir` directive allows you to publish the process output files to a specified folder. @@ -535,6 +597,8 @@ This is transformed to `[[ path: "/path/to/dir", mode: "cache" ]]`: **Type**: `String` / `List of String` +**Default**: `Empty` + The `queue` directory allows you to set the queue where jobs are scheduled when using a grid based executor in your pipeline. See [`queue`](https://www.nextflow.io/docs/latest/process.html#queue). @@ -556,7 +620,11 @@ See [`queue`](https://www.nextflow.io/docs/latest/process.html#queue). ## scratch -**Type**: `Option[Either of Boolean,String]` +**Type**: `Either + - Boolean + - String` + +**Default**: `Empty` The `scratch` directive allows you to execute the process in a temporary folder that is local to the execution node. @@ -585,6 +653,8 @@ true **Type**: `String` +**Default**: `Empty` + The `stageInMode` directive defines how input files are staged-in to the process work directory. The following values are allowed: | Value | Description | @@ -611,6 +681,8 @@ See [`stageInMode`](https://www.nextflow.io/docs/latest/process.html#stageinmode **Type**: `String` +**Default**: `Empty` + The `stageOutMode` directive defines how output files are staged-out from the scratch directory to the process work directory. The following values are allowed: | Value | Description | @@ -636,6 +708,8 @@ See [`stageOutMode`](https://www.nextflow.io/docs/latest/process.html#stageoutmo **Type**: `String` +**Default**: `Empty` + The `storeDir` directive allows you to define a directory that is used as a permanent cache for your process results. See [`storeDir`](https://www.nextflow.io/docs/latest/process.html#storeDir). @@ -651,6 +725,8 @@ See [`storeDir`](https://www.nextflow.io/docs/latest/process.html#storeDir). **Type**: `String` +**Default**: `Empty` + The `tag` directive allows you to associate each process execution with a custom label, so that it will be easier to identify them in the log file or in the trace execution report. See [`tag`](https://www.nextflow.io/docs/latest/process.html#tag). @@ -670,6 +746,8 @@ See [`tag`](https://www.nextflow.io/docs/latest/process.html#tag). **Type**: `String` +**Default**: `Empty` + The `time` directive allows you to define how long a process is allowed to run. See [`time`](https://www.nextflow.io/docs/latest/process.html#time). @@ -688,4 +766,3 @@ See [`time`](https://www.nextflow.io/docs/latest/process.html#time). ```yaml "1day 6hours 3minutes 30seconds" ``` - diff --git a/reference/config/platforms/nextflow/index.qmd b/reference/config/platforms/nextflow/index.qmd index 981faf43..78d9952c 100644 --- a/reference/config/platforms/nextflow/index.qmd +++ b/reference/config/platforms/nextflow/index.qmd @@ -1,10 +1,10 @@ --- -title: "Nextflow Vdsl3 Platform" +title: "Nextflow Platform" search: true order: 30 --- -Next-gen platform for generating NextFlow VDSL3 modules. +Platform for generating Nextflow VDSL3 modules. **Example:** @@ -20,6 +20,12 @@ platforms: **Type**: `NextflowAuto` +**Default**: `simplifyInput: true +simplifyOutput: true +transcript: false +publish: false +` + [Automated processing flags](/reference/config/platforms/nextflow/auto.html) which can be toggled on or off: | Flag | Description | Default | @@ -42,24 +48,32 @@ auto: **Type**: `NextflowConfig` +**Default**: `A series of default labels to specify memory and cpu constraints` + Allows tweaking how the [Nextflow Config](/reference/config/platforms/nextflow/config.html) file is generated. ## container **Type**: `String` +**Default**: `docker` + Specifies the Docker platform id to be used to run Nextflow. ## debug **Type**: `Boolean` +**Default**: `False` + Whether or not to print debug messages. ## directives **Type**: `NextflowDirectives` +**Default**: `Empty` + [Directives](/reference/config/platforms/nextflow/directives.html) are optional settings that affect the execution of the process. These mostly match up with the Nextflow counterparts. @@ -77,6 +91,8 @@ directives: **Type**: `String` +**Default**: `nextflow` + Every platform can be given a specific id that can later be referred to explicitly when running or building the Viash component. **Example:** @@ -90,4 +106,3 @@ id: foo **Type**: `String` Specifies the type of the platform. - diff --git a/reference/config/platforms/nextflowLegacy/_index.yaml b/reference/config/platforms/nextflowLegacy/_index.yaml deleted file mode 100644 index 94860c23..00000000 --- a/reference/config/platforms/nextflowLegacy/_index.yaml +++ /dev/null @@ -1,208 +0,0 @@ -data: -- description: Run a Viash component as a Nextflow module. - hierarchy: - - io.viash.platforms.NextflowLegacyPlatform - - io.viash.platforms.NextflowPlatform - - io.viash.platforms.Platform - name: __this__ - removed: - deprecation: 0.6.0 - message: 'Nextflow platform with `variant: legacy` was removed' - removal: 0.7.0 - type: NextflowLegacyPlatform -- description: Name of a container's [organization](https://docs.docker.com/docker-hub/orgs/). - example: - - example: 'organization: viash-io' - format: yaml - name: organization - type: Option of String -- description: 'If no image attributes are configured, Viash will use the auto-generated - image name from the Docker platform: - - - ``` - - [/]: - - ``` - - It''s possible to specify the container image explicitly with which to run the - module in different ways: - - - ``` - - image: dataintuitive/viash:0.4.0 - - ``` - - Exactly the same can be obtained with - - - ``` - - image: dataintuitive/viash - - registry: index.docker.io/v1/ - - tag: 0.4.0 - - ``` - - Specifying the attribute(s) like this will use the container `dataintuitive/viash:0.4.0` - from Docker hub (registry). - - - If no tag is specified Viash will use `functionality.version` as the tag. - - - If no registry is specified, Viash (and NextFlow) will assume the image is available - locally or on Docker Hub. In other words, the `registry: ...` attribute above - is superfluous. No other registry is checked automatically due to a limitation - from Docker itself. - - ' - name: image - type: Option of String -- description: Specify a Docker image based on its tag. - example: - - example: 'tag: 4.0' - format: yaml - name: tag - type: Option of String -- description: "When running the module in a cluster context and depending on the\ - \ cluster type, [NextFlow allows for attaching labels](https://www.nextflow.io/docs/latest/process.html#label)\ - \ to the process that can later be used as selectors for associating resources\ - \ to this process.\n\nIn order to attach one label to a process/component, one\ - \ can use the `label: ...` attribute, multiple labels can be added using `labels:\ - \ [ ..., ... ]` and the two can even be mixed.\n\nIn the main `nextflow.config`,\ - \ one can now use this label:\n\nprocess {\n ...\n withLabel: bigmem {\n \ - \ maxForks = 5\n ...\n }\n}\n" - example: - - example: 'label: highmem labels: [ highmem, highcpu ]' - format: yaml - name: label - type: Option of String -- description: 'By default NextFlow will create a symbolic link to the inputs for - a process/module and run the tool at hand using those symbolic links. Some applications - do not cope well with this strategy, in that case the files should effectively - be copied rather than linked to. This can be achieved by using `stageInMode: copy`. - - This attribute is optional, the default is `symlink`. - - ' - example: - - example: 'stageInMode: copy' - format: yaml - name: stageInMode - type: Option of String -- description: Every platform can be given a specific id that can later be referred - to explicitly when running or building the Viash component. - name: id - type: String -- description: "When running the module in a cluster context and depending on the\ - \ cluster type, [NextFlow allows for attaching labels](https://www.nextflow.io/docs/latest/process.html#label)\ - \ to the process that can later be used as selectors for associating resources\ - \ to this process.\n\nIn order to attach one label to a process/component, one\ - \ can use the `label: ...` attribute, multiple labels can be added using `labels:\ - \ [ ..., ... ]` and the two can even be mixed.\n\nIn the main `nextflow.config`,\ - \ one can now use this label:\n\nprocess {\n ...\n withLabel: bigmem {\n \ - \ maxForks = 5\n ...\n }\n}\n" - example: - - example: 'label: highmem labels: [ highmem, highcpu ]' - format: yaml - name: labels - type: OneOrMore of String -- description: Specifies the type of the platform. - name: type - type: String -- description: The URL to the a [custom Docker registry](https://docs.docker.com/registry/). - example: - - example: 'registry: https://my-docker-registry.org' - format: yaml - name: registry - type: Option of String -- description: "By default, a subdirectory is created corresponding to the unique\ - \ ID that is passed in the triplet. Let us illustrate this with an example. The\ - \ following code snippet uses the value of `--input` as an input of a workflow.\ - \ The input can include a wildcard so that multiple samples can run in parallel.\ - \ We use the parent directory name (`.getParent().baseName`) as an identifier\ - \ for the sample. We pass this as the first entry of the triplet:\n\n```\nChannel.fromPath(params.input)\ - \ \\\n | map{ it -> [ it.getParent().baseName , it ] } \\\n | map{ it ->\ - \ [ it[0] , it[1], params ] }\n | ...\n```\nSay the resulting sample names\ - \ are `SAMPLE1` and `SAMPLE2`. The next step in the pipeline will be published\ - \ (at least by default) under:\n```\n/SAMPLE1/\n/SAMPLE2/\n\ - ```\nThese per-ID subdirectories can be avoided by setting:\n```\nper_id: false\n\ - ```\n" - name: per_id - type: Option of Boolean -- description: 'When `publish: true`, this attribute defines where the output is written - relative to the `params.publishDir` setting. For example, `path: processed` in - combination with `--output s3://some_bucket/` will store the output of this component - under - - ``` - - s3://some_bucket/processed/ - - ``` - - This attribute gives control over the directory structure of the output. For example: - - ``` - - path: raw_data - - ``` - - Or even: - - ``` - - path: raw_data/bcl - - ``` - - Please note that `per_id` and `path` can be combined. - - ' - name: path - type: Option of String -- description: 'Separates the outputs generated by a Nextflow component with multiple - outputs as separate events on the channel. Default value: `true`.' - example: - - example: 'separate_multiple_outputs: false' - format: yaml - name: separate_multiple_outputs - type: Boolean -- description: The default namespace separator is "_". - example: - - example: 'namespace_separator: "+"' - format: yaml - name: namespace_separator - type: String -- description: "NextFlow uses the autogenerated `work` dirs to manage process IO under\ - \ the hood. In order effectively output something one can publish the results\ - \ a module or step in the pipeline. In order to do this, add `publish: true` to\ - \ the config:\n\n - publish is optional\n - Default value is false\n\nThis attribute\ - \ simply defines if output of a component should be published yes or no. The output\ - \ location has to be provided at pipeline launch by means of the option `--publishDir\ - \ ...` or as `params.publishDir` in `nextflow.config`:\n```\nparams.publishDir\ - \ = \"...\"\n```\n" - name: publish - type: Option of Boolean -- name: version - removed: - deprecation: 0.4.0 - message: 'nextflow platform: attribute ''version'' was removed' - removal: 0.7.0 - type: Option of String -- name: executor - removed: - deprecation: 0.6.3 - message: Undocumented & stale value - removal: 0.7.0 - type: Option of String -order: 40 -title: Nextflow Legacy Platform -topic: platforms diff --git a/reference/config/platforms/nextflowLegacy/index.qmd b/reference/config/platforms/nextflowLegacy/index.qmd deleted file mode 100644 index 9465c748..00000000 --- a/reference/config/platforms/nextflowLegacy/index.qmd +++ /dev/null @@ -1,251 +0,0 @@ ---- -title: "Nextflow Legacy Platform" -search: true -order: 40 ---- - -::: {.callout-warning} -Deprecated since 0.6.0. -Removed since 0.7.0. Nextflow platform with `variant: legacy` was removed -::: -Run a Viash component as a Nextflow module. - -## executor - -**Type**: `String` - -::: {.callout-warning} -Deprecated since 0.6.3. -Removed since 0.7.0. Undocumented & stale value -::: - -## id - -**Type**: `String` - -Every platform can be given a specific id that can later be referred to explicitly when running or building the Viash component. - -## image - -**Type**: `String` - -If no image attributes are configured, Viash will use the auto-generated image name from the Docker platform: - -``` -[/]: -``` -It's possible to specify the container image explicitly with which to run the module in different ways: - -``` -image: dataintuitive/viash:0.4.0 -``` -Exactly the same can be obtained with - -``` -image: dataintuitive/viash -registry: index.docker.io/v1/ -tag: 0.4.0 -``` -Specifying the attribute(s) like this will use the container `dataintuitive/viash:0.4.0` from Docker hub (registry). - -If no tag is specified Viash will use `functionality.version` as the tag. - -If no registry is specified, Viash (and NextFlow) will assume the image is available locally or on Docker Hub. In other words, the `registry: ...` attribute above is superfluous. No other registry is checked automatically due to a limitation from Docker itself. - - -## label - -**Type**: `String` - -When running the module in a cluster context and depending on the cluster type, [NextFlow allows for attaching labels](https://www.nextflow.io/docs/latest/process.html#label) to the process that can later be used as selectors for associating resources to this process. - -In order to attach one label to a process/component, one can use the `label: ...` attribute, multiple labels can be added using `labels: [ ..., ... ]` and the two can even be mixed. - -In the main `nextflow.config`, one can now use this label: - -process { - ... - withLabel: bigmem { - maxForks = 5 - ... - } -} - - -**Example:** - -```yaml -label: highmem labels: [ highmem, highcpu ] -``` - -## labels - -**Type**: `String` / `List of String` - -When running the module in a cluster context and depending on the cluster type, [NextFlow allows for attaching labels](https://www.nextflow.io/docs/latest/process.html#label) to the process that can later be used as selectors for associating resources to this process. - -In order to attach one label to a process/component, one can use the `label: ...` attribute, multiple labels can be added using `labels: [ ..., ... ]` and the two can even be mixed. - -In the main `nextflow.config`, one can now use this label: - -process { - ... - withLabel: bigmem { - maxForks = 5 - ... - } -} - - -**Example:** - -```yaml -label: highmem labels: [ highmem, highcpu ] -``` - -## namespace_separator - -**Type**: `String` - -The default namespace separator is "_". - -**Example:** - -```yaml -namespace_separator: "+" -``` - -## organization - -**Type**: `String` - -Name of a container's [organization](https://docs.docker.com/docker-hub/orgs/). - -**Example:** - -```yaml -organization: viash-io -``` - -## path - -**Type**: `String` - -When `publish: true`, this attribute defines where the output is written relative to the `params.publishDir` setting. For example, `path: processed` in combination with `--output s3://some_bucket/` will store the output of this component under -``` -s3://some_bucket/processed/ -``` -This attribute gives control over the directory structure of the output. For example: -``` -path: raw_data -``` -Or even: -``` -path: raw_data/bcl -``` -Please note that `per_id` and `path` can be combined. - - -## per_id - -**Type**: `Boolean` - -By default, a subdirectory is created corresponding to the unique ID that is passed in the triplet. Let us illustrate this with an example. The following code snippet uses the value of `--input` as an input of a workflow. The input can include a wildcard so that multiple samples can run in parallel. We use the parent directory name (`.getParent().baseName`) as an identifier for the sample. We pass this as the first entry of the triplet: - -``` -Channel.fromPath(params.input) \ - | map{ it -> [ it.getParent().baseName , it ] } \ - | map{ it -> [ it[0] , it[1], params ] } - | ... -``` -Say the resulting sample names are `SAMPLE1` and `SAMPLE2`. The next step in the pipeline will be published (at least by default) under: -``` -/SAMPLE1/ -/SAMPLE2/ -``` -These per-ID subdirectories can be avoided by setting: -``` -per_id: false -``` - - -## publish - -**Type**: `Boolean` - -NextFlow uses the autogenerated `work` dirs to manage process IO under the hood. In order effectively output something one can publish the results a module or step in the pipeline. In order to do this, add `publish: true` to the config: - - - publish is optional - - Default value is false - -This attribute simply defines if output of a component should be published yes or no. The output location has to be provided at pipeline launch by means of the option `--publishDir ...` or as `params.publishDir` in `nextflow.config`: -``` -params.publishDir = "..." -``` - - -## registry - -**Type**: `String` - -The URL to the a [custom Docker registry](https://docs.docker.com/registry/). - -**Example:** - -```yaml -registry: https://my-docker-registry.org -``` - -## separate_multiple_outputs - -**Type**: `Boolean` - -Separates the outputs generated by a Nextflow component with multiple outputs as separate events on the channel. Default value: `true`. - -**Example:** - -```yaml -separate_multiple_outputs: false -``` - -## stageInMode - -**Type**: `String` - -By default NextFlow will create a symbolic link to the inputs for a process/module and run the tool at hand using those symbolic links. Some applications do not cope well with this strategy, in that case the files should effectively be copied rather than linked to. This can be achieved by using `stageInMode: copy`. -This attribute is optional, the default is `symlink`. - - -**Example:** - -```yaml -stageInMode: copy -``` - -## tag - -**Type**: `String` - -Specify a Docker image based on its tag. - -**Example:** - -```yaml -tag: 4.0 -``` - -## type - -**Type**: `String` - -Specifies the type of the platform. - -## version - -**Type**: `String` - -::: {.callout-warning} -Deprecated since 0.4.0. -Removed since 0.7.0. nextflow platform: attribute 'version' was removed -::: - diff --git a/reference/config_schema_export.json b/reference/config_schema_export.json index d44c9f3b..f3871bdf 100644 --- a/reference/config_schema_export.json +++ b/reference/config_schema_export.json @@ -1,3569 +1,3979 @@ -{ - "config" : { - "config" : [ - { - "name" : "__this__", - "type" : "Config", - "hierarchy" : [ - "io.viash.config.Config" - ], - "description" : "A Viash configuration is a YAML file which contains metadata to describe the behaviour and build target(s) of a component. \nWe commonly name this file `config.vsh.yaml` in our examples, but you can name it however you choose. \n", - "example" : [ - { - "example" : "functionality:\n name: hello_world\n arguments:\n - type: string\n name: --input\n default: \"world\"\n resources:\n - type: bash_script\n path: script.sh\n text: echo Hello $par_input\nplatforms:\n - type: docker\n image: \"bash:4.0\"\n", - "format" : "yaml" - } - ] - }, - { - "name" : "functionality", - "type" : "Functionality", - "description" : "The @[functionality](functionality) describes the behaviour of the script in terms of arguments and resources.\nBy specifying a few restrictions (e.g. mandatory arguments) and adding some descriptions, Viash will automatically generate a stylish command-line interface for you.\n" - }, - { - "name" : "platforms", - "type" : "List of Platform", - "description" : "A list of platforms to generate target artifacts for.\n\n - @[Native](platform_native)\n - @[Docker](platform_docker)\n - @[Nextflow VDSL3](platform_nextflow)\n" - }, - { - "name" : "__merge__", - "type" : "Option of File", - "description" : "Config inheritance by including YAML partials. This is useful for defining common APIs in\nseparate files. `__merge__` can be used in any level of the YAML. For example,\nnot just in the config but also in the functionality or any of the platforms.\n", - "example" : [ - { - "example" : "__merge__: ../api/common_interface.yaml", - "format" : "yaml" - } - ], - "since" : "Viash 0.6.3" - } - ], - "project" : [ - { - "name" : "__this__", - "type" : "ViashProject", - "hierarchy" : [ - "io.viash.project.ViashProject" - ], - "description" : "A Viash project configuration file. It's name should be `_viash.yaml`.", - "example" : [ - { - "example" : "viash_version: 0.6.4\nsource: src\ntarget: target\nconfig_mods: |\n .platforms[.type == 'docker'].target_registry := 'ghcr.io'\n .platforms[.type == 'docker'].target_organization := 'viash-io'\n .platforms[.type == 'docker'].namespace_separator := '/'\n .platforms[.type == 'docker'].target_image_source := 'https://github.com/viash-io/viash'\n", - "format" : "yaml" - } - ], - "since" : "Viash 0.6.4" - }, - { - "name" : "source", - "type" : "Option of String", - "description" : "Which source directory to use for the `viash ns` commands.", - "example" : [ - { - "example" : "source: src", - "format" : "yaml" - } - ] - }, - { - "name" : "viash_version", - "type" : "Option of String", - "description" : "Which version of Viash to use.", - "example" : [ - { - "example" : "viash_versions: 0.6.4", - "format" : "yaml" - } - ] - }, - { - "name" : "config_mods", - "type" : "OneOrMore of String", - "description" : "Which config mods to apply.", - "example" : [ - { - "example" : "config_mods: \".functionality.name := 'foo'\"", - "format" : "yaml" - } - ] - }, - { - "name" : "target", - "type" : "Option of String", - "description" : "Which target directory to use for `viash ns build`.", - "example" : [ - { - "example" : "target: target", - "format" : "yaml" - } - ] - } - ] - }, - "functionality" : { - "functionality" : [ - { - "name" : "__this__", - "type" : "Functionality", - "hierarchy" : [ - "io.viash.functionality.Functionality" - ], - "description" : "The functionality-part of the config file describes the behaviour of the script in terms of arguments and resources.\nBy specifying a few restrictions (e.g. mandatory arguments) and adding some descriptions, Viash will automatically generate a stylish command-line interface for you.\n" - }, - { - "name" : "name", - "type" : "String", - "description" : "Name of the component and the filename of the executable when built with `viash build`.", - "example" : [ - { - "example" : "name: this_is_my_component", - "format" : "yaml" - } - ] - }, - { - "name" : "enabled", - "type" : "Boolean", - "description" : "Setting this to false with disable this component when using namespaces.", - "since" : "Viash 0.5.13", - "removed" : { - "message" : "Use `status` instead.", - "deprecation" : "0.6.0", - "removal" : "0.7.0" +[ + [ + { + "name" : "__this__", + "type" : "Config", + "niceType" : "Config", + "hierarchy" : [ + "io.viash.config.Config" + ], + "description" : "A Viash configuration is a YAML file which contains metadata to describe the behaviour and build target(s) of a component. \nWe commonly name this file `config.vsh.yaml` in our examples, but you can name it however you choose. \n", + "example" : [ + { + "example" : "functionality:\n name: hello_world\n arguments:\n - type: string\n name: --input\n default: \"world\"\n resources:\n - type: bash_script\n path: script.sh\n text: echo Hello $par_input\nplatforms:\n - type: docker\n image: \"bash:4.0\"\n", + "format" : "yaml" } - }, - { - "name" : "tests", - "type" : "List of Resource", - "description" : "One or more Bash/R/Python scripts to be used to test the component behaviour when `viash test` is invoked. Additional files of type `file` will be made available only during testing. Each test script should expect no command-line inputs, be platform-independent, and return an exit code >0 when unexpected behaviour occurs during testing.", - "removed" : { - "message" : "Use `test_resources` instead. No functional difference.", - "deprecation" : "0.5.13", - "removal" : "0.7.0" + ] + }, + { + "name" : "functionality", + "type" : "Functionality", + "niceType" : "Functionality", + "description" : "The @[functionality](functionality) describes the behaviour of the script in terms of arguments and resources.\nBy specifying a few restrictions (e.g. mandatory arguments) and adding some descriptions, Viash will automatically generate a stylish command-line interface for you.\n" + }, + { + "name" : "platforms", + "type" : "List[Platform]", + "niceType" : "List of Platform", + "description" : "A list of platforms to generate target artifacts for.\n\n - @[Native](platform_native)\n - @[Docker](platform_docker)\n - @[Nextflow](platform_nextflow)\n" + }, + { + "name" : "__merge__", + "type" : "Option[File]", + "niceType" : "Option of File", + "description" : "Config inheritance by including YAML partials. This is useful for defining common APIs in\nseparate files. `__merge__` can be used in any level of the YAML. For example,\nnot just in the config but also in the functionality or any of the platforms.\n", + "example" : [ + { + "example" : "__merge__: ../api/common_interface.yaml", + "format" : "yaml" } - }, - { - "name" : "info", - "type" : "Json", - "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", - "example" : [ - { - "example" : "info:\n twitter: wizzkid\n classes: [ one, two, three ]", - "format" : "yaml" - } - ], - "since" : "Viash 0.4.0" - }, - { - "name" : "version", - "type" : "Option of String", - "description" : "Version of the component. This field will be used to version the executable and the Docker container.", - "example" : [ - { - "example" : "version: 0.8", - "format" : "yaml" - } - ] - }, - { - "name" : "inputs", - "type" : "List of Argument", - "description" : "A list of input arguments in addition to the `arguments` list. Any arguments specified here will have their `type` set to `file` and the `direction` set to `input` by default.", - "example" : [ - { - "example" : "inputs:\n - name: input_file\n - name: another_input", - "format" : "yaml" - }, - { - "example" : "component_with_inputs\n \n Inputs:\n input_file\n type: file\n \n another_input\n type: file", - "format" : "bash", - "description" : "This results in the following output when calling the component with the `--help` argument:" - } - ], - "since" : "Viash 0.5.11", - "removed" : { - "message" : "Use `arguments` instead.", - "deprecation" : "0.6.0", - "removal" : "0.7.0" + ], + "since" : "Viash 0.6.3", + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "Project", + "niceType" : "Project", + "hierarchy" : [ + "io.viash.project.ViashProject" + ], + "description" : "A Viash project configuration file. It's name should be `_viash.yaml`.", + "example" : [ + { + "example" : "viash_version: 0.6.4\nsource: src\ntarget: target\nconfig_mods: |\n .platforms[.type == 'docker'].target_registry := 'ghcr.io'\n .platforms[.type == 'docker'].target_organization := 'viash-io'\n .platforms[.type == 'docker'].namespace_separator := '/'\n .platforms[.type == 'docker'].target_image_source := 'https://github.com/viash-io/viash'\n", + "format" : "yaml" } - }, - { - "name" : "authors", - "type" : "List of Author", - "description" : "A list of @[authors](author). An author must at least have a name, but can also have a list of roles, an e-mail address, and a map of custom properties.\n\nSuggested values for roles are:\n \n| Role | Abbrev. | Description |\n|------|---------|-------------|\n| maintainer | mnt | for the maintainer of the code. Ideally, exactly one maintainer is specified. |\n| author | aut | for persons who have made substantial contributions to the software. |\n| contributor | ctb| for persons who have made smaller contributions (such as code patches).\n| datacontributor | dtc | for persons or organisations that contributed data sets for the software\n| copyrightholder | cph | for all copyright holders. This is a legal concept so should use the legal name of an institution or corporate body.\n| funder | fnd | for persons or organizations that furnished financial support for the development of the software\n\nThe [full list of roles](https://www.loc.gov/marc/relators/relaterm.html) is extremely comprehensive.\n", - "example" : [ - { - "example" : "authors:\n - name: Jane Doe\n role: [author, maintainer]\n email: jane@doe.com\n info:\n github: janedoe\n twitter: janedoe\n orcid: XXAABBCCXX\n groups: [ one, two, three ]\n - name: Tim Farbe\n roles: [author]\n email: tim@far.be\n", - "format" : "yaml" - } - ], - "since" : "Viash 0.3.1" - }, - { - "name" : "status", - "type" : "Status", - "description" : "Allows setting a component to active, deprecated or disabled.", - "since" : "Viash 0.6.0" - }, - { - "name" : "requirements", - "type" : "ComputationalRequirements", - "description" : "@[Computational requirements](computational_requirements) related to running the component. \n`cpus` specifies the maximum number of (logical) cpus a component is allowed to use., whereas\n`memory` specifies the maximum amount of memory a component is allowed to allicate. Memory units must be\nin B, KB, MB, GB, TB or PB.", - "example" : [ - { - "example" : "requirements:\n cpus: 5\n memory: 10GB\n", - "format" : "yaml" - } - ], - "since" : "Viash 0.6.0" - }, - { - "name" : "resources", - "type" : "List of Resource", - "description" : "@[Resources](resources) are files that support the component. The first resource should be @[a script](scripting_languages) that will be executed when the functionality is run. Additional resources will be copied to the same directory.\n\nCommon properties:\n\n * type: `file` / `r_script` / `python_script` / `bash_script` / `javascript_script` / `scala_script` / `csharp_script`, specifies the type of the resource. The first resource cannot be of type `file`. When the type is not specified, the default type is simply `file`.\n * dest: filename, the resulting name of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter.\n * path: `path/to/file`, the path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`.\n * text: ...multiline text..., the content of the resulting file specified as a string. Mutually exclusive with `path`.\n * is_executable: `true` / `false`, whether the resulting resource file should be made executable.\n", - "example" : [ - { - "example" : "resources:\n - type: r_script\n path: script.R\n - type: file\n path: resource1.txt\n", - "format" : "yaml" - } - ] - }, - { - "name" : "test_resources", - "type" : "List of Resource", - "description" : "One or more @[scripts](scripting_languages) to be used to test the component behaviour when `viash test` is invoked. Additional files of type `file` will be made available only during testing. Each test script should expect no command-line inputs, be platform-independent, and return an exit code >0 when unexpected behaviour occurs during testing. See @[Unit Testing](unit_testing) for more info.", - "example" : [ - { - "example" : "test_resources:\n - type: bash_script\n path: tests/test1.sh\n - type: r_script\n path: tests/test2.R\n - path: resource1.txt\n", - "format" : "yaml" - } - ] - }, - { - "name" : "argument_groups", - "type" : "List of ArgumentGroup", - "description" : "A grouping of the @[arguments](argument), used to display the help message.\n\n - `name: foo`, the name of the argument group. \n - `description: Description of foo`, a description of the argument group. Multiline descriptions are supported.\n - `arguments: [arg1, arg2, ...]`, list of the arguments names.\n\n", - "example" : [ - { - "example" : "argument_groups:\n - name: \"Input\"\n arguments:\n - name: \"--id\"\n type: string\n required: true\n - name: \"--input\"\n type: file\n required: true\n - name: \"Output\"\n arguments:\n - name: \"--output\"\n type: file\n direction: output\n required: true\n - name: \"--output_optional\"\n type: file\n direction: output\n", - "format" : "yaml" - }, - { - "example" : "component_name\n\n Input:\n --id\n type: string\n\n --input\n type: file\n\n Output:\n --output\n type: file\n\n --optional_output\n type: file\n", - "format" : "bash", - "description" : "This results in the following output when calling the component with the `--help` argument:" - } - ], - "since" : "Viash 0.5.14" - }, - { - "name" : "description", - "type" : "Option of String", - "description" : "A description of the component. This will be displayed with `--help`.", - "example" : [ - { - "example" : "description: |\n This component performs function Y and Z.\n It is possible to make this a multiline string.\n", - "format" : "yaml" - } - ] - }, - { - "name" : "usage", - "type" : "Option of String", - "description" : "A description on how to use the component. This will be displayed with `--help` under the 'Usage:' section.", - "example" : [ - { - "example" : "usage: Place the executable in a directory containing TSV files and run it", - "format" : "yaml" - } - ] - }, - { - "name" : "add_resources_to_path", - "type" : "Boolean", - "description" : "Adds the resources directory to the PATH variable when set to true. This is set to false by default.", - "since" : "Viash 0.5.5", - "removed" : { - "message" : "Extending the PATH turned out to be not desirable.", - "deprecation" : "", - "removal" : "0.5.11" + ], + "since" : "Viash 0.6.4" + }, + { + "name" : "source", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Which source directory to use for the `viash ns` commands.", + "example" : [ + { + "example" : "source: src", + "format" : "yaml" } - }, - { - "name" : "outputs", - "type" : "List of Argument", - "description" : "A list of output arguments in addition to the `arguments` list. Any arguments specified here will have their `type` set to `file` and thr `direction` set to `output` by default.", - "example" : [ - { - "example" : "outputs:\n - name: output_file\n - name: another_output", - "format" : "yaml" - }, - { - "example" : "component_with_outputs\n \n Outputs:\n output_file\n type: file, output\n \n another_output\n type: file, output", - "format" : "bash", - "description" : "This results in the following output when calling the component with the `--help` argument:" - } - ], - "since" : "Viash 0.5.11", - "removed" : { - "message" : "Use `arguments` instead.", - "deprecation" : "0.6.0", - "removal" : "0.7.0" + ], + "default" : "Empty" + }, + { + "name" : "viash_version", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Which version of Viash to use.", + "example" : [ + { + "example" : "viash_versions: 0.6.4", + "format" : "yaml" } - }, - { - "name" : "namespace", - "type" : "Option of String", - "description" : "Namespace this component is a part of. See the @[Namespaces guide](namespace) for more information on namespaces.", - "example" : [ - { - "example" : "namespace: fancy_components", - "format" : "yaml" - } - ] - }, - { - "name" : "arguments", - "type" : "List of Argument", - "description" : "A list of @[arguments](argument) for this component. For each argument, a type and a name must be specified. Depending on the type of argument, different properties can be set. See these reference pages per type for more information: \n\n - @[string](arg_string)\n - @[file](arg_file)\n - @[integer](arg_integer)\n - @[double](arg_double)\n - @[boolean](arg_boolean)\n - @[boolean_true](arg_boolean_true)\n - @[boolean_false](arg_boolean_false)\n", - "example" : [ - { - "example" : "arguments:\n - name: --foo\n type: file\n alternatives: [-f]\n description: Description of foo\n default: \"/foo/bar\"\n must_exist: true\n direction: output\n required: false\n multiple: true\n multiple_sep: \",\"\n - name: --bar\n type: string\n", - "format" : "yaml" - } - ] - } - ], - "author" : [ - { - "name" : "__this__", - "type" : "Author", - "hierarchy" : [ - "io.viash.functionality.Author" - ], - "description" : "Author metadata.", - "example" : [ - { - "example" : "name: Jane Doe\nrole: [author, maintainer]\nemail: jane@doe.com\ninfo:\n github: janedoe\n twitter: janedoe\n orcid: XXAABBCCXX\n groups: [ one, two, three ]\n", - "format" : "yaml" - } - ], - "since" : "Viash 0.3.2" - }, - { - "name" : "name", - "type" : "String", - "description" : "Full name of the author, usually in the name of FirstName MiddleName LastName." - }, - { - "name" : "email", - "type" : "Option of String", - "description" : "E-mail of the author." - }, - { - "name" : "info", - "type" : "Json", - "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", - "since" : "Viash 0.7.4" - }, - { - "name" : "roles", - "type" : "OneOrMore of String", - "description" : "Role of the author. Suggested items:\n\n* `\"author\"`: Authors who have made substantial contributions to the component.\n* `\"maintainer\"`: The maintainer of the component.\n* `\"contributor\"`: Authors who have made smaller contributions (such as code patches etc.).\n" - }, - { - "name" : "props", - "type" : "Map of String,String", - "description" : "Author properties. Must be a map of strings.", - "deprecated" : { - "message" : "Use `info` instead.", - "deprecation" : "0.7.4", - "removal" : "0.8.0" - } - } - ], - "computationalRequirements" : [ - { - "name" : "__this__", - "type" : "ComputationalRequirements", - "hierarchy" : [ - "io.viash.functionality.ComputationalRequirements" - ], - "description" : "Computational requirements related to running the component.", - "since" : "Viash 0.6.0" - }, - { - "name" : "n_proc", - "type" : "Option of Int", - "removed" : { - "message" : "Use `cpus` instead.", - "deprecation" : "0.6.1", - "removal" : "0.7.0" + ], + "default" : "Empty" + }, + { + "name" : "config_mods", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Which config mods to apply.", + "example" : [ + { + "example" : "config_mods: \".functionality.name := 'foo'\"", + "format" : "yaml" } - }, - { - "name" : "cpus", - "type" : "Option of Int", - "description" : "The maximum number of (logical) cpus a component is allowed to use.", - "example" : [ - { - "example" : "cpus: 10", - "format" : "yaml" - } - ] - }, - { - "name" : "commands", - "type" : "List of String", - "description" : "A list of commands which should be present on the system for the script to function.", - "example" : [ - { - "example" : "commands: [ which, bash, awk, date, grep, egrep, ps, sed, tail, tee ]", - "format" : "yaml" - } - ] - }, - { - "name" : "memory", - "type" : "Option of String", - "description" : "The maximum amount of memory a component is allowed to allocate. Unit must be one of B, KB, MB, GB, TB or PB.", - "example" : [ - { - "example" : "memory: 10GB", - "format" : "yaml" - } - ] - } - ] - }, - "platforms" : { - "nextflowVdsl3Platform" : [ - { - "name" : "__this__", - "type" : "NextflowVdsl3Platform", - "hierarchy" : [ - "io.viash.platforms.NextflowVdsl3Platform", - "io.viash.platforms.NextflowPlatform", - "io.viash.platforms.Platform" - ], - "description" : "Next-gen platform for generating NextFlow VDSL3 modules.", - "example" : [ - { - "example" : "platforms:\n - type: nextflow\n directives:\n label: [lowcpu, midmem]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "auto", - "type" : "NextflowAuto", - "description" : "@[Automated processing flags](nextflow_auto) which can be toggled on or off:\n\n| Flag | Description | Default |\n|---|---------|----|\n| `simplifyInput` | If `true`, an input tuple only containing only a single File (e.g. `[\"foo\", file(\"in.h5ad\")]`) is automatically transformed to a map (i.e. `[\"foo\", [ input: file(\"in.h5ad\") ] ]`). | `true` |\n| `simplifyOutput` | If `true`, an output tuple containing a map with a File (e.g. `[\"foo\", [ output: file(\"out.h5ad\") ] ]`) is automatically transformed to a map (i.e. `[\"foo\", file(\"out.h5ad\")]`). | `true` |\n| `transcript` | If `true`, the module's transcripts from `work/` are automatically published to `params.transcriptDir`. If not defined, `params.publishDir + \"/_transcripts\"` will be used. Will throw an error if neither are defined. | `false` |\n| `publish` | If `true`, the module's outputs are automatically published to `params.publishDir`. Will throw an error if `params.publishDir` is not defined. | `false` |\n\n", - "example" : [ - { - "example" : "auto:\n publish: true", - "format" : "yaml" - } - ] - }, - { - "name" : "directives", - "type" : "NextflowDirectives", - "description" : "@[Directives](nextflow_directives) are optional settings that affect the execution of the process. These mostly match up with the Nextflow counterparts. \n", - "example" : [ - { - "example" : "directives:\n container: rocker/r-ver:4.1\n label: highcpu\n cpus: 4\n memory: 16 GB", - "format" : "yaml" - } - ] - }, - { - "name" : "container", - "type" : "String", - "description" : "Specifies the Docker platform id to be used to run Nextflow." - }, - { - "name" : "debug", - "type" : "Boolean", - "description" : "Whether or not to print debug messages." - }, - { - "name" : "id", - "type" : "String", - "description" : "Every platform can be given a specific id that can later be referred to explicitly when running or building the Viash component.", - "example" : [ - { - "example" : "id: foo", - "format" : "yaml" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the platform." - }, - { - "name" : "config", - "type" : "NextflowConfig", - "description" : "Allows tweaking how the @[Nextflow Config](nextflow_config) file is generated.", - "since" : "Viash 0.7.4" - } - ], - "dockerPlatform" : [ - { - "name" : "__this__", - "type" : "DockerPlatform", - "hierarchy" : [ - "io.viash.platforms.DockerPlatform", - "io.viash.platforms.Platform" - ], - "description" : "Run a Viash component on a Docker backend platform.\nBy specifying which dependencies your component needs, users will be able to build a docker container from scratch using the setup flag, or pull it from a docker repository.\n", - "example" : [ - { - "example" : "platforms:\n - type: docker\n image: \"bash:4.0\"\n setup:\n - type: apt\n packages: [ curl ]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "organization", - "type" : "Option of String", - "description" : "Name of a container's [organization](https://docs.docker.com/docker-hub/orgs/)." - }, - { - "name" : "registry", - "type" : "Option of String", - "description" : "The URL to the a [custom Docker registry](https://docs.docker.com/registry/)", - "example" : [ - { - "example" : "registry: https://my-docker-registry.org", - "format" : "yaml" - } - ] - }, - { - "name" : "image", - "type" : "String", - "description" : "The base container to start from. You can also add the tag here if you wish.", - "example" : [ - { - "example" : "image: \"bash:4.0\"", - "format" : "yaml" - } - ] - }, - { - "name" : "tag", - "type" : "Option of String", - "description" : "Specify a Docker image based on its tag.", - "example" : [ - { - "example" : "tag: 4.0", - "format" : "yaml" - } - ] - }, - { - "name" : "target_tag", - "type" : "Option of String", - "description" : "The tag the resulting image gets. Advanced usage only.", - "example" : [ - { - "example" : "target_tag: 0.5.0", - "format" : "yaml" - } - ] - }, - { - "name" : "run_args", - "type" : "OneOrMore of String", - "description" : "Add [docker run](https://docs.docker.com/engine/reference/run/) arguments." - }, - { - "name" : "namespace_separator", - "type" : "String", - "description" : "The separator between the namespace and the name of the component, used for determining the image name. Default: `\"/\"`.", - "example" : [ - { - "example" : "namespace_separator: \"_\"", - "format" : "yaml" - } - ] - }, - { - "name" : "resolve_volume", - "type" : "DockerResolveVolume", - "description" : "Enables or disables automatic volume mapping. Enabled when set to `Automatic` or disabled when set to `Manual`. Default: `Automatic`." - }, - { - "name" : "port", - "type" : "OneOrMore of String", - "description" : "A list of enabled ports. This doesn't change the Dockerfile but gets added as a command-line argument at runtime.", - "example" : [ - { - "example" : "port:\n - 80\n - 8080\n", - "format" : "yaml" - } - ] - }, - { - "name" : "python", - "type" : "Option of PythonRequirements", - "description" : "Specify which Python packages should be available in order to run the component.", - "example" : [ - { - "example" : "setup:\n - type: python\n pip: [ numpy ]\n git: [ https://some.git.repository/org/repo ]\n github: [ jkbr/httpie ]\n gitlab: [ foo/bar ]\n mercurial: [ http://... ]\n svn: [ http://...]\n bazaar: [ http://... ]\n url: [ http://... ]\n", - "format" : "yaml" - } - ], - "removed" : { - "message" : "Use `setup` instead, e.g. `{type: docker, setup: [{ type: python, ... }]}`. Will be removed.", - "deprecation" : "0.5.15", - "removal" : "0.7.0" + ], + "default" : "Empty" + }, + { + "name" : "target", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Which target directory to use for `viash ns build`.", + "example" : [ + { + "example" : "target: target", + "format" : "yaml" } - }, - { - "name" : "setup", - "type" : "List of Requirements", - "description" : "A list of requirements for installing the following types of packages:\n\n - @[apt](apt_req)\n - @[apk](apk_req)\n - @[Docker setup instructions](docker_req)\n - @[JavaScript](javascript_req)\n - @[Python](python_req)\n - @[R](r_req)\n - @[Ruby](ruby_req)\n - @[yum](yum_req)\n\nThe order in which these dependencies are specified determines the order in which they will be installed.\n" - }, - { - "name" : "workdir", - "type" : "Option of String", - "description" : "The working directory when starting the container. This doesn't change the Dockerfile but gets added as a command-line argument at runtime.", - "example" : [ - { - "example" : "workdir: /home/user", - "format" : "yaml" - } - ] - }, - { - "name" : "apk", - "type" : "Option of ApkRequirements", - "description" : "Specify which apk packages should be available in order to run the component.", - "example" : [ - { - "example" : "setup:\n - type: apk\n packages: [ sl ]\n", - "format" : "yaml" - } - ], - "removed" : { - "message" : "Use `setup` instead, e.g. `{type: docker, setup: [{ type: apk, ... }]}`. Will be removed.", - "deprecation" : "0.5.15", - "removal" : "0.7.0" + ], + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "Info", + "niceType" : "Info", + "hierarchy" : [ + "io.viash.config.Info" + ], + "description" : "Meta information fields filled in by Viash during build." + }, + { + "name" : "git_tag", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Git tag.", + "default" : "Empty" + }, + { + "name" : "git_remote", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Git remote name.", + "default" : "Empty" + }, + { + "name" : "viash_version", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The Viash version that was used to build the component.", + "default" : "Empty" + }, + { + "name" : "config", + "type" : "String", + "niceType" : "String", + "description" : "Path to the config used during build." + }, + { + "name" : "output", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Folder path to the build artifacts.", + "default" : "Empty" + }, + { + "name" : "platform", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The platform id used during build.", + "default" : "Empty" + }, + { + "name" : "git_commit", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Git commit hash.", + "default" : "Empty" + }, + { + "name" : "executable", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Output folder with main executable path.", + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "EnvironmentVariables", + "niceType" : "EnvironmentVariables", + "hierarchy" : [ + "io.viash.helpers.SysEnvTrait" + ], + "description" : "Viash checks several environment variables during operation." + }, + { + "name" : "VIASH_VERSION", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "A specific Viash version can be set to run the commands with. If so required, the specific Viash version will be downloaded.\nThis is useful when replicating older results or building Viash components that use outdated code.\n", + "example" : [ + { + "example" : "VIASH_VERSION=0.7.0 viash ns build", + "format" : "sh" } - }, - { - "name" : "target_image", - "type" : "Option of String", - "description" : "If anything is specified in the setup section, running the `---setup` will result in an image with the name of `:`. If nothing is specified in the `setup` section, simply `image` will be used. Advanced usage only.", - "example" : [ - { - "example" : "target_image: myfoo", - "format" : "yaml" - } - ] - }, - { - "name" : "cmd", - "type" : "Option[Either[String,List of String]]", - "description" : "Set the default command being executed when running the Docker container.", - "example" : [ - { - "example" : "cmd: [\"echo\", \"$HOME\"]", - "format" : "yaml", - "description" : "Set CMD using the exec format, which is the prefered form." - }, - { - "example" : "cmd: \"echo $HOME\"", - "format" : "yaml", - "description" : "Set CMD using the shell format." - } - ], - "since" : "Viash 0.7.4" - }, - { - "name" : "yum", - "type" : "Option of YumRequirements", - "description" : "Specify which yum packages should be available in order to run the component.", - "example" : [ - { - "example" : "setup:\n - type: yum\n packages: [ sl ]\n", - "format" : "yaml" - } - ], - "removed" : { - "message" : "Use `setup` instead, e.g. `{type: docker, setup: [{ type: yum, ... }]}`. Will be removed.", - "deprecation" : "0.5.15", - "removal" : "0.7.0" + ], + "default" : "Empty" + }, + { + "name" : "VIASH_HOME", + "type" : "String", + "niceType" : "String", + "description" : "If `VIASH_HOME` is not defined, the fallback `HOME`/.viash is used.\n\nLocation where specific downloaded versions of Viash will be cached and run from.\n" + } + ], + [ + { + "name" : "__this__", + "type" : "Functionality", + "niceType" : "Functionality", + "hierarchy" : [ + "io.viash.functionality.Functionality" + ], + "description" : "The functionality-part of the config file describes the behaviour of the script in terms of arguments and resources.\nBy specifying a few restrictions (e.g. mandatory arguments) and adding some descriptions, Viash will automatically generate a stylish command-line interface for you.\n" + }, + { + "name" : "name", + "type" : "String", + "niceType" : "String", + "description" : "Name of the component and the filename of the executable when built with `viash build`.", + "example" : [ + { + "example" : "name: this_is_my_component", + "format" : "yaml" } - }, - { - "name" : "target_image_source", - "type" : "Option of String", - "description" : "The source of the target image. This is used for defining labels in the dockerfile.", - "example" : [ - { - "example" : "target_image_source: https://github.com/foo/bar", - "format" : "yaml" - } - ] - }, - { - "name" : "test_setup", - "type" : "List of Requirements", - "description" : "Additional requirements specific for running unit tests.", - "since" : "Viash 0.5.13" - }, - { - "name" : "entrypoint", - "type" : "Option[Either[String,List of String]]", - "description" : "Override the entrypoint of the base container. Default set `ENTRYPOINT []`.", - "example" : [ - { - "example" : "entrypoint: ", - "format" : "yaml", - "description" : "Disable the default override." - }, - { - "example" : "entrypoint: [\"top\", \"-b\"]", - "format" : "yaml", - "description" : "Entrypoint of the container in the exec format, which is the prefered form." - }, - { - "example" : "entrypoint: \"top -b\"", - "format" : "yaml", - "description" : "Entrypoint of the container in the shell format." - } - ], - "since" : "Viash 0.7.4" - }, - { - "name" : "docker", - "type" : "Option of DockerRequirements", - "description" : "Specify which Docker commands should be run during setup.", - "example" : [ - { - "example" : "setup:\n - type: docker\n build_args: [ GITHUB_PAT=hello_world ]\n run: [ git clone ... ]\n add: [ \"http://foo.bar .\" ]\n copy: [ \"http://foo.bar .\" ]\n resources: \n - resource.txt /path/to/resource.txt\n", - "format" : "yaml" - } - ], - "removed" : { - "message" : "Use `setup` instead, e.g. `{type: docker, setup: [{ type: docker, ... }]}`. Will be removed.", - "deprecation" : "0.5.15", - "removal" : "0.7.0" + ] + }, + { + "name" : "info", + "type" : "Json", + "niceType" : "Json", + "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", + "example" : [ + { + "example" : "info:\n twitter: wizzkid\n classes: [ one, two, three ]", + "format" : "yaml" } - }, - { - "name" : "id", - "type" : "String", - "description" : "As with all platforms, you can give a platform a different name. By specifying `id: foo`, you can target this platform (only) by specifying `-p foo` in any of the Viash commands.", - "example" : [ - { - "example" : "id: foo", - "format" : "yaml" - } - ] - }, - { - "name" : "apt", - "type" : "Option of AptRequirements", - "description" : "Specify which apt packages should be available in order to run the component.", - "example" : [ - { - "example" : "setup:\n - type: apt\n packages: [ sl ]\n", - "format" : "yaml" - } - ], - "removed" : { - "message" : "Use `setup` instead, e.g. `{type: docker, setup: [{ type: apt, ... }]}`. Will be removed.", - "deprecation" : "0.5.15", - "removal" : "0.7.0" + ], + "since" : "Viash 0.4.0", + "default" : "Empty" + }, + { + "name" : "version", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Version of the component. This field will be used to version the executable and the Docker container.", + "example" : [ + { + "example" : "version: 0.8", + "format" : "yaml" } - }, - { - "name" : "target_registry", - "type" : "Option of String", - "description" : "The URL where the resulting image will be pushed to. Advanced usage only.", - "example" : [ - { - "example" : "target_registry: https://my-docker-registry.org", - "format" : "yaml" - } - ] - }, - { - "name" : "privileged", - "type" : "Option of Boolean", - "description" : "Adds a `privileged` flag to the docker run.", - "removed" : { - "message" : "Add a `privileged` flag in `run_args` instead, e.g. `{type: docker, run_args: \"--privileged\"}`.", - "deprecation" : "0.6.3", - "removal" : "0.7.0" + ], + "default" : "Empty" + }, + { + "name" : "authors", + "type" : "List[Author]", + "niceType" : "List of Author", + "description" : "A list of @[authors](author). An author must at least have a name, but can also have a list of roles, an e-mail address, and a map of custom properties.\n\nSuggested values for roles are:\n \n| Role | Abbrev. | Description |\n|------|---------|-------------|\n| maintainer | mnt | for the maintainer of the code. Ideally, exactly one maintainer is specified. |\n| author | aut | for persons who have made substantial contributions to the software. |\n| contributor | ctb| for persons who have made smaller contributions (such as code patches).\n| datacontributor | dtc | for persons or organisations that contributed data sets for the software\n| copyrightholder | cph | for all copyright holders. This is a legal concept so should use the legal name of an institution or corporate body.\n| funder | fnd | for persons or organizations that furnished financial support for the development of the software\n\nThe [full list of roles](https://www.loc.gov/marc/relators/relaterm.html) is extremely comprehensive.\n", + "example" : [ + { + "example" : "authors:\n - name: Jane Doe\n role: [author, maintainer]\n email: jane@doe.com\n info:\n github: janedoe\n twitter: janedoe\n orcid: XXAABBCCXX\n groups: [ one, two, three ]\n - name: Tim Farbe\n roles: [author]\n email: tim@far.be\n", + "format" : "yaml" } - }, - { - "name" : "setup_strategy", - "type" : "DockerSetupStrategy", - "description" : "The Docker setup strategy to use when building a container.\n\n| Strategy | Description |\n|-----|----------|\n| `alwaysbuild` / `build` / `b` | Always build the image from the dockerfile. This is the default setup strategy.\n| `alwayscachedbuild` / `cachedbuild` / `cb` | Always build the image from the dockerfile, with caching enabled.\n| `ifneedbebuild` | Build the image if it does not exist locally.\n| `ifneedbecachedbuild` | Build the image with caching enabled if it does not exist locally, with caching enabled.\n| `alwayspull` / `pull` / `p` | Try to pull the container from [Docker Hub](https://hub.docker.com) or the @[specified docker registry](docker_registry).\n| `alwayspullelsebuild` / `pullelsebuild` | Try to pull the image from a registry and build it if it doesn't exist.\n| `alwayspullelsecachedbuild` / `pullelsecachedbuild` | Try to pull the image from a registry and build it with caching if it doesn't exist.\n| `ifneedbepull` | If the image does not exist locally, pull the image.\n| `ifneedbepullelsebuild` | If the image does not exist locally, pull the image. If the image does exist, build it.\n| `ifneedbepullelsecachedbuild` | If the image does not exist locally, pull the image. If the image does exist, build it with caching enabled.\n| `push` | Push the container to [Docker Hub](https://hub.docker.com) or the @[specified docker registry](docker_registry).\n| `pushifnotpresent` | Push the container to [Docker Hub](https://hub.docker.com) or the @[specified docker registry](docker_registry) if the @[tag](docker_tag) does not exist yet.\n| `donothing` / `meh` | Do not build or pull anything.\n\n", - "example" : [ - { - "example" : "setup_strategy: alwaysbuild", - "format" : "yaml" - } - ] - }, - { - "name" : "r", - "type" : "Option of RRequirements", - "description" : "Specify which R packages should be available in order to run the component.", - "example" : [ - { - "example" : "setup: \n - type: r\n cran: [ dynutils ]\n bioc: [ AnnotationDbi ]\n git: [ https://some.git.repository/org/repo ]\n github: [ rcannood/SCORPIUS ]\n gitlab: [ org/package ]\n svn: [ https://path.to.svn/group/repo ]\n url: [ https://github.com/hadley/stringr/archive/HEAD.zip ]\n script: [ 'devtools::install(\".\")' ]\n", - "format" : "yaml" - } - ], - "removed" : { - "message" : "Use `setup` instead, e.g. `{type: docker, setup: [{ type: r, ... }]}`. Will be removed.", - "deprecation" : "0.5.15", - "removal" : "0.7.0" + ], + "since" : "Viash 0.3.1", + "default" : "Empty" + }, + { + "name" : "status", + "type" : "Status", + "niceType" : "Status", + "description" : "Allows setting a component to active, deprecated or disabled.", + "since" : "Viash 0.6.0", + "default" : "Enabled" + }, + { + "name" : "requirements", + "type" : "ComputationalRequirements", + "niceType" : "ComputationalRequirements", + "description" : "@[Computational requirements](computational_requirements) related to running the component. \n`cpus` specifies the maximum number of (logical) cpus a component is allowed to use., whereas\n`memory` specifies the maximum amount of memory a component is allowed to allicate. Memory units must be\nin B, KB, MB, GB, TB or PB.", + "example" : [ + { + "example" : "requirements:\n cpus: 5\n memory: 10GB\n", + "format" : "yaml" } - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the platform." - }, - { - "name" : "target_organization", - "type" : "Option of String", - "description" : "The organization set in the resulting image. Advanced usage only.", - "example" : [ - { - "example" : "target_organization: viash-io", - "format" : "yaml" - } - ] - }, - { - "name" : "chown", - "type" : "Boolean", - "description" : "In Linux, files created by a Docker container will be owned by `root`. With `chown: true`, Viash will automatically change the ownership of output files (arguments with `type: file` and `direction: output`) to the user running the Viash command after execution of the component. Default value: `true`.", - "example" : [ - { - "example" : "chown: false", - "format" : "yaml" - } - ] - } - ], - "nativePlatform" : [ - { - "name" : "__this__", - "type" : "NativePlatform", - "hierarchy" : [ - "io.viash.platforms.NativePlatform", - "io.viash.platforms.Platform" - ], - "description" : "Running a Viash component on a native platform means that the script will be executed in your current environment.\nAny dependencies are assumed to have been installed by the user, so the native platform is meant for developers (who know what they're doing) or for simple bash scripts (which have no extra dependencies).\n", - "example" : [ - { - "example" : "platforms:\n - type: native\n", - "format" : "yaml" - } - ] - }, - { - "name" : "id", - "type" : "String", - "description" : "As with all platforms, you can give a platform a different name. By specifying `id: foo`, you can target this platform (only) by specifying `-p foo` in any of the Viash commands.", - "example" : [ - { - "example" : "id: foo", - "format" : "yaml" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the platform." - } - ], - "nextflowLegacyPlatform" : [ - { - "name" : "__this__", - "type" : "NextflowLegacyPlatform", - "hierarchy" : [ - "io.viash.platforms.NextflowLegacyPlatform", - "io.viash.platforms.NextflowPlatform", - "io.viash.platforms.Platform" - ], - "description" : "Run a Viash component as a Nextflow module.", - "removed" : { - "message" : "Nextflow platform with `variant: legacy` was removed", - "deprecation" : "0.6.0", - "removal" : "0.7.0" + ], + "since" : "Viash 0.6.0", + "default" : "Empty" + }, + { + "name" : "resources", + "type" : "List[Resource]", + "niceType" : "List of Resource", + "description" : "@[Resources](resources) are files that support the component. The first resource should be @[a script](scripting_languages) that will be executed when the functionality is run. Additional resources will be copied to the same directory.\n\nCommon properties:\n\n * type: `file` / `r_script` / `python_script` / `bash_script` / `javascript_script` / `scala_script` / `csharp_script`, specifies the type of the resource. The first resource cannot be of type `file`. When the type is not specified, the default type is simply `file`.\n * dest: filename, the resulting name of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter.\n * path: `path/to/file`, the path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`.\n * text: ...multiline text..., the content of the resulting file specified as a string. Mutually exclusive with `path`.\n * is_executable: `true` / `false`, whether the resulting resource file should be made executable.\n", + "example" : [ + { + "example" : "resources:\n - type: r_script\n path: script.R\n - type: file\n path: resource1.txt\n", + "format" : "yaml" } - }, - { - "name" : "organization", - "type" : "Option of String", - "description" : "Name of a container's [organization](https://docs.docker.com/docker-hub/orgs/).", - "example" : [ - { - "example" : "organization: viash-io", - "format" : "yaml" - } - ] - }, - { - "name" : "image", - "type" : "Option of String", - "description" : "If no image attributes are configured, Viash will use the auto-generated image name from the Docker platform:\n\n```\n[/]:\n```\nIt's possible to specify the container image explicitly with which to run the module in different ways:\n\n```\nimage: dataintuitive/viash:0.4.0\n```\nExactly the same can be obtained with\n\n```\nimage: dataintuitive/viash\nregistry: index.docker.io/v1/\ntag: 0.4.0\n```\nSpecifying the attribute(s) like this will use the container `dataintuitive/viash:0.4.0` from Docker hub (registry).\n\nIf no tag is specified Viash will use `functionality.version` as the tag.\n\nIf no registry is specified, Viash (and NextFlow) will assume the image is available locally or on Docker Hub. In other words, the `registry: ...` attribute above is superfluous. No other registry is checked automatically due to a limitation from Docker itself.\n" - }, - { - "name" : "tag", - "type" : "Option of String", - "description" : "Specify a Docker image based on its tag.", - "example" : [ - { - "example" : "tag: 4.0", - "format" : "yaml" - } - ] - }, - { - "name" : "label", - "type" : "Option of String", - "description" : "When running the module in a cluster context and depending on the cluster type, [NextFlow allows for attaching labels](https://www.nextflow.io/docs/latest/process.html#label) to the process that can later be used as selectors for associating resources to this process.\n\nIn order to attach one label to a process/component, one can use the `label: ...` attribute, multiple labels can be added using `labels: [ ..., ... ]` and the two can even be mixed.\n\nIn the main `nextflow.config`, one can now use this label:\n\nprocess {\n ...\n withLabel: bigmem {\n maxForks = 5\n ...\n }\n}\n", - "example" : [ - { - "example" : "label: highmem labels: [ highmem, highcpu ]", - "format" : "yaml" - } - ] - }, - { - "name" : "stageInMode", - "type" : "Option of String", - "description" : "By default NextFlow will create a symbolic link to the inputs for a process/module and run the tool at hand using those symbolic links. Some applications do not cope well with this strategy, in that case the files should effectively be copied rather than linked to. This can be achieved by using `stageInMode: copy`.\nThis attribute is optional, the default is `symlink`.\n", - "example" : [ - { - "example" : "stageInMode: copy", - "format" : "yaml" - } - ] - }, - { - "name" : "id", - "type" : "String", - "description" : "Every platform can be given a specific id that can later be referred to explicitly when running or building the Viash component." - }, - { - "name" : "labels", - "type" : "OneOrMore of String", - "description" : "When running the module in a cluster context and depending on the cluster type, [NextFlow allows for attaching labels](https://www.nextflow.io/docs/latest/process.html#label) to the process that can later be used as selectors for associating resources to this process.\n\nIn order to attach one label to a process/component, one can use the `label: ...` attribute, multiple labels can be added using `labels: [ ..., ... ]` and the two can even be mixed.\n\nIn the main `nextflow.config`, one can now use this label:\n\nprocess {\n ...\n withLabel: bigmem {\n maxForks = 5\n ...\n }\n}\n", - "example" : [ - { - "example" : "label: highmem labels: [ highmem, highcpu ]", - "format" : "yaml" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the platform." - }, - { - "name" : "registry", - "type" : "Option of String", - "description" : "The URL to the a [custom Docker registry](https://docs.docker.com/registry/).", - "example" : [ - { - "example" : "registry: https://my-docker-registry.org", - "format" : "yaml" - } - ] - }, - { - "name" : "per_id", - "type" : "Option of Boolean", - "description" : "By default, a subdirectory is created corresponding to the unique ID that is passed in the triplet. Let us illustrate this with an example. The following code snippet uses the value of `--input` as an input of a workflow. The input can include a wildcard so that multiple samples can run in parallel. We use the parent directory name (`.getParent().baseName`) as an identifier for the sample. We pass this as the first entry of the triplet:\n\n```\nChannel.fromPath(params.input) \\\n | map{ it -> [ it.getParent().baseName , it ] } \\\n | map{ it -> [ it[0] , it[1], params ] }\n | ...\n```\nSay the resulting sample names are `SAMPLE1` and `SAMPLE2`. The next step in the pipeline will be published (at least by default) under:\n```\n/SAMPLE1/\n/SAMPLE2/\n```\nThese per-ID subdirectories can be avoided by setting:\n```\nper_id: false\n```\n" - }, - { - "name" : "path", - "type" : "Option of String", - "description" : "When `publish: true`, this attribute defines where the output is written relative to the `params.publishDir` setting. For example, `path: processed` in combination with `--output s3://some_bucket/` will store the output of this component under\n```\ns3://some_bucket/processed/\n```\nThis attribute gives control over the directory structure of the output. For example:\n```\npath: raw_data\n```\nOr even:\n```\npath: raw_data/bcl\n```\nPlease note that `per_id` and `path` can be combined.\n" - }, - { - "name" : "separate_multiple_outputs", - "type" : "Boolean", - "description" : "Separates the outputs generated by a Nextflow component with multiple outputs as separate events on the channel. Default value: `true`.", - "example" : [ - { - "example" : "separate_multiple_outputs: false", - "format" : "yaml" - } - ] - }, - { - "name" : "namespace_separator", - "type" : "String", - "description" : "The default namespace separator is \"_\".", - "example" : [ - { - "example" : "namespace_separator: \"+\"", - "format" : "yaml" - } - ] - }, - { - "name" : "publish", - "type" : "Option of Boolean", - "description" : "NextFlow uses the autogenerated `work` dirs to manage process IO under the hood. In order effectively output something one can publish the results a module or step in the pipeline. In order to do this, add `publish: true` to the config:\n\n - publish is optional\n - Default value is false\n\nThis attribute simply defines if output of a component should be published yes or no. The output location has to be provided at pipeline launch by means of the option `--publishDir ...` or as `params.publishDir` in `nextflow.config`:\n```\nparams.publishDir = \"...\"\n```\n" - }, - { - "name" : "version", - "type" : "Option of String", - "removed" : { - "message" : "nextflow platform: attribute 'version' was removed", - "deprecation" : "0.4.0", - "removal" : "0.7.0" + ], + "default" : "Empty" + }, + { + "name" : "test_resources", + "type" : "List[Resource]", + "niceType" : "List of Resource", + "description" : "One or more @[scripts](scripting_languages) to be used to test the component behaviour when `viash test` is invoked. Additional files of type `file` will be made available only during testing. Each test script should expect no command-line inputs, be platform-independent, and return an exit code >0 when unexpected behaviour occurs during testing. See @[Unit Testing](unit_testing) for more info.", + "example" : [ + { + "example" : "test_resources:\n - type: bash_script\n path: tests/test1.sh\n - type: r_script\n path: tests/test2.R\n - path: resource1.txt\n", + "format" : "yaml" } - }, - { - "name" : "executor", - "type" : "Option of String", - "removed" : { - "message" : "Undocumented & stale value", - "deprecation" : "0.6.3", - "removal" : "0.7.0" - } - } - ], - "platform" : [ - { - "name" : "__this__", - "type" : "Platform", - "hierarchy" : [ - "io.viash.platforms.Platform" - ], - "description" : "A list of platforms to generate target artifacts for.\n\n * @[Native](platform_native)\n * @[Docker](platform_docker)\n * @[Nextflow VDSL3](platform_nextflow)\n", - "example" : [ - { - "example" : "platforms:\n - type: docker\n image: \"bash:4.0\"\n - type: native\n - type: nextflow\n directives:\n label: [lowcpu, midmem]\n", - "format" : "yaml" - } - ] - } - ] - }, - "requirements" : { - "javascriptRequirements" : [ - { - "name" : "__this__", - "type" : "JavaScriptRequirements", - "hierarchy" : [ - "io.viash.platforms.requirements.JavaScriptRequirements", - "io.viash.platforms.requirements.Requirements" - ], - "description" : "Specify which JavaScript packages should be available in order to run the component.", - "example" : [ - { - "example" : "setup:\n - type: javascript\n npm: packagename\n git: \"https://some.git.repository/org/repo\"\n github: \"owner/repository\"\n url: \"https://github.com/org/repo/archive/HEAD.zip\"\n", - "format" : "yaml" - } - ] - }, - { - "name" : "github", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install from GitHub.", - "example" : [ - { - "example" : "github: [ owner/repository ]", - "format" : "yaml" - } - ] - }, - { - "name" : "url", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install using a generic URI.", - "example" : [ - { - "example" : "url: [ https://github.com/org/repo/archive/HEAD.zip ]", - "format" : "yaml" - } - ] - }, - { - "name" : "git", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install using a Git URI.", - "example" : [ - { - "example" : "git: [ https://some.git.repository/org/repo ]", - "format" : "yaml" - } - ] - }, - { - "name" : "npm", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install from npm.", - "example" : [ - { - "example" : "npm: [ packagename ]", - "format" : "yaml" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the requirement specification." - }, - { - "name" : "packages", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install from npm.", - "example" : [ - { - "example" : "packages: [ packagename ]", - "format" : "yaml" - } - ] - } - ], - "pythonRequirements" : [ - { - "name" : "__this__", - "type" : "PythonRequirements", - "hierarchy" : [ - "io.viash.platforms.requirements.PythonRequirements", - "io.viash.platforms.requirements.Requirements" - ], - "description" : "Specify which Python packages should be available in order to run the component.", - "example" : [ - { - "example" : "setup:\n - type: python\n pip: numpy\n github: [ jkbr/httpie, foo/bar ]\n url: \"https://github.com/some_org/some_pkg/zipball/master\"\n", - "format" : "yaml" - } - ] - }, - { - "name" : "github", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install from GitHub.", - "example" : [ - { - "example" : "github: [ jkbr/httpie ]", - "format" : "yaml" - } - ] - }, - { - "name" : "gitlab", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install from GitLab.", - "example" : [ - { - "example" : "gitlab: [ foo/bar ]", - "format" : "yaml" - } - ] - }, - { - "name" : "pip", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install from pip.", - "example" : [ - { - "example" : "pip: [ numpy ]", - "format" : "yaml" - } - ] - }, - { - "name" : "pypi", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install from PyPI using pip.", - "example" : [ - { - "example" : "pypi: [ numpy ]", - "format" : "yaml" - } - ] - }, - { - "name" : "git", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install using a Git URI.", - "example" : [ - { - "example" : "git: [ https://some.git.repository/org/repo ]", - "format" : "yaml" - } - ] - }, - { - "name" : "upgrade", - "type" : "Boolean", - "description" : "Sets the `--upgrade` flag when set to true. Default: true." - }, - { - "name" : "packages", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install from pip.", - "example" : [ - { - "example" : "packages: [ numpy ]", - "format" : "yaml" - } - ] - }, - { - "name" : "url", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install using a generic URI.", - "example" : [ - { - "example" : "url: [ https://github.com/some_org/some_pkg/zipball/master ]", - "format" : "yaml" - } - ] - }, - { - "name" : "svn", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install using an SVN URI.", - "example" : [ - { - "example" : "svn: [ http://svn.repo/some_pkg/trunk/#egg=SomePackage ]", - "format" : "yaml" - } - ] - }, - { - "name" : "bazaar", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install using a Bazaar URI.", - "example" : [ - { - "example" : "bazaar: [ http://bazaar.launchpad.net/some_pkg/some_pkg/release-0.1 ]", - "format" : "yaml" - } - ] - }, - { - "name" : "script", - "type" : "OneOrMore of String", - "description" : "Specifies a code block to run as part of the build.", - "example" : [ - { - "example" : "script: |\n print(\"Running custom code\")\n x = 1 + 1 == 2", - "format" : "yaml" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the requirement specification." - }, - { - "name" : "mercurial", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install using a Mercurial URI.", - "example" : [ - { - "example" : "mercurial: [ https://hg.myproject.org/MyProject/#egg=MyProject ]", - "format" : "yaml" - } - ] - }, - { - "name" : "user", - "type" : "Boolean", - "description" : "Sets the `--user` flag when set to true. Default: false." - } - ], - "requirements" : [ - { - "name" : "__this__", - "type" : "Requirements", - "hierarchy" : [ - "io.viash.platforms.requirements.Requirements" - ], - "description" : "Requirements for installing the following types of packages:\n\n - @[apt](apt_req)\n - @[apk](apk_req)\n - @[Docker setup instructions](docker_req)\n - @[JavaScript](javascript_req)\n - @[Python](python_req)\n - @[R](r_req)\n - @[Ruby](ruby_req)\n - @[yum](yum_req)\n" - } - ], - "rRequirements" : [ - { - "name" : "__this__", - "type" : "RRequirements", - "hierarchy" : [ - "io.viash.platforms.requirements.RRequirements", - "io.viash.platforms.requirements.Requirements" - ], - "description" : "Specify which R packages should be available in order to run the component.", - "example" : [ - { - "example" : "setup: \n - type: r\n cran: anndata\n bioc: [ AnnotationDbi, SingleCellExperiment ]\n github: rcannood/SCORPIUS\n", - "format" : "yaml" - } - ] - }, - { - "name" : "bioc", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install from BioConductor.", - "example" : [ - { - "example" : "bioc: [ AnnotationDbi ]", - "format" : "yaml" - } - ] - }, - { - "name" : "github", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install from GitHub.", - "example" : [ - { - "example" : "github: [ rcannood/SCORPIUS ]", - "format" : "yaml" - } - ] - }, - { - "name" : "gitlab", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install from GitLab.", - "example" : [ - { - "example" : "gitlab: [ org/package ]", - "format" : "yaml" - } - ] - }, - { - "name" : "url", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install using a generic URI.", - "example" : [ - { - "example" : "url: [ https://github.com/hadley/stringr/archive/HEAD.zip ]", - "format" : "yaml" - } - ] - }, - { - "name" : "bioc_force_install", - "type" : "Boolean", - "description" : "Forces packages specified in `bioc` to be reinstalled, even if they are already present in the container. Default: false.", - "example" : [ - { - "example" : "bioc_force_install: false", - "format" : "yaml" - } - ] - }, - { - "name" : "git", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install using a Git URI.", - "example" : [ - { - "example" : "git: [ https://some.git.repository/org/repo ]", - "format" : "yaml" - } - ] - }, - { - "name" : "cran", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install from CRAN.", - "example" : [ - { - "example" : "cran: [ anndata, ggplot2 ]", - "format" : "yaml" - } - ] - }, - { - "name" : "bitbucket", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install from Bitbucket.", - "example" : [ - { - "example" : "bitbucket: [ org/package ]", - "format" : "yaml" - } - ] - }, - { - "name" : "svn", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install using an SVN URI.", - "example" : [ - { - "example" : "svn: [ https://path.to.svn/group/repo ]", - "format" : "yaml" - } - ] - }, - { - "name" : "packages", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install from CRAN.", - "example" : [ - { - "example" : "packages: [ anndata, ggplot2 ]", - "format" : "yaml" - } - ] - }, - { - "name" : "script", - "type" : "OneOrMore of String", - "description" : "Specifies a code block to run as part of the build.", - "example" : [ - { - "example" : "script: |\n cat(\"Running custom code\n\")\n install.packages(\"anndata\")", - "format" : "yaml" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the requirement specification." - } - ], - "rubyRequirements" : [ - { - "name" : "__this__", - "type" : "RubyRequirements", - "hierarchy" : [ - "io.viash.platforms.requirements.RubyRequirements", - "io.viash.platforms.requirements.Requirements" - ], - "description" : "Specify which Ruby packages should be available in order to run the component.", - "example" : [ - { - "example" : "setup:\n - type: ruby\n packages: [ rspec ]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the requirement specification." - }, - { - "name" : "packages", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install.", - "example" : [ - { - "example" : "packages: [ rspec ]", - "format" : "yaml" - } - ] - } - ], - "yumRequirements" : [ - { - "name" : "__this__", - "type" : "YumRequirements", - "hierarchy" : [ - "io.viash.platforms.requirements.YumRequirements", - "io.viash.platforms.requirements.Requirements" - ], - "description" : "Specify which yum packages should be available in order to run the component.", - "example" : [ - { - "example" : "setup:\n - type: yum\n packages: [ sl ]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the requirement specification." - }, - { - "name" : "packages", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install.", - "example" : [ - { - "example" : "packages: [ sl ]", - "format" : "yaml" - } - ] - } - ], - "apkRequirements" : [ - { - "name" : "__this__", - "type" : "ApkRequirements", - "hierarchy" : [ - "io.viash.platforms.requirements.ApkRequirements", - "io.viash.platforms.requirements.Requirements" - ], - "description" : "Specify which apk packages should be available in order to run the component.", - "example" : [ - { - "example" : "setup:\n - type: apk\n packages: [ sl ]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the requirement specification." - }, - { - "name" : "packages", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install.", - "example" : [ - { - "example" : "packages: [ sl ]", - "format" : "yaml" - } - ] - } - ], - "dockerRequirements" : [ - { - "name" : "__this__", - "type" : "DockerRequirements", - "hierarchy" : [ - "io.viash.platforms.requirements.DockerRequirements", - "io.viash.platforms.requirements.Requirements" - ], - "description" : "Specify which Docker commands should be run during setup.", - "example" : [ - { - "example" : "setup:\n - type: docker\n build_args: \"R_VERSION=hello_world\"\n run: |\n echo 'Run a custom command'\n echo 'Foo' > /path/to/file.txt", - "format" : "yaml" - } - ] - }, - { - "name" : "run", - "type" : "OneOrMore of String", - "description" : "Specifies which `RUN` entries to add to the Dockerfile while building it.", - "example" : [ - { - "example" : "run: |\n echo 'Run a custom command'\n echo 'Foo' > /path/to/file.txt", - "format" : "yaml" - } - ] - }, - { - "name" : "label", - "type" : "OneOrMore of String", - "description" : "Specifies which `LABEL` entries to add to the Dockerfile while building it.", - "example" : [ - { - "example" : "label: [ component=\"foo\" ]", - "format" : "yaml" - } - ] - }, - { - "name" : "build_args", - "type" : "OneOrMore of String", - "description" : "Specifies which `ARG` entries to add to the Dockerfile while building it.", - "example" : [ - { - "example" : "build_args: [ \"R_VERSION=4.2\" ]", - "format" : "yaml" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the requirement specification." - }, - { - "name" : "add", - "type" : "OneOrMore of String", - "description" : "Specifies which `ADD` entries to add to the Dockerfile while building it.", - "example" : [ - { - "example" : "add: [ \"http://foo/bar .\" ]", - "format" : "yaml" - } - ] - }, - { - "name" : "env", - "type" : "OneOrMore of String", - "description" : "Specifies which `ENV` entries to add to the Dockerfile while building it. Unlike `ARG`, `ENV` entries are also accessible from inside the container.", - "example" : [ - { - "example" : "env: [ \"R_VERSION=4.2\" ]", - "format" : "yaml" - } - ] - }, - { - "name" : "resources", - "type" : "OneOrMore of String", - "description" : "Specifies which `COPY` entries to add to the Dockerfile while building it.", - "example" : [ - { - "example" : "resources: [ \"resource.txt /path/to/resource.txt\" ]", - "format" : "yaml" - } - ], - "removed" : { - "message" : "`resources` in `setup: {type: docker, resources: ...}` was removed. Please use `copy` instead.", - "deprecation" : "0.6.3", - "removal" : "0.7.0" + ], + "default" : "Empty" + }, + { + "name" : "argument_groups", + "type" : "List[ArgumentGroup]", + "niceType" : "List of ArgumentGroup", + "description" : "A grouping of the @[arguments](argument), used to display the help message.\n\n - `name: foo`, the name of the argument group. \n - `description: Description of foo`, a description of the argument group. Multiline descriptions are supported.\n - `arguments: [arg1, arg2, ...]`, list of the arguments.\n\n", + "example" : [ + { + "example" : "argument_groups:\n - name: \"Input\"\n arguments:\n - name: \"--id\"\n type: string\n required: true\n - name: \"--input\"\n type: file\n required: true\n - name: \"Output\"\n arguments:\n - name: \"--output\"\n type: file\n direction: output\n required: true\n - name: \"--output_optional\"\n type: file\n direction: output\n", + "format" : "yaml" + }, + { + "example" : "component_name\n\n Input:\n --id\n type: string\n\n --input\n type: file\n\n Output:\n --output\n type: file\n\n --optional_output\n type: file\n", + "format" : "bash", + "description" : "This results in the following output when calling the component with the `--help` argument:" } - }, - { - "name" : "copy", - "type" : "OneOrMore of String", - "description" : "Specifies which `COPY` entries to add to the Dockerfile while building it.", - "example" : [ - { - "example" : "copy: [ \"resource.txt /path/to/resource.txt\" ]", - "format" : "yaml" - } - ] - } - ], - "aptRequirements" : [ - { - "name" : "__this__", - "type" : "AptRequirements", - "hierarchy" : [ - "io.viash.platforms.requirements.AptRequirements", - "io.viash.platforms.requirements.Requirements" - ], - "description" : "Specify which apt packages should be available in order to run the component.", - "example" : [ - { - "example" : "setup:\n - type: apt\n packages: [ sl ]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "interactive", - "type" : "Boolean", - "description" : "If `false`, the Debian frontend is set to non-interactive (recommended). Default: false." - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the requirement specification." - }, - { - "name" : "packages", - "type" : "OneOrMore of String", - "description" : "Specifies which packages to install.", - "example" : [ - { - "example" : "packages: [ sl ]", - "format" : "yaml" - } - ] - } - ] - }, - "arguments" : { - "string" : [ - { - "name" : "__this__", - "type" : "StringArgument", - "hierarchy" : [ - "io.viash.functionality.arguments.StringArgument", - "io.viash.functionality.arguments.Argument" - ], - "description" : "A `string` type argument has a value made up of an ordered sequences of characters, like \"Hello\" or \"I'm a string\".", - "example" : [ - { - "example" : "arguments:\n - name: --search_query\n type: string\n default: \"meaning of life\"\n description: The term to search for\n alternatives: [\"-q\"]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "alternatives", - "type" : "OneOrMore of String", - "description" : "List of alternative format variations for this argument." - }, - { - "name" : "name", - "type" : "String", - "description" : "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The number of dashes determines how values can be passed: \n\n - `--foo` is a long option, which can be passed with `executable_name --foo=value` or `executable_name --foo value`\n - `-f` is a short option, which can be passed with `executable_name -f value`\n - `foo` is an argument, which can be passed with `executable_name value` \n" - }, - { - "name" : "choices", - "type" : "List of String", - "description" : "Limit the amount of valid values for this argument to those set in this list. When set and a value not present in the list is provided, an error will be produced.", - "example" : [ - { - "example" : "- name: --language\n type: string\n choices: [\"python\", \"r\", \"javascript\"]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "info", - "type" : "Json", - "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", - "example" : [ - { - "example" : "info:\n category: cat1\n labels: [one, two, three]", - "format" : "yaml" - } - ], - "since" : "Viash 0.6.3" - }, - { - "name" : "default", - "type" : "OneOrMore of String", - "description" : "The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled.", - "example" : [ - { - "example" : "- name: --my_string\n type: string\n default: \"The answer is 42\"\n", - "format" : "yaml" - } - ] - }, - { - "name" : "example", - "type" : "OneOrMore of String", - "description" : "An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose.", - "example" : [ - { - "example" : "- name: --my_string\n type: string\n example: \"Hello World\"\n", - "format" : "yaml" - } - ] - }, - { - "name" : "description", - "type" : "Option of String", - "description" : "A description of the argument. This will be displayed with `--help`." - }, - { - "name" : "multiple_sep", - "type" : "String", - "description" : "The delimiter character for providing [`multiple`](#multiple) values. `:` by default.", - "example" : [ - { - "example" : "- name: --my_string\n type: string\n multiple: true\n multiple_sep: \",\"\n", - "format" : "yaml" - }, - { - "example" : "my_component --my_string=Marc,Susan,Paul", - "format" : "bash", - "description" : "Here's an example of how to use this:" - } - ] - }, - { - "name" : "multiple", - "type" : "Boolean", - "description" : "Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default.", - "example" : [ - { - "example" : "- name: --my_string\n type: string\n multiple: true\n", - "format" : "yaml" - }, - { - "example" : "my_component --my_string=Marc:Susan:Paul", - "format" : "bash", - "description" : "Here's an example of how to use this:" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the argument." - }, - { - "name" : "required", - "type" : "Boolean", - "description" : "Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default.", - "example" : [ - { - "example" : "- name: --my_string\n type: string\n required: true\n", - "format" : "yaml" - } - ] - } - ], - "double" : [ - { - "name" : "__this__", - "type" : "DoubleArgument", - "hierarchy" : [ - "io.viash.functionality.arguments.DoubleArgument", - "io.viash.functionality.arguments.Argument" - ], - "description" : "A `double` type argument has a numeric value with decimal points", - "example" : [ - { - "example" : "arguments:\n - name: --litres\n type: double\n default: 1.5\n description: Litres of fluid to process\n alternatives: [\"-l\"]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "alternatives", - "type" : "OneOrMore of String", - "description" : "List of alternative format variations for this argument." - }, - { - "name" : "name", - "type" : "String", - "description" : "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The number of dashes determines how values can be passed: \n\n - `--foo` is a long option, which can be passed with `executable_name --foo=value` or `executable_name --foo value`\n - `-f` is a short option, which can be passed with `executable_name -f value`\n - `foo` is an argument, which can be passed with `executable_name value` \n" - }, - { - "name" : "info", - "type" : "Json", - "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", - "example" : [ - { - "example" : "info:\n category: cat1\n labels: [one, two, three]", - "format" : "yaml" - } - ], - "since" : "Viash 0.6.3" - }, - { - "name" : "max", - "type" : "Option of Double", - "description" : "Maximum allowed value for this argument. If set and the provided value is higher than the maximum, an error will be produced. Can be combined with [`min`](#min) to clamp values.", - "example" : [ - { - "example" : "- name: --my_double\n type: double\n max: 80.4\n", - "format" : "yaml" - } - ] - }, - { - "name" : "default", - "type" : "OneOrMore of Double", - "description" : "The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled.", - "example" : [ - { - "example" : "- name: --my_double\n type: double\n default: 5.8\n", - "format" : "yaml" - } - ] - }, - { - "name" : "example", - "type" : "OneOrMore of Double", - "description" : "An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose.", - "example" : [ - { - "example" : "- name: --my_double\n type: double\n example: 5.8\n", - "format" : "yaml" - } - ] - }, - { - "name" : "description", - "type" : "Option of String", - "description" : "A description of the argument. This will be displayed with `--help`." - }, - { - "name" : "multiple_sep", - "type" : "String", - "description" : "The delimiter character for providing [`multiple`](#multiple) values. `:` by default.", - "example" : [ - { - "example" : "- name: --my_double\n type: double\n multiple: true\n multiple_sep: \",\"\n", - "format" : "yaml" - }, - { - "example" : "my_component --my_double=5.8,22.6,200.4", - "format" : "bash", - "description" : "Here's an example of how to use this:" - } - ] - }, - { - "name" : "min", - "type" : "Option of Double", - "description" : "Minimum allowed value for this argument. If set and the provided value is lower than the minimum, an error will be produced. Can be combined with [`max`](#max) to clamp values.", - "example" : [ - { - "example" : "- name: --my_double\n type: double\n min: 25.5\n", - "format" : "yaml" - } - ] - }, - { - "name" : "multiple", - "type" : "Boolean", - "description" : "Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default.", - "example" : [ - { - "example" : "- name: --my_double\n type: double\n multiple: true\n", - "format" : "yaml" - }, - { - "example" : "my_component --my_double=5.8:22.6:200.4", - "format" : "bash", - "description" : "Here's an example of how to use this:" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the argument." - }, - { - "name" : "required", - "type" : "Boolean", - "description" : "Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default.", - "example" : [ - { - "example" : "- name: --my_double\n type: double\n required: true\n", - "format" : "yaml" - } - ] - } - ], - "long" : [ - { - "name" : "__this__", - "type" : "LongArgument", - "hierarchy" : [ - "io.viash.functionality.arguments.LongArgument", - "io.viash.functionality.arguments.Argument" - ], - "description" : "An `long` type argument has a numeric value without decimal points.", - "example" : [ - { - "example" : "arguments:\n - name: --core_amount\n type: long\n default: 16\n description: Amount of CPU cores to use\n alternatives: [\"-c\"]\n", - "format" : "yaml" - } - ], - "since" : "Viash 0.6.1" - }, - { - "name" : "alternatives", - "type" : "OneOrMore of String", - "description" : "List of alternative format variations for this argument." - }, - { - "name" : "name", - "type" : "String", - "description" : "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The number of dashes determines how values can be passed: \n\n - `--foo` is a long option, which can be passed with `executable_name --foo=value` or `executable_name --foo value`\n - `-f` is a short option, which can be passed with `executable_name -f value`\n - `foo` is an argument, which can be passed with `executable_name value` \n" - }, - { - "name" : "choices", - "type" : "List of Long", - "description" : "Limit the amount of valid values for this argument to those set in this list. When set and a value not present in the list is provided, an error will be produced.", - "example" : [ - { - "example" : "- name: --values\n type: long\n choices: [1024, 2048, 4096]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "info", - "type" : "Json", - "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", - "example" : [ - { - "example" : "info:\n category: cat1\n labels: [one, two, three]", - "format" : "yaml" - } - ], - "since" : "Viash 0.6.3" - }, - { - "name" : "max", - "type" : "Option of Long", - "description" : "Maximum allowed value for this argument. If set and the provided value is higher than the maximum, an error will be produced. Can be combined with [`min`](#min) to clamp values.", - "example" : [ - { - "example" : "- name: --my_long\n type: long\n max: 150\n", - "format" : "yaml" - } - ] - }, - { - "name" : "default", - "type" : "OneOrMore of Long", - "description" : "The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled.", - "example" : [ - { - "example" : "- name: --my_long\n type: long\n default: 100\n", - "format" : "yaml" - } - ] - }, - { - "name" : "example", - "type" : "OneOrMore of Long", - "description" : "An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose.", - "example" : [ - { - "example" : "- name: --my_long\n type: long\n example: 100\n", - "format" : "yaml" - } - ] - }, - { - "name" : "description", - "type" : "Option of String", - "description" : "A description of the argument. This will be displayed with `--help`." - }, - { - "name" : "multiple_sep", - "type" : "String", - "description" : "The delimiter character for providing [`multiple`](#multiple) values. `:` by default.", - "example" : [ - { - "example" : "- name: --my_long\n type: long\n multiple: true\n multiple_sep: \",\"\n", - "format" : "yaml" - }, - { - "example" : "my_component --my_long=10:80:152", - "format" : "bash", - "description" : "Here's an example of how to use this:" - } - ] - }, - { - "name" : "min", - "type" : "Option of Long", - "description" : "Minimum allowed value for this argument. If set and the provided value is lower than the minimum, an error will be produced. Can be combined with [`max`](#max) to clamp values.", - "example" : [ - { - "example" : "- name: --my_long\n type: long\n min: 50\n", - "format" : "yaml" - } - ] - }, - { - "name" : "multiple", - "type" : "Boolean", - "description" : "Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default.", - "example" : [ - { - "example" : "- name: --my_long\n type: long\n multiple: true\n", - "format" : "yaml" - }, - { - "example" : "my_component --my_long=10:80:152", - "format" : "bash", - "description" : "Here's an example of how to use this:" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the argument." - }, - { - "name" : "required", - "type" : "Boolean", - "description" : "Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default.", - "example" : [ - { - "example" : "- name: --my_long\n type: long\n required: true\n", - "format" : "yaml" - } - ] - } - ], - "boolean" : [ - { - "name" : "__this__", - "type" : "BooleanArgument", - "hierarchy" : [ - "io.viash.functionality.arguments.BooleanArgument", - "io.viash.functionality.arguments.BooleanArgumentBase", - "io.viash.functionality.arguments.Argument" - ], - "description" : "A `boolean` type argument has two possible values: `true` or `false`.", - "example" : [ - { - "example" : "arguments:\n - name: --trim\n type: boolean\n default: true\n description: Trim whitespace from the final output\n alternatives: [\"-t\"]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "alternatives", - "type" : "OneOrMore of String", - "description" : "List of alternative format variations for this argument." - }, - { - "name" : "name", - "type" : "String", - "description" : "The name of the argument. Can be in the formats `--trim`, `-t` or `trim`. The number of dashes determines how values can be passed: \n\n - `--trim` is a long option, which can be passed with `executable_name --trim`\n - `-t` is a short option, which can be passed with `executable_name -t`\n - `trim` is an argument, which can be passed with `executable_name trim` \n" - }, - { - "name" : "info", - "type" : "Json", - "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", - "example" : [ - { - "example" : "info:\n category: cat1\n labels: [one, two, three]", - "format" : "yaml" - } - ], - "since" : "Viash 0.6.3" - }, - { - "name" : "default", - "type" : "OneOrMore of Boolean", - "description" : "The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled.", - "example" : [ - { - "example" : "- name: --my_boolean\n type: boolean\n default: true\n", - "format" : "yaml" - } - ] - }, - { - "name" : "example", - "type" : "OneOrMore of Boolean", - "description" : "An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose.", - "example" : [ - { - "example" : "- name: --my_boolean\n type: boolean\n example: true\n", - "format" : "yaml" - } - ] - }, - { - "name" : "description", - "type" : "Option of String", - "description" : "A description of the argument. This will be displayed with `--help`." - }, - { - "name" : "multiple_sep", - "type" : "String", - "description" : "The delimiter character for providing [`multiple`](#multiple) values. `:` by default.", - "example" : [ - { - "example" : "- name: --my_boolean\n type: boolean\n multiple: true\n multiple_sep: \",\"\n", - "format" : "yaml" - }, - { - "example" : "my_component --my_boolean=true,true,false", - "format" : "bash", - "description" : "Here's an example of how to use this:" - } - ] - }, - { - "name" : "multiple", - "type" : "Boolean", - "description" : "Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default.", - "example" : [ - { - "example" : "- name: --my_boolean\n type: boolean\n multiple: true\n", - "format" : "yaml" - }, - { - "example" : "my_component --my_boolean=true:true:false", - "format" : "bash", - "description" : "Here's an example of how to use this:" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the argument." - }, - { - "name" : "required", - "type" : "Boolean", - "description" : "Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default.", - "example" : [ - { - "example" : "- name: --my_boolean\n type: boolean\n required: true\n", - "format" : "yaml" - } - ] - } - ], - "integer" : [ - { - "name" : "__this__", - "type" : "IntegerArgument", - "hierarchy" : [ - "io.viash.functionality.arguments.IntegerArgument", - "io.viash.functionality.arguments.Argument" - ], - "description" : "An `integer` type argument has a numeric value without decimal points.", - "example" : [ - { - "example" : "arguments:\n - name: --core_amount\n type: integer\n default: 16\n description: Amount of CPU cores to use\n alternatives: [\"-c\"]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "alternatives", - "type" : "OneOrMore of String", - "description" : "List of alternative format variations for this argument." - }, - { - "name" : "name", - "type" : "String", - "description" : "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The number of dashes determines how values can be passed: \n\n - `--foo` is a long option, which can be passed with `executable_name --foo=value` or `executable_name --foo value`\n - `-f` is a short option, which can be passed with `executable_name -f value`\n - `foo` is an argument, which can be passed with `executable_name value` \n" - }, - { - "name" : "choices", - "type" : "List of Int", - "description" : "Limit the amount of valid values for this argument to those set in this list. When set and a value not present in the list is provided, an error will be produced.", - "example" : [ - { - "example" : "- name: --values\n type: integer\n choices: [1024, 2048, 4096]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "info", - "type" : "Json", - "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", - "example" : [ - { - "example" : "info:\n category: cat1\n labels: [one, two, three]", - "format" : "yaml" - } - ], - "since" : "Viash 0.6.3" - }, - { - "name" : "max", - "type" : "Option of Int", - "description" : "Maximum allowed value for this argument. If set and the provided value is higher than the maximum, an error will be produced. Can be combined with [`min`](#min) to clamp values.", - "example" : [ - { - "example" : "- name: --my_integer\n type: integer\n max: 150\n", - "format" : "yaml" - } - ] - }, - { - "name" : "default", - "type" : "OneOrMore of Int", - "description" : "The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled.", - "example" : [ - { - "example" : "- name: --my_integer\n type: integer\n default: 100\n", - "format" : "yaml" - } - ] - }, - { - "name" : "example", - "type" : "OneOrMore of Int", - "description" : "An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose.", - "example" : [ - { - "example" : "- name: --my_integer\n type: integer\n example: 100\n", - "format" : "yaml" - } - ] - }, - { - "name" : "description", - "type" : "Option of String", - "description" : "A description of the argument. This will be displayed with `--help`." - }, - { - "name" : "multiple_sep", - "type" : "String", - "description" : "The delimiter character for providing [`multiple`](#multiple) values. `:` by default.", - "example" : [ - { - "example" : "- name: --my_integer\n type: integer\n multiple: true\n multiple_sep: \",\"\n", - "format" : "yaml" - }, - { - "example" : "my_component --my_integer=10:80:152", - "format" : "bash", - "description" : "Here's an example of how to use this:" - } - ] - }, - { - "name" : "min", - "type" : "Option of Int", - "description" : "Minimum allowed value for this argument. If set and the provided value is lower than the minimum, an error will be produced. Can be combined with [`max`](#max) to clamp values.", - "example" : [ - { - "example" : "- name: --my_integer\n type: integer\n min: 50\n", - "format" : "yaml" - } - ] - }, - { - "name" : "multiple", - "type" : "Boolean", - "description" : "Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default.", - "example" : [ - { - "example" : "- name: --my_integer\n type: integer\n multiple: true\n", - "format" : "yaml" - }, - { - "example" : "my_component --my_integer=10:80:152", - "format" : "bash", - "description" : "Here's an example of how to use this:" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the argument." - }, - { - "name" : "required", - "type" : "Boolean", - "description" : "Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default.", - "example" : [ - { - "example" : "- name: --my_integer\n type: integer\n required: true\n", - "format" : "yaml" - } - ] - } - ], - "file" : [ - { - "name" : "__this__", - "type" : "FileArgument", - "hierarchy" : [ - "io.viash.functionality.arguments.FileArgument", - "io.viash.functionality.arguments.Argument" - ], - "description" : "A `file` type argument has a string value that points to a file or folder path.", - "example" : [ - { - "example" : "arguments:\n - name: --input_csv\n type: file\n must_exist: true\n description: CSV file to read contents from\n alternatives: [\"-i\"]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "alternatives", - "type" : "OneOrMore of String", - "description" : "List of alternative format variations for this argument." - }, - { - "name" : "name", - "type" : "String", - "description" : "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The number of dashes determines how values can be passed: \n\n - `--foo` is a long option, which can be passed with `executable_name --foo=value` or `executable_name --foo value`\n - `-f` is a short option, which can be passed with `executable_name -f value`\n - `foo` is an argument, which can be passed with `executable_name value` \n" - }, - { - "name" : "create_parent", - "type" : "Boolean", - "description" : "If the output filename is a path and it does not exist, create it before executing the script (only for `direction: output`).", - "example" : [ - { - "example" : "- name: --my_file\n type: file\n direction: output\n create_parent: true\n", - "format" : "yaml" - } - ] - }, - { - "name" : "direction", - "type" : "Direction", - "description" : "Makes this argument an `input` or an `output`, as in does the file/folder needs to be read or written. `input` by default.", - "example" : [ - { - "example" : "- name: --my_output_file\n type: file\n direction: output\n", - "format" : "yaml" - } - ] - }, - { - "name" : "info", - "type" : "Json", - "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", - "example" : [ - { - "example" : "info:\n category: cat1\n labels: [one, two, three]", - "format" : "yaml" - } - ], - "since" : "Viash 0.6.3" - }, - { - "name" : "must_exist", - "type" : "Boolean", - "description" : "Checks whether the file or folder exists. For input files, this check will happen before the execution of the script, while for output files the check will happen afterwards.", - "example" : [ - { - "example" : "- name: --my_file\n type: file\n must_exist: true\n", - "format" : "yaml" - } - ] - }, - { - "name" : "default", - "type" : "OneOrMore of Path", - "description" : "The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled.", - "example" : [ - { - "example" : "- name: --my_file\n type: file\n default: data.csv\n", - "format" : "yaml" - } - ] - }, - { - "name" : "example", - "type" : "OneOrMore of Path", - "description" : "An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose.", - "example" : [ - { - "example" : "- name: --my_file\n type: file\n example: data.csv\n", - "format" : "yaml" - } - ] - }, - { - "name" : "description", - "type" : "Option of String", - "description" : "A description of the argument. This will be displayed with `--help`." - }, - { - "name" : "multiple_sep", - "type" : "String", - "description" : "The delimiter character for providing [`multiple`](#multiple) values. `:` by default.", - "example" : [ - { - "example" : "- name: --my_files\n type: file\n multiple: true\n multiple_sep: \",\"\n", - "format" : "yaml" - }, - { - "example" : "my_component --my_files=firstFile.csv,anotherFile.csv,yetAnother.csv", - "format" : "bash", - "description" : "Here's an example of how to use this:" - } - ] - }, - { - "name" : "multiple", - "type" : "Boolean", - "description" : "Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default.", - "example" : [ - { - "example" : "- name: --my_files\n type: file\n multiple: true\n", - "format" : "yaml" - }, - { - "example" : "my_component --my_files=firstFile.csv:anotherFile.csv:yetAnother.csv", - "format" : "bash", - "description" : "Here's an example of how to use this:" - } - ] - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the argument." - }, - { - "name" : "required", - "type" : "Boolean", - "description" : "Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default.", - "example" : [ - { - "example" : "- name: --my_file\n type: file\n required: true\n", - "format" : "yaml" - } - ] - } - ], - "boolean_false" : [ - { - "name" : "__this__", - "type" : "BooleanFalseArgument", - "hierarchy" : [ - "io.viash.functionality.arguments.BooleanFalseArgument", - "io.viash.functionality.arguments.BooleanArgumentBase", - "io.viash.functionality.arguments.Argument" - ], - "description" : "An argument of the `boolean_false` type acts like an inverted `boolean` flag with a default value of `true`. When called as an argument it sets the `boolean` to `false`.", - "example" : [ - { - "example" : "arguments:\n - name: --no-log\n type: boolean_false\n description: Disable logging\n alternatives: [\"-nl\"]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "alternatives", - "type" : "OneOrMore of String", - "description" : "List of alternative format variations for this argument." - }, - { - "name" : "name", - "type" : "String", - "description" : "The name of the argument. Can be in the formats `--no-log`, `-n` or `no-log`. The number of dashes determines how values can be passed: \n\n - `--no-log` is a long option, which can be passed with `executable_name --no-log`\n - `-n` is a short option, which can be passed with `executable_name -n`\n - `no-log` is an argument, which can be passed with `executable_name no-log` \n" - }, - { - "name" : "info", - "type" : "Json", - "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", - "example" : [ - { - "example" : "info:\n category: cat1\n labels: [one, two, three]", - "format" : "yaml" - } - ], - "since" : "Viash 0.6.3" - }, - { - "name" : "description", - "type" : "Option of String", - "description" : "A description of the argument. This will be displayed with `--help`." - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the argument." - } - ], - "argument" : [ - { - "name" : "__this__", - "type" : "Argument", - "hierarchy" : [ - "io.viash.functionality.arguments.Argument" - ], - "description" : "For each argument, a type and a name must be specified. Depending on the type of argument, different properties can be set. See these reference pages per type for more information: \n\n - @[string](arg_string)\n - @[file](arg_file)\n - @[integer](arg_integer)\n - @[double](arg_double)\n - @[boolean](arg_boolean)\n - @[boolean_true](arg_boolean_true)\n - @[boolean_false](arg_boolean_false)\n", - "example" : [ - { - "example" : "arguments:\n - name: --foo\n type: file\n alternatives: [-f]\n description: Description of foo\n default: \"/foo/bar\"\n must_exist: true\n direction: output\n required: false\n multiple: true\n multiple_sep: \",\"\n - name: --bar\n type: string\n", - "format" : "yaml" - } - ] - } - ], - "boolean_true" : [ - { - "name" : "__this__", - "type" : "BooleanTrueArgument", - "hierarchy" : [ - "io.viash.functionality.arguments.BooleanTrueArgument", - "io.viash.functionality.arguments.BooleanArgumentBase", - "io.viash.functionality.arguments.Argument" - ], - "description" : "An argument of the `boolean_true` type acts like a `boolean` flag with a default value of `false`. When called as an argument it sets the `boolean` to `true`.", - "example" : [ - { - "example" : "arguments:\n - name: --silent\n type: boolean_true\n description: Ignore console output\n alternatives: [\"-s\"]\n", - "format" : "yaml" - } - ] - }, - { - "name" : "alternatives", - "type" : "OneOrMore of String", - "description" : "List of alternative format variations for this argument." - }, - { - "name" : "name", - "type" : "String", - "description" : "The name of the argument. Can be in the formats `--silent`, `-s` or `silent`. The number of dashes determines how values can be passed: \n\n - `--silent` is a long option, which can be passed with `executable_name --silent`\n - `-s` is a short option, which can be passed with `executable_name -s`\n - `silent` is an argument, which can be passed with `executable_name silent` \n" - }, - { - "name" : "info", - "type" : "Json", - "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", - "example" : [ - { - "example" : "info:\n category: cat1\n labels: [one, two, three]", - "format" : "yaml" - } - ], - "since" : "Viash 0.6.3" - }, - { - "name" : "description", - "type" : "Option of String", - "description" : "A description of the argument. This will be displayed with `--help`." - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the type of the argument." - } - ] - }, - "resources" : { - "cSharpScript" : [ - { - "name" : "__this__", - "type" : "CSharpScript", - "hierarchy" : [ - "io.viash.functionality.resources.CSharpScript", - "io.viash.functionality.resources.Script", - "io.viash.functionality.resources.Resource" - ], - "description" : "An executable C# script.\nWhen defined in functionality.resources, only the first entry will be executed when running the built component or when running `viash run`.\nWhen defined in functionality.test_resources, all entries will be executed during `viash test`." - }, - { - "name" : "path", - "type" : "Option of String", - "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`." - }, - { - "name" : "text", - "type" : "Option of String", - "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`." - }, - { - "name" : "is_executable", - "type" : "Option of Boolean", - "description" : "Whether the resulting resource file should be made executable." - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the resource as a C# script." - }, - { - "name" : "dest", - "type" : "Option of String", - "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter." - } - ], - "pythonScript" : [ - { - "name" : "__this__", - "type" : "PythonScript", - "hierarchy" : [ - "io.viash.functionality.resources.PythonScript", - "io.viash.functionality.resources.Script", - "io.viash.functionality.resources.Resource" - ], - "description" : "An executable Python script.\nWhen defined in functionality.resources, only the first entry will be executed when running the built component or when running `viash run`.\nWhen defined in functionality.test_resources, all entries will be executed during `viash test`." - }, - { - "name" : "path", - "type" : "Option of String", - "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`." - }, - { - "name" : "text", - "type" : "Option of String", - "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`." - }, - { - "name" : "is_executable", - "type" : "Option of Boolean", - "description" : "Whether the resulting resource file should be made executable." - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the resource as a Python script." - }, - { - "name" : "dest", - "type" : "Option of String", - "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter." - } - ], - "plainFile" : [ - { - "name" : "__this__", - "type" : "PlainFile", - "hierarchy" : [ - "io.viash.functionality.resources.PlainFile", - "io.viash.functionality.resources.Resource" - ], - "description" : "A plain file. This can only be used as a supporting resource for the main script or unit tests." - }, - { - "name" : "path", - "type" : "Option of String", - "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`." - }, - { - "name" : "text", - "type" : "Option of String", - "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`." - }, - { - "name" : "is_executable", - "type" : "Option of Boolean", - "description" : "Whether the resulting resource file should be made executable." - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the resource as a plain file." - }, - { - "name" : "dest", - "type" : "Option of String", - "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter." - } - ], - "resource" : [ - { - "name" : "__this__", - "type" : "Resource", - "hierarchy" : [ - "io.viash.functionality.resources.Resource" - ], - "description" : "Resources are files that support the component. The first resource should be @[a script](scripting_languages) that will be executed when the functionality is run. Additional resources will be copied to the same directory.\n\nCommon properties:\n\n * type: `file` / `r_script` / `python_script` / `bash_script` / `javascript_script` / `scala_script` / `csharp_script`, specifies the type of the resource. The first resource cannot be of type `file`. When the type is not specified, the default type is simply `file`.\n * dest: filename, the resulting name of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter.\n * path: `path/to/file`, the path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`.\n * text: ...multiline text..., the content of the resulting file specified as a string. Mutually exclusive with `path`.\n * is_executable: `true` / `false`, whether the resulting resource file should be made executable.\n", - "example" : [ - { - "example" : "resources:\n - type: r_script\n path: script.R\n - type: file\n path: resource1.txt\n", - "format" : "yaml" - } - ] - } - ], - "bashScript" : [ - { - "name" : "__this__", - "type" : "BashScript", - "hierarchy" : [ - "io.viash.functionality.resources.BashScript", - "io.viash.functionality.resources.Script", - "io.viash.functionality.resources.Resource" - ], - "description" : "An executable Bash script.\nWhen defined in functionality.resources, only the first entry will be executed when running the built component or when running `viash run`.\nWhen defined in functionality.test_resources, all entries will be executed during `viash test`." - }, - { - "name" : "path", - "type" : "Option of String", - "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`." - }, - { - "name" : "text", - "type" : "Option of String", - "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`." - }, - { - "name" : "is_executable", - "type" : "Option of Boolean", - "description" : "Whether the resulting resource file should be made executable." - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the resource as a Bash script." - }, - { - "name" : "dest", - "type" : "Option of String", - "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter." - } - ], - "scalaScript" : [ - { - "name" : "__this__", - "type" : "ScalaScript", - "hierarchy" : [ - "io.viash.functionality.resources.ScalaScript", - "io.viash.functionality.resources.Script", - "io.viash.functionality.resources.Resource" - ], - "description" : "An executable Scala script.\nWhen defined in functionality.resources, only the first entry will be executed when running the built component or when running `viash run`.\nWhen defined in functionality.test_resources, all entries will be executed during `viash test`." - }, - { - "name" : "path", - "type" : "Option of String", - "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`." - }, - { - "name" : "text", - "type" : "Option of String", - "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`." - }, - { - "name" : "is_executable", - "type" : "Option of Boolean", - "description" : "Whether the resulting resource file should be made executable." - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the resource as a Scala script." - }, - { - "name" : "dest", - "type" : "Option of String", - "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter." - } - ], - "nextflowScript" : [ - { - "name" : "__this__", - "type" : "NextflowScript", - "hierarchy" : [ - "io.viash.functionality.resources.NextflowScript", - "io.viash.functionality.resources.Script", - "io.viash.functionality.resources.Resource" - ], - "description" : "A Nextflow script. Work in progress; added mainly for annotation at the moment." - }, - { - "name" : "path", - "type" : "Option of String", - "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`." - }, - { - "name" : "text", - "type" : "Option of String", - "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`." - }, - { - "name" : "entrypoint", - "type" : "Option of String", - "description" : "The name of the workflow to be executed." - }, - { - "name" : "is_executable", - "type" : "Option of Boolean", - "description" : "Whether the resulting resource file should be made executable." - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the resource as a Nextflow script." - }, - { - "name" : "dest", - "type" : "Option of String", - "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter." - } - ], - "rScript" : [ - { - "name" : "__this__", - "type" : "RScript", - "hierarchy" : [ - "io.viash.functionality.resources.RScript", - "io.viash.functionality.resources.Script", - "io.viash.functionality.resources.Resource" - ], - "description" : "An executable R script.\nWhen defined in functionality.resources, only the first entry will be executed when running the built component or when running `viash run`.\nWhen defined in functionality.test_resources, all entries will be executed during `viash test`." - }, - { - "name" : "path", - "type" : "Option of String", - "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`." - }, - { - "name" : "text", - "type" : "Option of String", - "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`." - }, - { - "name" : "is_executable", - "type" : "Option of Boolean", - "description" : "Whether the resulting resource file should be made executable." - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the resource as a R script." - }, - { - "name" : "dest", - "type" : "Option of String", - "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter." - } - ], - "executable" : [ - { - "name" : "__this__", - "type" : "Executable", - "hierarchy" : [ - "io.viash.functionality.resources.Executable", - "io.viash.functionality.resources.Script", - "io.viash.functionality.resources.Resource" - ], - "description" : "An executable file." - }, - { - "name" : "path", - "type" : "Option of String", - "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`." - }, - { - "name" : "text", - "type" : "Option of String", - "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`." - }, - { - "name" : "is_executable", - "type" : "Option of Boolean", - "description" : "Whether the resulting resource file should be made executable." - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the resource as an executable." - }, - { - "name" : "dest", - "type" : "Option of String", - "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter." - } - ], - "javaScriptScript" : [ - { - "name" : "__this__", - "type" : "JavaScriptScript", - "hierarchy" : [ - "io.viash.functionality.resources.JavaScriptScript", - "io.viash.functionality.resources.Script", - "io.viash.functionality.resources.Resource" - ], - "description" : "An executable JavaScript script.\nWhen defined in functionality.resources, only the first entry will be executed when running the built component or when running `viash run`.\nWhen defined in functionality.test_resources, all entries will be executed during `viash test`." - }, - { - "name" : "path", - "type" : "Option of String", - "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`." - }, - { - "name" : "text", - "type" : "Option of String", - "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`." - }, - { - "name" : "is_executable", - "type" : "Option of Boolean", - "description" : "Whether the resulting resource file should be made executable." - }, - { - "name" : "type", - "type" : "String", - "description" : "Specifies the resource as a JavaScript script." - }, - { - "name" : "dest", - "type" : "Option of String", - "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter." - } - ] - }, - "nextflowParameters" : { - "nextflowDirectives" : [ - { - "name" : "__this__", - "type" : "NextflowDirectives", - "hierarchy" : [ - "io.viash.platforms.nextflow.NextflowDirectives" - ], - "description" : "Directives are optional settings that affect the execution of the process.\n", - "example" : [ - { - "example" : "directives:\n container: rocker/r-ver:4.1\n label: highcpu\n cpus: 4\n memory: 16 GB", - "format" : "yaml" - } - ] - }, - { - "name" : "beforeScript", - "type" : "Option of String", - "description" : "The `beforeScript` directive allows you to execute a custom (Bash) snippet before the main process script is run. This may be useful to initialise the underlying cluster environment or for other custom initialisation.\n\nSee [`beforeScript`](https://www.nextflow.io/docs/latest/process.html#beforeScript).\n", - "example" : [ - { - "example" : "source /cluster/bin/setup", - "format" : "yaml" - } - ] - }, - { - "name" : "module", - "type" : "OneOrMore of String", - "description" : "Environment Modules is a package manager that allows you to dynamically configure your execution environment and easily switch between multiple versions of the same software tool.\n\nIf it is available in your system you can use it with Nextflow in order to configure the processes execution environment in your pipeline.\n\nIn a process definition you can use the `module` directive to load a specific module version to be used in the process execution environment.\n\nSee [`module`](https://www.nextflow.io/docs/latest/process.html#module).\n", - "example" : [ - { - "example" : "\"ncbi-blast/2.2.27\"", - "format" : "yaml" - }, - { - "example" : "\"ncbi-blast/2.2.27:t_coffee/10.0\"", - "format" : "yaml" - }, - { - "example" : "[\"ncbi-blast/2.2.27\", \"t_coffee/10.0\"]", - "format" : "yaml" - } - ] - }, - { - "name" : "queue", - "type" : "OneOrMore of String", - "description" : "The `queue` directory allows you to set the queue where jobs are scheduled when using a grid based executor in your pipeline.\n\nSee [`queue`](https://www.nextflow.io/docs/latest/process.html#queue).\n", - "example" : [ - { - "example" : "\"long\"", - "format" : "yaml" - }, - { - "example" : "\"short,long\"", - "format" : "yaml" - }, - { - "example" : "[\"short\", \"long\"]", - "format" : "yaml" - } - ] - }, - { - "name" : "label", - "type" : "OneOrMore of String", - "description" : "The `label` directive allows the annotation of processes with mnemonic identifier of your choice.\n\nSee [`label`](https://www.nextflow.io/docs/latest/process.html#label).\n", - "example" : [ - { - "example" : "\"big_mem\"", - "format" : "yaml" - }, - { - "example" : "\"big_cpu\"", - "format" : "yaml" - }, - { - "example" : "[\"big_mem\", \"big_cpu\"]", - "format" : "yaml" - } - ] - }, - { - "name" : "container", - "type" : "Option[Either[Map of String,String,String]]", - "description" : "The `container` directive allows you to execute the process script in a Docker container.\n\nIt requires the Docker daemon to be running in machine where the pipeline is executed, i.e. the local machine when using the local executor or the cluster nodes when the pipeline is deployed through a grid executor.\n\nViash implements allows either a string value or a map. In case a map is used, the allowed keys are: `registry`, `image`, and `tag`. The `image` value must be specified.\n\nSee [`container`](https://www.nextflow.io/docs/latest/process.html#container).\n", - "example" : [ - { - "example" : "\"foo/bar:tag\"", - "format" : "yaml" - }, - { - "example" : "[ registry: \"reg\", image: \"im\", tag: \"ta\" ]", - "format" : "yaml", - "description" : "This is transformed to `\"reg/im:ta\"`:" - }, - { - "example" : "[ image: \"im\" ]", - "format" : "yaml", - "description" : "This is transformed to `\"im:latest\"`:" - } - ] - }, - { - "name" : "publishDir", - "type" : "OneOrMore[Either[String,Map of String,String]]", - "description" : "The `publishDir` directive allows you to publish the process output files to a specified folder.\n\nViash implements this directive as a plain string or a map. The allowed keywords for the map are: `path`, `mode`, `overwrite`, `pattern`, `saveAs`, `enabled`. The `path` key and value are required.\nThe allowed values for `mode` are: `symlink`, `rellink`, `link`, `copy`, `copyNoFollow`, `move`.\n\nSee [`publishDir`](https://www.nextflow.io/docs/latest/process.html#publishdir).\n", - "example" : [ - { - "example" : "[]", - "format" : "yaml" - }, - { - "example" : "[ [ path: \"foo\", enabled: true ], [ path: \"bar\", enabled: false ] ]", - "format" : "yaml" - }, - { - "example" : "\"/path/to/dir\"", - "format" : "yaml", - "description" : "This is transformed to `[[ path: \"/path/to/dir\" ]]`:" - }, - { - "example" : "[ path: \"/path/to/dir\", mode: \"cache\" ]", - "format" : "yaml", - "description" : "This is transformed to `[[ path: \"/path/to/dir\", mode: \"cache\" ]]`:" - } - ] - }, - { - "name" : "maxForks", - "type" : "Option[Either of String,Int]", - "description" : "The `maxForks` directive allows you to define the maximum number of process instances that can be executed in parallel. By default this value is equals to the number of CPU cores available minus 1.\n\nIf you want to execute a process in a sequential manner, set this directive to one.\n\nSee [`maxForks`](https://www.nextflow.io/docs/latest/process.html#maxforks).\n", - "example" : [ - { - "example" : "1", - "format" : "yaml" - }, - { - "example" : "3", - "format" : "yaml" - } - ] - }, - { - "name" : "maxErrors", - "type" : "Option[Either of String,Int]", - "description" : "The `maxErrors` directive allows you to specify the maximum number of times a process can fail when using the `retry` error strategy. By default this directive is disabled.\n\nSee [`maxErrors`](https://www.nextflow.io/docs/latest/process.html#maxerrors).\n", - "example" : [ - { - "example" : "1", - "format" : "yaml" - }, - { - "example" : "3", - "format" : "yaml" - } - ] - }, - { - "name" : "cpus", - "type" : "Option[Either of Int,String]", - "description" : "The `cpus` directive allows you to define the number of (logical) CPU required by the process' task.\n\nSee [`cpus`](https://www.nextflow.io/docs/latest/process.html#cpus).\n", - "example" : [ - { - "example" : "1", - "format" : "yaml" - }, - { - "example" : "10", - "format" : "yaml" - } - ] - }, - { - "name" : "accelerator", - "type" : "Map of String,String", - "description" : "The `accelerator` directive allows you to specify the hardware accelerator requirement for the task execution e.g. GPU processor.\n\nViash implements this directive as a map with accepted keywords: `type`, `limit`, `request`, and `runtime`.\n\nSee [`accelerator`](https://www.nextflow.io/docs/latest/process.html#accelerator).\n", - "example" : [ - { - "example" : "[ limit: 4, type: \"nvidia-tesla-k80\" ]", - "format" : "yaml" - } - ] - }, - { - "name" : "time", - "type" : "Option of String", - "description" : "The `time` directive allows you to define how long a process is allowed to run.\n\nSee [`time`](https://www.nextflow.io/docs/latest/process.html#time).\n", - "example" : [ - { - "example" : "\"1h\"", - "format" : "yaml" - }, - { - "example" : "\"2days\"", - "format" : "yaml" - }, - { - "example" : "\"1day 6hours 3minutes 30seconds\"", - "format" : "yaml" - } - ] - }, - { - "name" : "afterScript", - "type" : "Option of String", - "description" : "The `afterScript` directive allows you to execute a custom (Bash) snippet immediately after the main process has run. This may be useful to clean up your staging area.\n\nSee [`afterScript`](https://www.nextflow.io/docs/latest/process.html#afterscript).\n", - "example" : [ - { - "example" : "source /cluster/bin/cleanup", - "format" : "yaml" - } - ] - }, - { - "name" : "executor", - "type" : "Option of String", - "description" : "The `executor` defines the underlying system where processes are executed. By default a process uses the executor defined globally in the nextflow.config file.\n\nThe `executor` directive allows you to configure what executor has to be used by the process, overriding the default configuration. The following values can be used:\n\n| Name | Executor |\n|------|----------|\n| awsbatch | The process is executed using the AWS Batch service. | \n| azurebatch | The process is executed using the Azure Batch service. | \n| condor | The process is executed using the HTCondor job scheduler. | \n| google-lifesciences | The process is executed using the Google Genomics Pipelines service. | \n| ignite | The process is executed using the Apache Ignite cluster. | \n| k8s | The process is executed using the Kubernetes cluster. | \n| local | The process is executed in the computer where Nextflow is launched. | \n| lsf | The process is executed using the Platform LSF job scheduler. | \n| moab | The process is executed using the Moab job scheduler. | \n| nqsii | The process is executed using the NQSII job scheduler. | \n| oge | Alias for the sge executor. | \n| pbs | The process is executed using the PBS/Torque job scheduler. | \n| pbspro | The process is executed using the PBS Pro job scheduler. | \n| sge | The process is executed using the Sun Grid Engine / Open Grid Engine. | \n| slurm | The process is executed using the SLURM job scheduler. | \n| tes | The process is executed using the GA4GH TES service. | \n| uge | Alias for the sge executor. |\n\nSee [`executor`](https://www.nextflow.io/docs/latest/process.html#executor).\n", - "example" : [ - { - "example" : "\"local\"", - "format" : "yaml" - }, - { - "example" : "\"sge\"", - "format" : "yaml" - } - ] - }, - { - "name" : "containerOptions", - "type" : "OneOrMore of String", - "description" : "The `containerOptions` directive allows you to specify any container execution option supported by the underlying container engine (ie. Docker, Singularity, etc). This can be useful to provide container settings only for a specific process e.g. mount a custom path.\n\nSee [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#containeroptions).\n", - "example" : [ - { - "example" : "\"--foo bar\"", - "format" : "yaml" - }, - { - "example" : "[\"--foo bar\", \"-f b\"]", - "format" : "yaml" - } - ] - }, - { - "name" : "disk", - "type" : "Option of String", - "description" : "The `disk` directive allows you to define how much local disk storage the process is allowed to use.\n\nSee [`disk`](https://www.nextflow.io/docs/latest/process.html#disk).\n", - "example" : [ - { - "example" : "\"1 GB\"", - "format" : "yaml" - }, - { - "example" : "\"2TB\"", - "format" : "yaml" - }, - { - "example" : "\"3.2KB\"", - "format" : "yaml" - }, - { - "example" : "\"10.B\"", - "format" : "yaml" - } - ] - }, - { - "name" : "tag", - "type" : "Option of String", - "description" : "The `tag` directive allows you to associate each process execution with a custom label, so that it will be easier to identify them in the log file or in the trace execution report.\n\nSee [`tag`](https://www.nextflow.io/docs/latest/process.html#tag).\n", - "example" : [ - { - "example" : "\"foo\"", - "format" : "yaml" - }, - { - "example" : "'$id'", - "format" : "yaml" - } - ] - }, - { - "name" : "conda", - "type" : "OneOrMore of String", - "description" : "The `conda` directive allows for the definition of the process dependencies using the Conda package manager.\n\nNextflow automatically sets up an environment for the given package names listed by in the `conda` directive.\n\nSee [`conda`](https://www.nextflow.io/docs/latest/process.html#conda).\n", - "example" : [ - { - "example" : "\"bwa=0.7.15\"", - "format" : "yaml" - }, - { - "example" : "\"bwa=0.7.15 fastqc=0.11.5\"", - "format" : "yaml" - }, - { - "example" : "[\"bwa=0.7.15\", \"fastqc=0.11.5\"]", - "format" : "yaml" - } - ] - }, - { - "name" : "machineType", - "type" : "Option of String", - "description" : " The `machineType` can be used to specify a predefined Google Compute Platform machine type when running using the Google Life Sciences executor.\n\nSee [`machineType`](https://www.nextflow.io/docs/latest/process.html#machinetype).\n", - "example" : [ - { - "example" : "\"n1-highmem-8\"", - "format" : "yaml" - } - ] - }, - { - "name" : "stageInMode", - "type" : "Option of String", - "description" : "The `stageInMode` directive defines how input files are staged-in to the process work directory. The following values are allowed:\n\n| Value | Description |\n|-------|-------------| \n| copy | Input files are staged in the process work directory by creating a copy. | \n| link | Input files are staged in the process work directory by creating an (hard) link for each of them. | \n| symlink | Input files are staged in the process work directory by creating a symbolic link with an absolute path for each of them (default). | \n| rellink | Input files are staged in the process work directory by creating a symbolic link with a relative path for each of them. | \n\nSee [`stageInMode`](https://www.nextflow.io/docs/latest/process.html#stageinmode).\n", - "example" : [ - { - "example" : "\"copy\"", - "format" : "yaml" - }, - { - "example" : "\"link\"", - "format" : "yaml" - } - ] - }, - { - "name" : "cache", - "type" : "Option[Either of Boolean,String]", - "description" : "The `cache` directive allows you to store the process results to a local cache. When the cache is enabled and the pipeline is launched with the resume option, any following attempt to execute the process, along with the same inputs, will cause the process execution to be skipped, producing the stored data as the actual results.\n\nThe caching feature generates a unique key by indexing the process script and inputs. This key is used to identify univocally the outputs produced by the process execution.\n\nThe `cache` is enabled by default, you can disable it for a specific process by setting the cache directive to `false`.\n\nAccepted values are: `true`, `false`, `\"deep\"`, and `\"lenient\"`.\n\nSee [`cache`](https://www.nextflow.io/docs/latest/process.html#cache).\n", - "example" : [ - { - "example" : "true", - "format" : "yaml" - }, - { - "example" : "false", - "format" : "yaml" - }, - { - "example" : "\"deep\"", - "format" : "yaml" - }, - { - "example" : "\"lenient\"", - "format" : "yaml" - } - ] - }, - { - "name" : "pod", - "type" : "OneOrMore[Map of String,String]", - "description" : "The `pod` directive allows the definition of pods specific settings, such as environment variables, secrets and config maps when using the Kubernetes executor.\n\nSee [`pod`](https://www.nextflow.io/docs/latest/process.html#pod).\n", - "example" : [ - { - "example" : "[ label: \"key\", value: \"val\" ]", - "format" : "yaml" - }, - { - "example" : "[ annotation: \"key\", value: \"val\" ]", - "format" : "yaml" - }, - { - "example" : "[ env: \"key\", value: \"val\" ]", - "format" : "yaml" - }, - { - "example" : "[ [label: \"l\", value: \"v\"], [env: \"e\", value: \"v\"]]", - "format" : "yaml" - } - ] - }, - { - "name" : "penv", - "type" : "Option of String", - "description" : "The `penv` directive allows you to define the parallel environment to be used when submitting a parallel task to the SGE resource manager.\n\nSee [`penv`](https://www.nextflow.io/docs/latest/process.html#penv).\n", - "example" : [ - { - "example" : "\"smp\"", - "format" : "yaml" - } - ] - }, - { - "name" : "scratch", - "type" : "Option[Either of Boolean,String]", - "description" : "The `scratch` directive allows you to execute the process in a temporary folder that is local to the execution node.\n\nSee [`scratch`](https://www.nextflow.io/docs/latest/process.html#scratch).\n", - "example" : [ - { - "example" : "true", - "format" : "yaml" - }, - { - "example" : "\"/path/to/scratch\"", - "format" : "yaml" - }, - { - "example" : "'$MY_PATH_TO_SCRATCH'", - "format" : "yaml" - }, - { - "example" : "\"ram-disk\"", - "format" : "yaml" - } - ] - }, - { - "name" : "storeDir", - "type" : "Option of String", - "description" : "The `storeDir` directive allows you to define a directory that is used as a permanent cache for your process results.\n\nSee [`storeDir`](https://www.nextflow.io/docs/latest/process.html#storeDir).\n", - "example" : [ - { - "example" : "\"/path/to/storeDir\"", - "format" : "yaml" - } - ] - }, - { - "name" : "maxRetries", - "type" : "Option[Either of String,Int]", - "description" : "The `maxRetries` directive allows you to define the maximum number of times a process instance can be re-submitted in case of failure. This value is applied only when using the retry error strategy. By default only one retry is allowed.\n\nSee [`maxRetries`](https://www.nextflow.io/docs/latest/process.html#maxretries).\n", - "example" : [ - { - "example" : "1", - "format" : "yaml" - }, - { - "example" : "3", - "format" : "yaml" - } - ] - }, - { - "name" : "echo", - "type" : "Option[Either of Boolean,String]", - "description" : "By default the stdout produced by the commands executed in all processes is ignored. By setting the `echo` directive to true, you can forward the process stdout to the current top running process stdout file, showing it in the shell terminal.\n \nSee [`echo`](https://www.nextflow.io/docs/latest/process.html#echo).\n", - "example" : [ - { - "example" : "true", - "format" : "yaml" - }, - { - "example" : "false", - "format" : "yaml" - } - ] - }, - { - "name" : "errorStrategy", - "type" : "Option of String", - "description" : "The `errorStrategy` directive allows you to define how an error condition is managed by the process. By default when an error status is returned by the executed script, the process stops immediately. This in turn forces the entire pipeline to terminate.\n\nTable of available error strategies:\n| Name | Executor |\n|------|----------|\n| `terminate` | Terminates the execution as soon as an error condition is reported. Pending jobs are killed (default) |\n| `finish` | Initiates an orderly pipeline shutdown when an error condition is raised, waiting the completion of any submitted job. |\n| `ignore` | Ignores processes execution errors. |\n| `retry` | Re-submit for execution a process returning an error condition. |\n\nSee [`errorStrategy`](https://www.nextflow.io/docs/latest/process.html#errorstrategy).\n", - "example" : [ - { - "example" : "\"terminate\"", - "format" : "yaml" - }, - { - "example" : "\"finish\"", - "format" : "yaml" - } - ] - }, - { - "name" : "memory", - "type" : "Option of String", - "description" : "The `memory` directive allows you to define how much memory the process is allowed to use.\n\nSee [`memory`](https://www.nextflow.io/docs/latest/process.html#memory).\n", - "example" : [ - { - "example" : "\"1 GB\"", - "format" : "yaml" - }, - { - "example" : "\"2TB\"", - "format" : "yaml" - }, - { - "example" : "\"3.2KB\"", - "format" : "yaml" - }, - { - "example" : "\"10.B\"", - "format" : "yaml" - } - ] - }, - { - "name" : "stageOutMode", - "type" : "Option of String", - "description" : "The `stageOutMode` directive defines how output files are staged-out from the scratch directory to the process work directory. The following values are allowed:\n\n| Value | Description |\n|-------|-------------| \n| copy | Output files are copied from the scratch directory to the work directory. | \n| move | Output files are moved from the scratch directory to the work directory. | \n| rsync | Output files are copied from the scratch directory to the work directory by using the rsync utility. |\n\nSee [`stageOutMode`](https://www.nextflow.io/docs/latest/process.html#stageoutmode).\n", - "example" : [ - { - "example" : "\"copy\"", - "format" : "yaml" - }, - { - "example" : "\"link\"", - "format" : "yaml" - } - ] - } - ], - "nextflowAuto" : [ - { - "name" : "__this__", - "type" : "NextflowAuto", - "hierarchy" : [ - "io.viash.platforms.nextflow.NextflowAuto" - ], - "description" : "Automated processing flags which can be toggled on or off." - }, - { - "name" : "simplifyInput", - "type" : "Boolean", - "description" : "If `true`, an input tuple only containing only a single File (e.g. `[\"foo\", file(\"in.h5ad\")]`) is automatically transformed to a map (i.e. `[\"foo\", [ input: file(\"in.h5ad\") ] ]`).\n\nDefault: `true`.\n" - }, - { - "name" : "simplifyOutput", - "type" : "Boolean", - "description" : "If `true`, an output tuple containing a map with a File (e.g. `[\"foo\", [ output: file(\"out.h5ad\") ] ]`) is automatically transformed to a map (i.e. `[\"foo\", file(\"out.h5ad\")]`).\n\nDefault: `true`.\n" - }, - { - "name" : "publish", - "type" : "Boolean", - "description" : "If `true`, the module's outputs are automatically published to `params.publishDir`.\nWill throw an error if `params.publishDir` is not defined.\n\nDefault: `false`.\n" - }, - { - "name" : "transcript", - "type" : "Boolean", - "description" : "If `true`, the module's transcripts from `work/` are automatically published to `params.transcriptDir`.\nIf not defined, `params.publishDir + \"/_transcripts\"` will be used.\nWill throw an error if neither are defined.\n\nDefault: `false`.\n" - } - ], - "nextflowConfig" : [ - { - "name" : "__this__", - "type" : "NextflowConfig", - "hierarchy" : [ - "io.viash.platforms.nextflow.NextflowConfig" - ], - "description" : "Allows tweaking how the Nextflow Config file is generated.", - "since" : "Viash 0.7.4" - }, - { - "name" : "labels", - "type" : "ListMap of String,String", - "description" : "A series of default labels to specify memory and cpu constraints.\n\nThe default memory labels are defined as \"mem1gb\", \"mem2gb\", \"mem4gb\", ... upto \"mem512tb\" and follows powers of 2.\nThe default cpu labels are defined as \"cpu1\", \"cpu2\", \"cpu5\", \"cpu10\", ... upto \"cpu1000\" and follows a semi logarithmic scale (1, 2, 5 per decade).\n\nConceptually it is possible for a Viash Config to overwrite the full labels parameter, however likely it is more efficient to add additional labels\nin the Viash Project with a config mod.\n", - "example" : [ - { - "example" : "labels:\n lowmem: \"memory = 4.GB\"\n lowcpu: \"cpus = 4\"\n midmem: \"memory = 25.GB\"\n midcpu: \"cpus = 10\"\n highmem: \"memory = 50.GB\"\n highcpu: \"cpus = 20\"\n vhighmem: \"memory = 100.GB\"\n vhighcpu: \"cpus = 40\"\n", - "format" : "yaml", - "description" : "Replace the default labels with a different set of labels" - }, - { - "example" : "-c '.platforms[.type == \"nextflow\"].config.labels.lowmem := \"memory = 4.GB\";.platforms[.type == \"nextflow\"].config.labels.lowcpu := \"cpus = 4\"'", - "format" : "viash_config_mod", - "description" : "Add 'lowmem' and 'lowcpu' to the default labels by using a config mod" - }, - { - "example" : "config_mods: |\n .platforms[.type == \"nextflow\"].config.labels.lowmem := \"memory = 4.GB\"\n .platforms[.type == \"nextflow\"].config.labels.lowcpu := \"cpus = 4\"\n", - "format" : "viash_project_file", - "description" : "Add 'lowmem' and 'lowcpu' to the default labels by using the Viash Project file" - }, - { - "example" : "config_mods: |\n .platforms[.type == \"nextflow\"].config.labels := { lowmem: \"memory = 4.GB\", lowcpu: \"cpus = 4\", midmem: \"memory = 25.GB\", midcpu: \"cpus = 10\", highmem: \"memory = 50.GB\", highcpu: \"cpus = 20\", vhighmem: \"memory = 100.GB\", vhighcpu: \"cpus = 40\" }\n", - "format" : "viash_project_file", - "description" : "Replace the default labels with a different set of labels by using the Viash Project file" - } - ] - }, - { - "name" : "script", - "type" : "OneOrMore of String", - "description" : "Includes a single string or list of strings into the nextflow.config file.\nThis can be used to add custom profiles or include an additional config file.\n", - "example" : [ - { - "example" : "script:\n - |\n profiles {\n ...\n }\n", - "format" : "yaml" - }, - { - "example" : "script: includeConfig(\"config.config\")", - "format" : "yaml" - } - ] - } - ] - } -} \ No newline at end of file + ], + "since" : "Viash 0.5.14", + "default" : "Empty" + }, + { + "name" : "description", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "A description of the component. This will be displayed with `--help`.", + "example" : [ + { + "example" : "description: |\n This component performs function Y and Z.\n It is possible to make this a multiline string.\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "usage", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "A description on how to use the component. This will be displayed with `--help` under the 'Usage:' section.", + "example" : [ + { + "example" : "usage: Place the executable in a directory containing TSV files and run it", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "namespace", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Namespace this component is a part of. See the @[Namespaces guide](namespace) for more information on namespaces.", + "example" : [ + { + "example" : "namespace: fancy_components", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "arguments", + "type" : "List[Argument]", + "niceType" : "List of Argument", + "description" : "A list of @[arguments](argument) for this component. For each argument, a type and a name must be specified. Depending on the type of argument, different properties can be set. See these reference pages per type for more information: \n\n - @[string](arg_string)\n - @[file](arg_file)\n - @[integer](arg_integer)\n - @[double](arg_double)\n - @[boolean](arg_boolean)\n - @[boolean_true](arg_boolean_true)\n - @[boolean_false](arg_boolean_false)\n", + "example" : [ + { + "example" : "arguments:\n - name: --foo\n type: file\n alternatives: [-f]\n description: Description of foo\n default: \"/foo/bar\"\n must_exist: true\n direction: output\n required: false\n multiple: true\n multiple_sep: \",\"\n - name: --bar\n type: string\n", + "format" : "yaml" + } + ], + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "Author", + "niceType" : "Author", + "hierarchy" : [ + "io.viash.functionality.Author" + ], + "description" : "Author metadata.", + "example" : [ + { + "example" : "name: Jane Doe\nrole: [author, maintainer]\nemail: jane@doe.com\ninfo:\n github: janedoe\n twitter: janedoe\n orcid: XXAABBCCXX\n groups: [ one, two, three ]\n", + "format" : "yaml" + } + ], + "since" : "Viash 0.3.2" + }, + { + "name" : "name", + "type" : "String", + "niceType" : "String", + "description" : "Full name of the author, usually in the name of FirstName MiddleName LastName." + }, + { + "name" : "email", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "E-mail of the author.", + "default" : "Empty" + }, + { + "name" : "info", + "type" : "Json", + "niceType" : "Json", + "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", + "since" : "Viash 0.7.4", + "default" : "Empty" + }, + { + "name" : "roles", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Role of the author. Suggested items:\n\n* `\"author\"`: Authors who have made substantial contributions to the component.\n* `\"maintainer\"`: The maintainer of the component.\n* `\"contributor\"`: Authors who have made smaller contributions (such as code patches etc.).\n", + "default" : "Empty" + }, + { + "name" : "props", + "type" : "Map[String,String]", + "niceType" : "Map of String to String", + "description" : "Author properties. Must be a map of strings.", + "deprecated" : { + "message" : "Use `info` instead.", + "deprecation" : "0.7.4", + "removal" : "0.8.0" + }, + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "ComputationalRequirements", + "niceType" : "ComputationalRequirements", + "hierarchy" : [ + "io.viash.functionality.ComputationalRequirements" + ], + "description" : "Computational requirements related to running the component.", + "since" : "Viash 0.6.0" + }, + { + "name" : "cpus", + "type" : "Option[Int]", + "niceType" : "Option of Int", + "description" : "The maximum number of (logical) cpus a component is allowed to use.", + "example" : [ + { + "example" : "cpus: 10", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "commands", + "type" : "List[String]", + "niceType" : "List of String", + "description" : "A list of commands which should be present on the system for the script to function.", + "example" : [ + { + "example" : "commands: [ which, bash, awk, date, grep, egrep, ps, sed, tail, tee ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "memory", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The maximum amount of memory a component is allowed to allocate. Unit must be one of B, KB, MB, GB, TB or PB.", + "example" : [ + { + "example" : "memory: 10GB", + "format" : "yaml" + } + ], + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "ArgumentGroup", + "niceType" : "ArgumentGroup", + "hierarchy" : [ + "io.viash.functionality.ArgumentGroup" + ], + "description" : "A grouping of the @[arguments](argument), used to display the help message." + }, + { + "name" : "name", + "type" : "String", + "niceType" : "String", + "description" : "The name of the argument group." + }, + { + "name" : "description", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Description of foo`, a description of the argument group. Multiline descriptions are supported.", + "default" : "Empty" + }, + { + "name" : "arguments", + "type" : "List[Argument]", + "niceType" : "List of Argument", + "description" : "List of arguments." + } + ], + [ + { + "name" : "__this__", + "type" : "Platform", + "niceType" : "Platform", + "hierarchy" : [ + "io.viash.platforms.Platform" + ], + "description" : "A list of platforms to generate target artifacts for.\n\n * @[Native](platform_native)\n * @[Docker](platform_docker)\n * @[Nextflow](platform_nextflow)\n", + "example" : [ + { + "example" : "platforms:\n - type: docker\n image: \"bash:4.0\"\n - type: native\n - type: nextflow\n directives:\n label: [lowcpu, midmem]\n", + "format" : "yaml" + } + ], + "subclass" : [ + "NativePlatform", + "DockerPlatform", + "NextflowPlatform" + ] + } + ], + [ + { + "name" : "__this__", + "type" : "NativePlatform", + "niceType" : "NativePlatform", + "hierarchy" : [ + "io.viash.platforms.NativePlatform", + "io.viash.platforms.Platform" + ], + "description" : "Running a Viash component on a native platform means that the script will be executed in your current environment.\nAny dependencies are assumed to have been installed by the user, so the native platform is meant for developers (who know what they're doing) or for simple bash scripts (which have no extra dependencies).\n", + "example" : [ + { + "example" : "platforms:\n - type: native\n", + "format" : "yaml" + } + ], + "subclass" : [ + "native" + ] + }, + { + "name" : "id", + "type" : "String", + "niceType" : "String", + "description" : "As with all platforms, you can give a platform a different name. By specifying `id: foo`, you can target this platform (only) by specifying `-p foo` in any of the Viash commands.", + "example" : [ + { + "example" : "id: foo", + "format" : "yaml" + } + ], + "default" : "native" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the platform." + } + ], + [ + { + "name" : "__this__", + "type" : "DockerPlatform", + "niceType" : "DockerPlatform", + "hierarchy" : [ + "io.viash.platforms.DockerPlatform", + "io.viash.platforms.Platform" + ], + "description" : "Run a Viash component on a Docker backend platform.\nBy specifying which dependencies your component needs, users will be able to build a docker container from scratch using the setup flag, or pull it from a docker repository.\n", + "example" : [ + { + "example" : "platforms:\n - type: docker\n image: \"bash:4.0\"\n setup:\n - type: apt\n packages: [ curl ]\n", + "format" : "yaml" + } + ], + "subclass" : [ + "docker" + ] + }, + { + "name" : "organization", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Name of a container's [organization](https://docs.docker.com/docker-hub/orgs/).", + "default" : "Empty" + }, + { + "name" : "registry", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The URL to the a [custom Docker registry](https://docs.docker.com/registry/)", + "example" : [ + { + "example" : "registry: https://my-docker-registry.org", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "image", + "type" : "String", + "niceType" : "String", + "description" : "The base container to start from. You can also add the tag here if you wish.", + "example" : [ + { + "example" : "image: \"bash:4.0\"", + "format" : "yaml" + } + ] + }, + { + "name" : "tag", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Specify a Docker image based on its tag.", + "example" : [ + { + "example" : "tag: 4.0", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "target_tag", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The tag the resulting image gets. Advanced usage only.", + "example" : [ + { + "example" : "target_tag: 0.5.0", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "run_args", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Add [docker run](https://docs.docker.com/engine/reference/run/) arguments.", + "default" : "Empty" + }, + { + "name" : "namespace_separator", + "type" : "String", + "niceType" : "String", + "description" : "The separator between the namespace and the name of the component, used for determining the image name. Default: `\"/\"`.", + "example" : [ + { + "example" : "namespace_separator: \"_\"", + "format" : "yaml" + } + ], + "default" : "/" + }, + { + "name" : "resolve_volume", + "type" : "DockerResolveVolume", + "niceType" : "DockerResolveVolume", + "description" : "Enables or disables automatic volume mapping. Enabled when set to `Automatic` or disabled when set to `Manual`. Default: `Automatic`.", + "default" : "Automatic" + }, + { + "name" : "id", + "type" : "String", + "niceType" : "String", + "description" : "As with all platforms, you can give a platform a different name. By specifying `id: foo`, you can target this platform (only) by specifying `-p foo` in any of the Viash commands.", + "example" : [ + { + "example" : "id: foo", + "format" : "yaml" + } + ], + "default" : "docker" + }, + { + "name" : "port", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "A list of enabled ports. This doesn't change the Dockerfile but gets added as a command-line argument at runtime.", + "example" : [ + { + "example" : "port:\n - 80\n - 8080\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "setup", + "type" : "List[Requirements]", + "niceType" : "List of Requirements", + "description" : "A list of requirements for installing the following types of packages:\n\n - @[apt](apt_req)\n - @[apk](apk_req)\n - @[Docker setup instructions](docker_req)\n - @[JavaScript](javascript_req)\n - @[Python](python_req)\n - @[R](r_req)\n - @[Ruby](ruby_req)\n - @[yum](yum_req)\n\nThe order in which these dependencies are specified determines the order in which they will be installed.\n", + "default" : "Empty" + }, + { + "name" : "workdir", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The working directory when starting the container. This doesn't change the Dockerfile but gets added as a command-line argument at runtime.", + "example" : [ + { + "example" : "workdir: /home/user", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "target_image", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "If anything is specified in the setup section, running the `---setup` will result in an image with the name of `:`. If nothing is specified in the `setup` section, simply `image` will be used. Advanced usage only.", + "example" : [ + { + "example" : "target_image: myfoo", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "cmd", + "type" : "Option[Either[String,List[String]]]", + "niceType" : "Option of Either\n - String\n - List of String", + "description" : "Set the default command being executed when running the Docker container.", + "example" : [ + { + "example" : "cmd: [\"echo\", \"$HOME\"]", + "format" : "yaml", + "description" : "Set CMD using the exec format, which is the prefered form." + }, + { + "example" : "cmd: \"echo $HOME\"", + "format" : "yaml", + "description" : "Set CMD using the shell format." + } + ], + "since" : "Viash 0.7.4", + "default" : "Empty" + }, + { + "name" : "target_image_source", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The source of the target image. This is used for defining labels in the dockerfile.", + "example" : [ + { + "example" : "target_image_source: https://github.com/foo/bar", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "test_setup", + "type" : "List[Requirements]", + "niceType" : "List of Requirements", + "description" : "Additional requirements specific for running unit tests.", + "since" : "Viash 0.5.13", + "default" : "Empty" + }, + { + "name" : "entrypoint", + "type" : "Option[Either[String,List[String]]]", + "niceType" : "Option of Either\n - String\n - List of String", + "description" : "Override the entrypoint of the base container. Default set `ENTRYPOINT []`.", + "example" : [ + { + "example" : "entrypoint: ", + "format" : "yaml", + "description" : "Disable the default override." + }, + { + "example" : "entrypoint: [\"top\", \"-b\"]", + "format" : "yaml", + "description" : "Entrypoint of the container in the exec format, which is the prefered form." + }, + { + "example" : "entrypoint: \"top -b\"", + "format" : "yaml", + "description" : "Entrypoint of the container in the shell format." + } + ], + "since" : "Viash 0.7.4", + "default" : "[]" + }, + { + "name" : "target_registry", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The URL where the resulting image will be pushed to. Advanced usage only.", + "example" : [ + { + "example" : "target_registry: https://my-docker-registry.org", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "setup_strategy", + "type" : "DockerSetupStrategy", + "niceType" : "DockerSetupStrategy", + "description" : "The Docker setup strategy to use when building a container.\n\n| Strategy | Description |\n|-----|----------|\n| `alwaysbuild` / `build` / `b` | Always build the image from the dockerfile. This is the default setup strategy.\n| `alwayscachedbuild` / `cachedbuild` / `cb` | Always build the image from the dockerfile, with caching enabled.\n| `ifneedbebuild` | Build the image if it does not exist locally.\n| `ifneedbecachedbuild` | Build the image with caching enabled if it does not exist locally, with caching enabled.\n| `alwayspull` / `pull` / `p` | Try to pull the container from [Docker Hub](https://hub.docker.com) or the @[specified docker registry](docker_registry).\n| `alwayspullelsebuild` / `pullelsebuild` | Try to pull the image from a registry and build it if it doesn't exist.\n| `alwayspullelsecachedbuild` / `pullelsecachedbuild` | Try to pull the image from a registry and build it with caching if it doesn't exist.\n| `ifneedbepull` | If the image does not exist locally, pull the image.\n| `ifneedbepullelsebuild` | If the image does not exist locally, pull the image. If the image does exist, build it.\n| `ifneedbepullelsecachedbuild` | If the image does not exist locally, pull the image. If the image does exist, build it with caching enabled.\n| `push` | Push the container to [Docker Hub](https://hub.docker.com) or the @[specified docker registry](docker_registry).\n| `pushifnotpresent` | Push the container to [Docker Hub](https://hub.docker.com) or the @[specified docker registry](docker_registry) if the @[tag](docker_tag) does not exist yet.\n| `donothing` / `meh` | Do not build or pull anything.\n\n", + "example" : [ + { + "example" : "setup_strategy: alwaysbuild", + "format" : "yaml" + } + ], + "default" : "ifneedbepullelsecachedbuild" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the platform." + }, + { + "name" : "target_organization", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The organization set in the resulting image. Advanced usage only.", + "example" : [ + { + "example" : "target_organization: viash-io", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "chown", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "In Linux, files created by a Docker container will be owned by `root`. With `chown: true`, Viash will automatically change the ownership of output files (arguments with `type: file` and `direction: output`) to the user running the Viash command after execution of the component. Default value: `true`.", + "example" : [ + { + "example" : "chown: false", + "format" : "yaml" + } + ], + "default" : "True" + } + ], + [ + { + "name" : "__this__", + "type" : "NextflowPlatform", + "niceType" : "NextflowPlatform", + "hierarchy" : [ + "io.viash.platforms.NextflowPlatform", + "io.viash.platforms.Platform" + ], + "description" : "Platform for generating Nextflow VDSL3 modules.", + "example" : [ + { + "example" : "platforms:\n - type: nextflow\n directives:\n label: [lowcpu, midmem]\n", + "format" : "yaml" + } + ], + "subclass" : [ + "nextflow" + ] + }, + { + "name" : "auto", + "type" : "NextflowAuto", + "niceType" : "NextflowAuto", + "description" : "@[Automated processing flags](nextflow_auto) which can be toggled on or off:\n\n| Flag | Description | Default |\n|---|---------|----|\n| `simplifyInput` | If `true`, an input tuple only containing only a single File (e.g. `[\"foo\", file(\"in.h5ad\")]`) is automatically transformed to a map (i.e. `[\"foo\", [ input: file(\"in.h5ad\") ] ]`). | `true` |\n| `simplifyOutput` | If `true`, an output tuple containing a map with a File (e.g. `[\"foo\", [ output: file(\"out.h5ad\") ] ]`) is automatically transformed to a map (i.e. `[\"foo\", file(\"out.h5ad\")]`). | `true` |\n| `transcript` | If `true`, the module's transcripts from `work/` are automatically published to `params.transcriptDir`. If not defined, `params.publishDir + \"/_transcripts\"` will be used. Will throw an error if neither are defined. | `false` |\n| `publish` | If `true`, the module's outputs are automatically published to `params.publishDir`. Will throw an error if `params.publishDir` is not defined. | `false` |\n\n", + "example" : [ + { + "example" : "auto:\n publish: true", + "format" : "yaml" + } + ], + "default" : "simplifyInput: true\nsimplifyOutput: true\ntranscript: false\npublish: false\n" + }, + { + "name" : "directives", + "type" : "NextflowDirectives", + "niceType" : "NextflowDirectives", + "description" : "@[Directives](nextflow_directives) are optional settings that affect the execution of the process. These mostly match up with the Nextflow counterparts. \n", + "example" : [ + { + "example" : "directives:\n container: rocker/r-ver:4.1\n label: highcpu\n cpus: 4\n memory: 16 GB", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "container", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the Docker platform id to be used to run Nextflow.", + "default" : "docker" + }, + { + "name" : "config", + "type" : "NextflowConfig", + "niceType" : "NextflowConfig", + "description" : "Allows tweaking how the @[Nextflow Config](nextflow_config) file is generated.", + "since" : "Viash 0.7.4", + "default" : "A series of default labels to specify memory and cpu constraints" + }, + { + "name" : "debug", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Whether or not to print debug messages.", + "default" : "False" + }, + { + "name" : "id", + "type" : "String", + "niceType" : "String", + "description" : "Every platform can be given a specific id that can later be referred to explicitly when running or building the Viash component.", + "example" : [ + { + "example" : "id: foo", + "format" : "yaml" + } + ], + "default" : "nextflow" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the platform." + } + ], + [ + { + "name" : "__this__", + "type" : "Requirements", + "niceType" : "Requirements", + "hierarchy" : [ + "io.viash.platforms.requirements.Requirements" + ], + "description" : "Requirements for installing the following types of packages:\n\n - @[apt](apt_req)\n - @[apk](apk_req)\n - @[Docker setup instructions](docker_req)\n - @[JavaScript](javascript_req)\n - @[Python](python_req)\n - @[R](r_req)\n - @[Ruby](ruby_req)\n - @[yum](yum_req)\n", + "subclass" : [ + "ApkRequirements", + "AptRequirements", + "DockerRequirements", + "JavaScriptRequirements", + "PythonRequirements", + "RRequirements", + "RubyRequirements", + "YumRequirements" + ] + } + ], + [ + { + "name" : "__this__", + "type" : "ApkRequirements", + "niceType" : "ApkRequirements", + "hierarchy" : [ + "io.viash.platforms.requirements.ApkRequirements", + "io.viash.platforms.requirements.Requirements" + ], + "description" : "Specify which apk packages should be available in order to run the component.", + "example" : [ + { + "example" : "setup:\n - type: apk\n packages: [ sl ]\n", + "format" : "yaml" + } + ], + "subclass" : [ + "apk" + ] + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the requirement specification." + }, + { + "name" : "packages", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install.", + "example" : [ + { + "example" : "packages: [ sl ]", + "format" : "yaml" + } + ], + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "AptRequirements", + "niceType" : "AptRequirements", + "hierarchy" : [ + "io.viash.platforms.requirements.AptRequirements", + "io.viash.platforms.requirements.Requirements" + ], + "description" : "Specify which apt packages should be available in order to run the component.", + "example" : [ + { + "example" : "setup:\n - type: apt\n packages: [ sl ]\n", + "format" : "yaml" + } + ], + "subclass" : [ + "apt" + ] + }, + { + "name" : "interactive", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "If `false`, the Debian frontend is set to non-interactive (recommended). Default: false.", + "default" : "False" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the requirement specification." + }, + { + "name" : "packages", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install.", + "example" : [ + { + "example" : "packages: [ sl ]", + "format" : "yaml" + } + ], + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "DockerRequirements", + "niceType" : "DockerRequirements", + "hierarchy" : [ + "io.viash.platforms.requirements.DockerRequirements", + "io.viash.platforms.requirements.Requirements" + ], + "description" : "Specify which Docker commands should be run during setup.", + "example" : [ + { + "example" : "setup:\n - type: docker\n build_args: \"R_VERSION=hello_world\"\n run: |\n echo 'Run a custom command'\n echo 'Foo' > /path/to/file.txt", + "format" : "yaml" + } + ], + "subclass" : [ + "docker" + ] + }, + { + "name" : "run", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which `RUN` entries to add to the Dockerfile while building it.", + "example" : [ + { + "example" : "run: |\n echo 'Run a custom command'\n echo 'Foo' > /path/to/file.txt", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "label", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which `LABEL` entries to add to the Dockerfile while building it.", + "example" : [ + { + "example" : "label: [ component=\"foo\" ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "build_args", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which `ARG` entries to add to the Dockerfile while building it.", + "example" : [ + { + "example" : "build_args: [ \"R_VERSION=4.2\" ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "copy", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which `COPY` entries to add to the Dockerfile while building it.", + "example" : [ + { + "example" : "copy: [ \"resource.txt /path/to/resource.txt\" ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the requirement specification." + }, + { + "name" : "add", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which `ADD` entries to add to the Dockerfile while building it.", + "example" : [ + { + "example" : "add: [ \"http://foo/bar .\" ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "env", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which `ENV` entries to add to the Dockerfile while building it. Unlike `ARG`, `ENV` entries are also accessible from inside the container.", + "example" : [ + { + "example" : "env: [ \"R_VERSION=4.2\" ]", + "format" : "yaml" + } + ], + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "JavaScriptRequirements", + "niceType" : "JavaScriptRequirements", + "hierarchy" : [ + "io.viash.platforms.requirements.JavaScriptRequirements", + "io.viash.platforms.requirements.Requirements" + ], + "description" : "Specify which JavaScript packages should be available in order to run the component.", + "example" : [ + { + "example" : "setup:\n - type: javascript\n npm: packagename\n git: \"https://some.git.repository/org/repo\"\n github: \"owner/repository\"\n url: \"https://github.com/org/repo/archive/HEAD.zip\"\n", + "format" : "yaml" + } + ], + "subclass" : [ + "javascript" + ] + }, + { + "name" : "github", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install from GitHub.", + "example" : [ + { + "example" : "github: [ owner/repository ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "url", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install using a generic URI.", + "example" : [ + { + "example" : "url: [ https://github.com/org/repo/archive/HEAD.zip ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "git", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install using a Git URI.", + "example" : [ + { + "example" : "git: [ https://some.git.repository/org/repo ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "npm", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install from npm.", + "example" : [ + { + "example" : "npm: [ packagename ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the requirement specification." + }, + { + "name" : "packages", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install from npm.", + "example" : [ + { + "example" : "packages: [ packagename ]", + "format" : "yaml" + } + ], + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "PythonRequirements", + "niceType" : "PythonRequirements", + "hierarchy" : [ + "io.viash.platforms.requirements.PythonRequirements", + "io.viash.platforms.requirements.Requirements" + ], + "description" : "Specify which Python packages should be available in order to run the component.", + "example" : [ + { + "example" : "setup:\n - type: python\n pip: numpy\n github: [ jkbr/httpie, foo/bar ]\n url: \"https://github.com/some_org/some_pkg/zipball/master\"\n", + "format" : "yaml" + } + ], + "subclass" : [ + "python" + ] + }, + { + "name" : "github", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install from GitHub.", + "example" : [ + { + "example" : "github: [ jkbr/httpie ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "gitlab", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install from GitLab.", + "example" : [ + { + "example" : "gitlab: [ foo/bar ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "pip", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install from pip.", + "example" : [ + { + "example" : "pip: [ numpy ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "pypi", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install from PyPI using pip.", + "example" : [ + { + "example" : "pypi: [ numpy ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "git", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install using a Git URI.", + "example" : [ + { + "example" : "git: [ https://some.git.repository/org/repo ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "upgrade", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Sets the `--upgrade` flag when set to true. Default: true.", + "default" : "True" + }, + { + "name" : "packages", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install from pip.", + "example" : [ + { + "example" : "packages: [ numpy ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "url", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install using a generic URI.", + "example" : [ + { + "example" : "url: [ https://github.com/some_org/some_pkg/zipball/master ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "svn", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install using an SVN URI.", + "example" : [ + { + "example" : "svn: [ http://svn.repo/some_pkg/trunk/#egg=SomePackage ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "bazaar", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install using a Bazaar URI.", + "example" : [ + { + "example" : "bazaar: [ http://bazaar.launchpad.net/some_pkg/some_pkg/release-0.1 ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "script", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies a code block to run as part of the build.", + "example" : [ + { + "example" : "script: |\n print(\"Running custom code\")\n x = 1 + 1 == 2", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the requirement specification." + }, + { + "name" : "mercurial", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install using a Mercurial URI.", + "example" : [ + { + "example" : "mercurial: [ https://hg.myproject.org/MyProject/#egg=MyProject ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "user", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Sets the `--user` flag when set to true. Default: false.", + "default" : "False" + } + ], + [ + { + "name" : "__this__", + "type" : "RRequirements", + "niceType" : "RRequirements", + "hierarchy" : [ + "io.viash.platforms.requirements.RRequirements", + "io.viash.platforms.requirements.Requirements" + ], + "description" : "Specify which R packages should be available in order to run the component.", + "example" : [ + { + "example" : "setup: \n - type: r\n cran: anndata\n bioc: [ AnnotationDbi, SingleCellExperiment ]\n github: rcannood/SCORPIUS\n", + "format" : "yaml" + } + ], + "subclass" : [ + "r" + ] + }, + { + "name" : "bioc", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install from BioConductor.", + "example" : [ + { + "example" : "bioc: [ AnnotationDbi ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "github", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install from GitHub.", + "example" : [ + { + "example" : "github: [ rcannood/SCORPIUS ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "gitlab", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install from GitLab.", + "example" : [ + { + "example" : "gitlab: [ org/package ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "url", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install using a generic URI.", + "example" : [ + { + "example" : "url: [ https://github.com/hadley/stringr/archive/HEAD.zip ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "bioc_force_install", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Forces packages specified in `bioc` to be reinstalled, even if they are already present in the container. Default: false.", + "example" : [ + { + "example" : "bioc_force_install: false", + "format" : "yaml" + } + ], + "default" : "False" + }, + { + "name" : "git", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install using a Git URI.", + "example" : [ + { + "example" : "git: [ https://some.git.repository/org/repo ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "cran", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install from CRAN.", + "example" : [ + { + "example" : "cran: [ anndata, ggplot2 ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "bitbucket", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install from Bitbucket.", + "example" : [ + { + "example" : "bitbucket: [ org/package ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "svn", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install using an SVN URI.", + "example" : [ + { + "example" : "svn: [ https://path.to.svn/group/repo ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "packages", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install from CRAN.", + "example" : [ + { + "example" : "packages: [ anndata, ggplot2 ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "script", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies a code block to run as part of the build.", + "example" : [ + { + "example" : "script: |\n cat(\"Running custom code\n\")\n install.packages(\"anndata\")", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the requirement specification." + } + ], + [ + { + "name" : "__this__", + "type" : "RubyRequirements", + "niceType" : "RubyRequirements", + "hierarchy" : [ + "io.viash.platforms.requirements.RubyRequirements", + "io.viash.platforms.requirements.Requirements" + ], + "description" : "Specify which Ruby packages should be available in order to run the component.", + "example" : [ + { + "example" : "setup:\n - type: ruby\n packages: [ rspec ]\n", + "format" : "yaml" + } + ], + "subclass" : [ + "ruby" + ] + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the requirement specification." + }, + { + "name" : "packages", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install.", + "example" : [ + { + "example" : "packages: [ rspec ]", + "format" : "yaml" + } + ], + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "YumRequirements", + "niceType" : "YumRequirements", + "hierarchy" : [ + "io.viash.platforms.requirements.YumRequirements", + "io.viash.platforms.requirements.Requirements" + ], + "description" : "Specify which yum packages should be available in order to run the component.", + "example" : [ + { + "example" : "setup:\n - type: yum\n packages: [ sl ]\n", + "format" : "yaml" + } + ], + "subclass" : [ + "yum" + ] + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the requirement specification." + }, + { + "name" : "packages", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Specifies which packages to install.", + "example" : [ + { + "example" : "packages: [ sl ]", + "format" : "yaml" + } + ], + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "Argument", + "niceType" : "Argument", + "hierarchy" : [ + "io.viash.functionality.arguments.Argument" + ], + "description" : "For each argument, a type and a name must be specified. Depending on the type of argument, different properties can be set. See these reference pages per type for more information: \n\n - @[string](arg_string)\n - @[file](arg_file)\n - @[integer](arg_integer)\n - @[double](arg_double)\n - @[boolean](arg_boolean)\n - @[boolean_true](arg_boolean_true)\n - @[boolean_false](arg_boolean_false)\n", + "example" : [ + { + "example" : "arguments:\n - name: --foo\n type: file\n alternatives: [-f]\n description: Description of foo\n default: \"/foo/bar\"\n must_exist: true\n direction: output\n required: false\n multiple: true\n multiple_sep: \",\"\n - name: --bar\n type: string\n", + "format" : "yaml" + } + ], + "subclass" : [ + "BooleanArgument", + "BooleanTrueArgument", + "BooleanFalseArgument", + "DoubleArgument", + "FileArgument", + "IntegerArgument", + "LongArgument", + "StringArgument" + ] + } + ], + [ + { + "name" : "__this__", + "type" : "BooleanArgument", + "niceType" : "BooleanArgument", + "hierarchy" : [ + "io.viash.functionality.arguments.BooleanArgument", + "io.viash.functionality.arguments.BooleanArgumentBase", + "io.viash.functionality.arguments.Argument" + ], + "description" : "A `boolean` type argument has two possible values: `true` or `false`.", + "example" : [ + { + "example" : "arguments:\n - name: --trim\n type: boolean\n default: true\n description: Trim whitespace from the final output\n alternatives: [\"-t\"]\n", + "format" : "yaml" + } + ], + "subclass" : [ + "boolean" + ] + }, + { + "name" : "alternatives", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "List of alternative format variations for this argument.", + "default" : "Empty" + }, + { + "name" : "name", + "type" : "String", + "niceType" : "String", + "description" : "The name of the argument. Can be in the formats `--trim`, `-t` or `trim`. The number of dashes determines how values can be passed: \n\n - `--trim` is a long option, which can be passed with `executable_name --trim`\n - `-t` is a short option, which can be passed with `executable_name -t`\n - `trim` is an argument, which can be passed with `executable_name trim` \n" + }, + { + "name" : "info", + "type" : "Json", + "niceType" : "Json", + "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", + "example" : [ + { + "example" : "info:\n category: cat1\n labels: [one, two, three]", + "format" : "yaml" + } + ], + "since" : "Viash 0.6.3", + "default" : "Empty" + }, + { + "name" : "default", + "type" : "OneOrMore[Boolean]", + "niceType" : "OneOrMore of Boolean", + "description" : "The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled.", + "example" : [ + { + "example" : "- name: --my_boolean\n type: boolean\n default: true\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "example", + "type" : "OneOrMore[Boolean]", + "niceType" : "OneOrMore of Boolean", + "description" : "An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose.", + "example" : [ + { + "example" : "- name: --my_boolean\n type: boolean\n example: true\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "description", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "A description of the argument. This will be displayed with `--help`.", + "default" : "Empty" + }, + { + "name" : "multiple_sep", + "type" : "String", + "niceType" : "String", + "description" : "The delimiter character for providing [`multiple`](#multiple) values. `:` by default.", + "example" : [ + { + "example" : "- name: --my_boolean\n type: boolean\n multiple: true\n multiple_sep: \",\"\n", + "format" : "yaml" + }, + { + "example" : "my_component --my_boolean=true,true,false", + "format" : "bash", + "description" : "Here's an example of how to use this:" + } + ], + "default" : ":" + }, + { + "name" : "multiple", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default.", + "example" : [ + { + "example" : "- name: --my_boolean\n type: boolean\n multiple: true\n", + "format" : "yaml" + }, + { + "example" : "my_component --my_boolean=true:true:false", + "format" : "bash", + "description" : "Here's an example of how to use this:" + } + ], + "default" : "False" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the argument." + }, + { + "name" : "required", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default.", + "example" : [ + { + "example" : "- name: --my_boolean\n type: boolean\n required: true\n", + "format" : "yaml" + } + ], + "default" : "False" + } + ], + [ + { + "name" : "__this__", + "type" : "BooleanTrueArgument", + "niceType" : "BooleanTrueArgument", + "hierarchy" : [ + "io.viash.functionality.arguments.BooleanTrueArgument", + "io.viash.functionality.arguments.BooleanArgumentBase", + "io.viash.functionality.arguments.Argument" + ], + "description" : "An argument of the `boolean_true` type acts like a `boolean` flag with a default value of `false`. When called as an argument it sets the `boolean` to `true`.", + "example" : [ + { + "example" : "arguments:\n - name: --silent\n type: boolean_true\n description: Ignore console output\n alternatives: [\"-s\"]\n", + "format" : "yaml" + } + ], + "subclass" : [ + "boolean_true" + ] + }, + { + "name" : "alternatives", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "List of alternative format variations for this argument.", + "default" : "Empty" + }, + { + "name" : "name", + "type" : "String", + "niceType" : "String", + "description" : "The name of the argument. Can be in the formats `--silent`, `-s` or `silent`. The number of dashes determines how values can be passed: \n\n - `--silent` is a long option, which can be passed with `executable_name --silent`\n - `-s` is a short option, which can be passed with `executable_name -s`\n - `silent` is an argument, which can be passed with `executable_name silent` \n" + }, + { + "name" : "info", + "type" : "Json", + "niceType" : "Json", + "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", + "example" : [ + { + "example" : "info:\n category: cat1\n labels: [one, two, three]", + "format" : "yaml" + } + ], + "since" : "Viash 0.6.3", + "default" : "Empty" + }, + { + "name" : "description", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "A description of the argument. This will be displayed with `--help`.", + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the argument." + } + ], + [ + { + "name" : "__this__", + "type" : "BooleanFalseArgument", + "niceType" : "BooleanFalseArgument", + "hierarchy" : [ + "io.viash.functionality.arguments.BooleanFalseArgument", + "io.viash.functionality.arguments.BooleanArgumentBase", + "io.viash.functionality.arguments.Argument" + ], + "description" : "An argument of the `boolean_false` type acts like an inverted `boolean` flag with a default value of `true`. When called as an argument it sets the `boolean` to `false`.", + "example" : [ + { + "example" : "arguments:\n - name: --no-log\n type: boolean_false\n description: Disable logging\n alternatives: [\"-nl\"]\n", + "format" : "yaml" + } + ], + "subclass" : [ + "boolean_false" + ] + }, + { + "name" : "alternatives", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "List of alternative format variations for this argument.", + "default" : "Empty" + }, + { + "name" : "name", + "type" : "String", + "niceType" : "String", + "description" : "The name of the argument. Can be in the formats `--no-log`, `-n` or `no-log`. The number of dashes determines how values can be passed: \n\n - `--no-log` is a long option, which can be passed with `executable_name --no-log`\n - `-n` is a short option, which can be passed with `executable_name -n`\n - `no-log` is an argument, which can be passed with `executable_name no-log` \n" + }, + { + "name" : "info", + "type" : "Json", + "niceType" : "Json", + "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", + "example" : [ + { + "example" : "info:\n category: cat1\n labels: [one, two, three]", + "format" : "yaml" + } + ], + "since" : "Viash 0.6.3", + "default" : "Empty" + }, + { + "name" : "description", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "A description of the argument. This will be displayed with `--help`.", + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the argument." + } + ], + [ + { + "name" : "__this__", + "type" : "DoubleArgument", + "niceType" : "DoubleArgument", + "hierarchy" : [ + "io.viash.functionality.arguments.DoubleArgument", + "io.viash.functionality.arguments.Argument" + ], + "description" : "A `double` type argument has a numeric value with decimal points", + "example" : [ + { + "example" : "arguments:\n - name: --litres\n type: double\n default: 1.5\n description: Litres of fluid to process\n alternatives: [\"-l\"]\n", + "format" : "yaml" + } + ], + "subclass" : [ + "double" + ] + }, + { + "name" : "alternatives", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "List of alternative format variations for this argument.", + "default" : "Empty" + }, + { + "name" : "name", + "type" : "String", + "niceType" : "String", + "description" : "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The number of dashes determines how values can be passed: \n\n - `--foo` is a long option, which can be passed with `executable_name --foo=value` or `executable_name --foo value`\n - `-f` is a short option, which can be passed with `executable_name -f value`\n - `foo` is an argument, which can be passed with `executable_name value` \n" + }, + { + "name" : "info", + "type" : "Json", + "niceType" : "Json", + "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", + "example" : [ + { + "example" : "info:\n category: cat1\n labels: [one, two, three]", + "format" : "yaml" + } + ], + "since" : "Viash 0.6.3", + "default" : "Empty" + }, + { + "name" : "max", + "type" : "Option[Double]", + "niceType" : "Option of Double", + "description" : "Maximum allowed value for this argument. If set and the provided value is higher than the maximum, an error will be produced. Can be combined with [`min`](#min) to clamp values.", + "example" : [ + { + "example" : "- name: --my_double\n type: double\n max: 80.4\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "default", + "type" : "OneOrMore[Double]", + "niceType" : "OneOrMore of Double", + "description" : "The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled.", + "example" : [ + { + "example" : "- name: --my_double\n type: double\n default: 5.8\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "example", + "type" : "OneOrMore[Double]", + "niceType" : "OneOrMore of Double", + "description" : "An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose.", + "example" : [ + { + "example" : "- name: --my_double\n type: double\n example: 5.8\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "description", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "A description of the argument. This will be displayed with `--help`.", + "default" : "Empty" + }, + { + "name" : "multiple_sep", + "type" : "String", + "niceType" : "String", + "description" : "The delimiter character for providing [`multiple`](#multiple) values. `:` by default.", + "example" : [ + { + "example" : "- name: --my_double\n type: double\n multiple: true\n multiple_sep: \",\"\n", + "format" : "yaml" + }, + { + "example" : "my_component --my_double=5.8,22.6,200.4", + "format" : "bash", + "description" : "Here's an example of how to use this:" + } + ], + "default" : ":" + }, + { + "name" : "min", + "type" : "Option[Double]", + "niceType" : "Option of Double", + "description" : "Minimum allowed value for this argument. If set and the provided value is lower than the minimum, an error will be produced. Can be combined with [`max`](#max) to clamp values.", + "example" : [ + { + "example" : "- name: --my_double\n type: double\n min: 25.5\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "multiple", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default.", + "example" : [ + { + "example" : "- name: --my_double\n type: double\n multiple: true\n", + "format" : "yaml" + }, + { + "example" : "my_component --my_double=5.8:22.6:200.4", + "format" : "bash", + "description" : "Here's an example of how to use this:" + } + ], + "default" : "False" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the argument." + }, + { + "name" : "required", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default.", + "example" : [ + { + "example" : "- name: --my_double\n type: double\n required: true\n", + "format" : "yaml" + } + ], + "default" : "False" + } + ], + [ + { + "name" : "__this__", + "type" : "FileArgument", + "niceType" : "FileArgument", + "hierarchy" : [ + "io.viash.functionality.arguments.FileArgument", + "io.viash.functionality.arguments.Argument" + ], + "description" : "A `file` type argument has a string value that points to a file or folder path.", + "example" : [ + { + "example" : "arguments:\n - name: --input_csv\n type: file\n must_exist: true\n description: CSV file to read contents from\n alternatives: [\"-i\"]\n", + "format" : "yaml" + } + ], + "subclass" : [ + "file" + ] + }, + { + "name" : "alternatives", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "List of alternative format variations for this argument.", + "default" : "Empty" + }, + { + "name" : "name", + "type" : "String", + "niceType" : "String", + "description" : "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The number of dashes determines how values can be passed: \n\n - `--foo` is a long option, which can be passed with `executable_name --foo=value` or `executable_name --foo value`\n - `-f` is a short option, which can be passed with `executable_name -f value`\n - `foo` is an argument, which can be passed with `executable_name value` \n" + }, + { + "name" : "create_parent", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "If the output filename is a path and it does not exist, create it before executing the script (only for `direction: output`).", + "example" : [ + { + "example" : "- name: --my_file\n type: file\n direction: output\n create_parent: true\n", + "format" : "yaml" + } + ], + "default" : "True" + }, + { + "name" : "direction", + "type" : "Direction", + "niceType" : "Direction", + "description" : "Makes this argument an `input` or an `output`, as in does the file/folder needs to be read or written. `input` by default.", + "example" : [ + { + "example" : "- name: --my_output_file\n type: file\n direction: output\n", + "format" : "yaml" + } + ], + "default" : "Input" + }, + { + "name" : "info", + "type" : "Json", + "niceType" : "Json", + "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", + "example" : [ + { + "example" : "info:\n category: cat1\n labels: [one, two, three]", + "format" : "yaml" + } + ], + "since" : "Viash 0.6.3", + "default" : "Empty" + }, + { + "name" : "must_exist", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Checks whether the file or folder exists. For input files, this check will happen before the execution of the script, while for output files the check will happen afterwards.", + "example" : [ + { + "example" : "- name: --my_file\n type: file\n must_exist: true\n", + "format" : "yaml" + } + ], + "default" : "True" + }, + { + "name" : "default", + "type" : "OneOrMore[Path]", + "niceType" : "OneOrMore of Path", + "description" : "The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled.", + "example" : [ + { + "example" : "- name: --my_file\n type: file\n default: data.csv\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "example", + "type" : "OneOrMore[Path]", + "niceType" : "OneOrMore of Path", + "description" : "An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose.", + "example" : [ + { + "example" : "- name: --my_file\n type: file\n example: data.csv\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "description", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "A description of the argument. This will be displayed with `--help`.", + "default" : "Empty" + }, + { + "name" : "multiple_sep", + "type" : "String", + "niceType" : "String", + "description" : "The delimiter character for providing [`multiple`](#multiple) values. `:` by default.", + "example" : [ + { + "example" : "- name: --my_files\n type: file\n multiple: true\n multiple_sep: \",\"\n", + "format" : "yaml" + }, + { + "example" : "my_component --my_files=firstFile.csv,anotherFile.csv,yetAnother.csv", + "format" : "bash", + "description" : "Here's an example of how to use this:" + } + ], + "default" : ":" + }, + { + "name" : "multiple", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default.", + "example" : [ + { + "example" : "- name: --my_files\n type: file\n multiple: true\n", + "format" : "yaml" + }, + { + "example" : "my_component --my_files=firstFile.csv:anotherFile.csv:yetAnother.csv", + "format" : "bash", + "description" : "Here's an example of how to use this:" + } + ], + "default" : "False" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the argument." + }, + { + "name" : "required", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default.", + "example" : [ + { + "example" : "- name: --my_file\n type: file\n required: true\n", + "format" : "yaml" + } + ], + "default" : "False" + } + ], + [ + { + "name" : "__this__", + "type" : "IntegerArgument", + "niceType" : "IntegerArgument", + "hierarchy" : [ + "io.viash.functionality.arguments.IntegerArgument", + "io.viash.functionality.arguments.Argument" + ], + "description" : "An `integer` type argument has a numeric value without decimal points.", + "example" : [ + { + "example" : "arguments:\n - name: --core_amount\n type: integer\n default: 16\n description: Amount of CPU cores to use\n alternatives: [\"-c\"]\n", + "format" : "yaml" + } + ], + "subclass" : [ + "integer" + ] + }, + { + "name" : "alternatives", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "List of alternative format variations for this argument.", + "default" : "Empty" + }, + { + "name" : "name", + "type" : "String", + "niceType" : "String", + "description" : "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The number of dashes determines how values can be passed: \n\n - `--foo` is a long option, which can be passed with `executable_name --foo=value` or `executable_name --foo value`\n - `-f` is a short option, which can be passed with `executable_name -f value`\n - `foo` is an argument, which can be passed with `executable_name value` \n" + }, + { + "name" : "choices", + "type" : "List[Int]", + "niceType" : "List of Int", + "description" : "Limit the amount of valid values for this argument to those set in this list. When set and a value not present in the list is provided, an error will be produced.", + "example" : [ + { + "example" : "- name: --values\n type: integer\n choices: [1024, 2048, 4096]\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "info", + "type" : "Json", + "niceType" : "Json", + "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", + "example" : [ + { + "example" : "info:\n category: cat1\n labels: [one, two, three]", + "format" : "yaml" + } + ], + "since" : "Viash 0.6.3", + "default" : "Empty" + }, + { + "name" : "max", + "type" : "Option[Int]", + "niceType" : "Option of Int", + "description" : "Maximum allowed value for this argument. If set and the provided value is higher than the maximum, an error will be produced. Can be combined with [`min`](#min) to clamp values.", + "example" : [ + { + "example" : "- name: --my_integer\n type: integer\n max: 150\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "default", + "type" : "OneOrMore[Int]", + "niceType" : "OneOrMore of Int", + "description" : "The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled.", + "example" : [ + { + "example" : "- name: --my_integer\n type: integer\n default: 100\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "example", + "type" : "OneOrMore[Int]", + "niceType" : "OneOrMore of Int", + "description" : "An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose.", + "example" : [ + { + "example" : "- name: --my_integer\n type: integer\n example: 100\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "description", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "A description of the argument. This will be displayed with `--help`.", + "default" : "Empty" + }, + { + "name" : "multiple_sep", + "type" : "String", + "niceType" : "String", + "description" : "The delimiter character for providing [`multiple`](#multiple) values. `:` by default.", + "example" : [ + { + "example" : "- name: --my_integer\n type: integer\n multiple: true\n multiple_sep: \",\"\n", + "format" : "yaml" + }, + { + "example" : "my_component --my_integer=10:80:152", + "format" : "bash", + "description" : "Here's an example of how to use this:" + } + ], + "default" : ":" + }, + { + "name" : "min", + "type" : "Option[Int]", + "niceType" : "Option of Int", + "description" : "Minimum allowed value for this argument. If set and the provided value is lower than the minimum, an error will be produced. Can be combined with [`max`](#max) to clamp values.", + "example" : [ + { + "example" : "- name: --my_integer\n type: integer\n min: 50\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "multiple", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default.", + "example" : [ + { + "example" : "- name: --my_integer\n type: integer\n multiple: true\n", + "format" : "yaml" + }, + { + "example" : "my_component --my_integer=10:80:152", + "format" : "bash", + "description" : "Here's an example of how to use this:" + } + ], + "default" : "False" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the argument." + }, + { + "name" : "required", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default.", + "example" : [ + { + "example" : "- name: --my_integer\n type: integer\n required: true\n", + "format" : "yaml" + } + ], + "default" : "False" + } + ], + [ + { + "name" : "__this__", + "type" : "LongArgument", + "niceType" : "LongArgument", + "hierarchy" : [ + "io.viash.functionality.arguments.LongArgument", + "io.viash.functionality.arguments.Argument" + ], + "description" : "An `long` type argument has a numeric value without decimal points.", + "example" : [ + { + "example" : "arguments:\n - name: --core_amount\n type: long\n default: 16\n description: Amount of CPU cores to use\n alternatives: [\"-c\"]\n", + "format" : "yaml" + } + ], + "since" : "Viash 0.6.1", + "subclass" : [ + "long" + ] + }, + { + "name" : "alternatives", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "List of alternative format variations for this argument.", + "default" : "Empty" + }, + { + "name" : "name", + "type" : "String", + "niceType" : "String", + "description" : "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The number of dashes determines how values can be passed: \n\n - `--foo` is a long option, which can be passed with `executable_name --foo=value` or `executable_name --foo value`\n - `-f` is a short option, which can be passed with `executable_name -f value`\n - `foo` is an argument, which can be passed with `executable_name value` \n" + }, + { + "name" : "choices", + "type" : "List[Long]", + "niceType" : "List of Long", + "description" : "Limit the amount of valid values for this argument to those set in this list. When set and a value not present in the list is provided, an error will be produced.", + "example" : [ + { + "example" : "- name: --values\n type: long\n choices: [1024, 2048, 4096]\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "info", + "type" : "Json", + "niceType" : "Json", + "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", + "example" : [ + { + "example" : "info:\n category: cat1\n labels: [one, two, three]", + "format" : "yaml" + } + ], + "since" : "Viash 0.6.3", + "default" : "Empty" + }, + { + "name" : "max", + "type" : "Option[Long]", + "niceType" : "Option of Long", + "description" : "Maximum allowed value for this argument. If set and the provided value is higher than the maximum, an error will be produced. Can be combined with [`min`](#min) to clamp values.", + "example" : [ + { + "example" : "- name: --my_long\n type: long\n max: 150\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "default", + "type" : "OneOrMore[Long]", + "niceType" : "OneOrMore of Long", + "description" : "The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled.", + "example" : [ + { + "example" : "- name: --my_long\n type: long\n default: 100\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "example", + "type" : "OneOrMore[Long]", + "niceType" : "OneOrMore of Long", + "description" : "An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose.", + "example" : [ + { + "example" : "- name: --my_long\n type: long\n example: 100\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "description", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "A description of the argument. This will be displayed with `--help`.", + "default" : "Empty" + }, + { + "name" : "multiple_sep", + "type" : "String", + "niceType" : "String", + "description" : "The delimiter character for providing [`multiple`](#multiple) values. `:` by default.", + "example" : [ + { + "example" : "- name: --my_long\n type: long\n multiple: true\n multiple_sep: \",\"\n", + "format" : "yaml" + }, + { + "example" : "my_component --my_long=10:80:152", + "format" : "bash", + "description" : "Here's an example of how to use this:" + } + ], + "default" : ":" + }, + { + "name" : "min", + "type" : "Option[Long]", + "niceType" : "Option of Long", + "description" : "Minimum allowed value for this argument. If set and the provided value is lower than the minimum, an error will be produced. Can be combined with [`max`](#max) to clamp values.", + "example" : [ + { + "example" : "- name: --my_long\n type: long\n min: 50\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "multiple", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default.", + "example" : [ + { + "example" : "- name: --my_long\n type: long\n multiple: true\n", + "format" : "yaml" + }, + { + "example" : "my_component --my_long=10:80:152", + "format" : "bash", + "description" : "Here's an example of how to use this:" + } + ], + "default" : "False" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the argument." + }, + { + "name" : "required", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default.", + "example" : [ + { + "example" : "- name: --my_long\n type: long\n required: true\n", + "format" : "yaml" + } + ], + "default" : "False" + } + ], + [ + { + "name" : "__this__", + "type" : "StringArgument", + "niceType" : "StringArgument", + "hierarchy" : [ + "io.viash.functionality.arguments.StringArgument", + "io.viash.functionality.arguments.Argument" + ], + "description" : "A `string` type argument has a value made up of an ordered sequences of characters, like \"Hello\" or \"I'm a string\".", + "example" : [ + { + "example" : "arguments:\n - name: --search_query\n type: string\n default: \"meaning of life\"\n description: The term to search for\n alternatives: [\"-q\"]\n", + "format" : "yaml" + } + ], + "subclass" : [ + "string" + ] + }, + { + "name" : "alternatives", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "List of alternative format variations for this argument.", + "default" : "Empty" + }, + { + "name" : "name", + "type" : "String", + "niceType" : "String", + "description" : "The name of the argument. Can be in the formats `--foo`, `-f` or `foo`. The number of dashes determines how values can be passed: \n\n - `--foo` is a long option, which can be passed with `executable_name --foo=value` or `executable_name --foo value`\n - `-f` is a short option, which can be passed with `executable_name -f value`\n - `foo` is an argument, which can be passed with `executable_name value` \n" + }, + { + "name" : "choices", + "type" : "List[String]", + "niceType" : "List of String", + "description" : "Limit the amount of valid values for this argument to those set in this list. When set and a value not present in the list is provided, an error will be produced.", + "example" : [ + { + "example" : "- name: --language\n type: string\n choices: [\"python\", \"r\", \"javascript\"]\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "info", + "type" : "Json", + "niceType" : "Json", + "description" : "Structured information. Can be any shape: a string, vector, map or even nested map.", + "example" : [ + { + "example" : "info:\n category: cat1\n labels: [one, two, three]", + "format" : "yaml" + } + ], + "since" : "Viash 0.6.3", + "default" : "Empty" + }, + { + "name" : "default", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "The default value when no argument value is provided. This will not work if the [`required`](#required) property is enabled.", + "example" : [ + { + "example" : "- name: --my_string\n type: string\n default: \"The answer is 42\"\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "example", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "An example value for this argument. If no [`default`](#default) property was specified, this will be used for that purpose.", + "example" : [ + { + "example" : "- name: --my_string\n type: string\n example: \"Hello World\"\n", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "description", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "A description of the argument. This will be displayed with `--help`.", + "default" : "Empty" + }, + { + "name" : "multiple_sep", + "type" : "String", + "niceType" : "String", + "description" : "The delimiter character for providing [`multiple`](#multiple) values. `:` by default.", + "example" : [ + { + "example" : "- name: --my_string\n type: string\n multiple: true\n multiple_sep: \",\"\n", + "format" : "yaml" + }, + { + "example" : "my_component --my_string=Marc,Susan,Paul", + "format" : "bash", + "description" : "Here's an example of how to use this:" + } + ], + "default" : ":" + }, + { + "name" : "multiple", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Treat the argument value as an array. Arrays can be passed using the delimiter `--foo=1:2:3` or by providing the same argument multiple times `--foo 1 --foo 2`. You can use a custom delimiter by using the [`multiple_sep`](#multiple_sep) property. `false` by default.", + "example" : [ + { + "example" : "- name: --my_string\n type: string\n multiple: true\n", + "format" : "yaml" + }, + { + "example" : "my_component --my_string=Marc:Susan:Paul", + "format" : "bash", + "description" : "Here's an example of how to use this:" + } + ], + "default" : "False" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the type of the argument." + }, + { + "name" : "required", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "Make the value for this argument required. If set to `true`, an error will be produced if no value was provided. `false` by default.", + "example" : [ + { + "example" : "- name: --my_string\n type: string\n required: true\n", + "format" : "yaml" + } + ], + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "Resource", + "niceType" : "Resource", + "hierarchy" : [ + "io.viash.functionality.resources.Resource" + ], + "description" : "Resources are files that support the component. The first resource should be @[a script](scripting_languages) that will be executed when the functionality is run. Additional resources will be copied to the same directory.\n\nCommon properties:\n\n * type: `file` / `r_script` / `python_script` / `bash_script` / `javascript_script` / `scala_script` / `csharp_script`, specifies the type of the resource. The first resource cannot be of type `file`. When the type is not specified, the default type is simply `file`.\n * dest: filename, the resulting name of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter.\n * path: `path/to/file`, the path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`.\n * text: ...multiline text..., the content of the resulting file specified as a string. Mutually exclusive with `path`.\n * is_executable: `true` / `false`, whether the resulting resource file should be made executable.\n", + "example" : [ + { + "example" : "resources:\n - type: r_script\n path: script.R\n - type: file\n path: resource1.txt\n", + "format" : "yaml" + } + ], + "subclass" : [ + "BashScript", + "CSharpScript", + "Executable", + "JavaScriptScript", + "NextflowScript", + "PlainFile", + "PythonScript", + "RScript", + "ScalaScript" + ] + } + ], + [ + { + "name" : "__this__", + "type" : "BashScript", + "niceType" : "BashScript", + "hierarchy" : [ + "io.viash.functionality.resources.BashScript", + "io.viash.functionality.resources.Script", + "io.viash.functionality.resources.Resource" + ], + "description" : "An executable Bash script.\nWhen defined in functionality.resources, only the first entry will be executed when running the built component or when running `viash run`.\nWhen defined in functionality.test_resources, all entries will be executed during `viash test`.", + "subclass" : [ + "bash_script" + ] + }, + { + "name" : "path", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`.", + "default" : "Empty" + }, + { + "name" : "text", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`.", + "default" : "Empty" + }, + { + "name" : "is_executable", + "type" : "Option[Boolean]", + "niceType" : "Option of Boolean", + "description" : "Whether the resulting resource file should be made executable.", + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the resource as a Bash script." + }, + { + "name" : "dest", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter.", + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "CSharpScript", + "niceType" : "CSharpScript", + "hierarchy" : [ + "io.viash.functionality.resources.CSharpScript", + "io.viash.functionality.resources.Script", + "io.viash.functionality.resources.Resource" + ], + "description" : "An executable C# script.\nWhen defined in functionality.resources, only the first entry will be executed when running the built component or when running `viash run`.\nWhen defined in functionality.test_resources, all entries will be executed during `viash test`.", + "subclass" : [ + "csharp_script" + ] + }, + { + "name" : "path", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`.", + "default" : "Empty" + }, + { + "name" : "text", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`.", + "default" : "Empty" + }, + { + "name" : "is_executable", + "type" : "Option[Boolean]", + "niceType" : "Option of Boolean", + "description" : "Whether the resulting resource file should be made executable.", + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the resource as a C# script." + }, + { + "name" : "dest", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter.", + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "Executable", + "niceType" : "Executable", + "hierarchy" : [ + "io.viash.functionality.resources.Executable", + "io.viash.functionality.resources.Script", + "io.viash.functionality.resources.Resource" + ], + "description" : "An executable file.", + "subclass" : [ + "executable" + ] + }, + { + "name" : "path", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`.", + "default" : "Empty" + }, + { + "name" : "text", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`.", + "default" : "Empty" + }, + { + "name" : "is_executable", + "type" : "Option[Boolean]", + "niceType" : "Option of Boolean", + "description" : "Whether the resulting resource file should be made executable.", + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the resource as an executable." + }, + { + "name" : "dest", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter.", + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "JavaScriptScript", + "niceType" : "JavaScriptScript", + "hierarchy" : [ + "io.viash.functionality.resources.JavaScriptScript", + "io.viash.functionality.resources.Script", + "io.viash.functionality.resources.Resource" + ], + "description" : "An executable JavaScript script.\nWhen defined in functionality.resources, only the first entry will be executed when running the built component or when running `viash run`.\nWhen defined in functionality.test_resources, all entries will be executed during `viash test`.", + "subclass" : [ + "javascript_script" + ] + }, + { + "name" : "path", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`.", + "default" : "Empty" + }, + { + "name" : "text", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`.", + "default" : "Empty" + }, + { + "name" : "is_executable", + "type" : "Option[Boolean]", + "niceType" : "Option of Boolean", + "description" : "Whether the resulting resource file should be made executable.", + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the resource as a JavaScript script." + }, + { + "name" : "dest", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter.", + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "NextflowScript", + "niceType" : "NextflowScript", + "hierarchy" : [ + "io.viash.functionality.resources.NextflowScript", + "io.viash.functionality.resources.Script", + "io.viash.functionality.resources.Resource" + ], + "description" : "A Nextflow script. Work in progress; added mainly for annotation at the moment.", + "subclass" : [ + "nextflow_script" + ] + }, + { + "name" : "path", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`.", + "default" : "Empty" + }, + { + "name" : "text", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`.", + "default" : "Empty" + }, + { + "name" : "entrypoint", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The name of the workflow to be executed.", + "default" : "Empty" + }, + { + "name" : "is_executable", + "type" : "Option[Boolean]", + "niceType" : "Option of Boolean", + "description" : "Whether the resulting resource file should be made executable.", + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the resource as a Nextflow script." + }, + { + "name" : "dest", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter.", + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "PlainFile", + "niceType" : "PlainFile", + "hierarchy" : [ + "io.viash.functionality.resources.PlainFile", + "io.viash.functionality.resources.Resource" + ], + "description" : "A plain file. This can only be used as a supporting resource for the main script or unit tests.", + "subclass" : [ + "file" + ] + }, + { + "name" : "path", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`.", + "default" : "Empty" + }, + { + "name" : "text", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`.", + "default" : "Empty" + }, + { + "name" : "is_executable", + "type" : "Option[Boolean]", + "niceType" : "Option of Boolean", + "description" : "Whether the resulting resource file should be made executable.", + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the resource as a plain file." + }, + { + "name" : "dest", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter.", + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "PythonScript", + "niceType" : "PythonScript", + "hierarchy" : [ + "io.viash.functionality.resources.PythonScript", + "io.viash.functionality.resources.Script", + "io.viash.functionality.resources.Resource" + ], + "description" : "An executable Python script.\nWhen defined in functionality.resources, only the first entry will be executed when running the built component or when running `viash run`.\nWhen defined in functionality.test_resources, all entries will be executed during `viash test`.", + "subclass" : [ + "python_script" + ] + }, + { + "name" : "path", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`.", + "default" : "Empty" + }, + { + "name" : "text", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`.", + "default" : "Empty" + }, + { + "name" : "is_executable", + "type" : "Option[Boolean]", + "niceType" : "Option of Boolean", + "description" : "Whether the resulting resource file should be made executable.", + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the resource as a Python script." + }, + { + "name" : "dest", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter.", + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "RScript", + "niceType" : "RScript", + "hierarchy" : [ + "io.viash.functionality.resources.RScript", + "io.viash.functionality.resources.Script", + "io.viash.functionality.resources.Resource" + ], + "description" : "An executable R script.\nWhen defined in functionality.resources, only the first entry will be executed when running the built component or when running `viash run`.\nWhen defined in functionality.test_resources, all entries will be executed during `viash test`.", + "subclass" : [ + "r_script" + ] + }, + { + "name" : "path", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`.", + "default" : "Empty" + }, + { + "name" : "text", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`.", + "default" : "Empty" + }, + { + "name" : "is_executable", + "type" : "Option[Boolean]", + "niceType" : "Option of Boolean", + "description" : "Whether the resulting resource file should be made executable.", + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the resource as a R script." + }, + { + "name" : "dest", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter.", + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "ScalaScript", + "niceType" : "ScalaScript", + "hierarchy" : [ + "io.viash.functionality.resources.ScalaScript", + "io.viash.functionality.resources.Script", + "io.viash.functionality.resources.Resource" + ], + "description" : "An executable Scala script.\nWhen defined in functionality.resources, only the first entry will be executed when running the built component or when running `viash run`.\nWhen defined in functionality.test_resources, all entries will be executed during `viash test`.", + "subclass" : [ + "scala_script" + ] + }, + { + "name" : "path", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The path of the input file. Can be a relative or an absolute path, or a URI. Mutually exclusive with `text`.", + "default" : "Empty" + }, + { + "name" : "text", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The content of the resulting file specified as a string. Mutually exclusive with `path`.", + "default" : "Empty" + }, + { + "name" : "is_executable", + "type" : "Option[Boolean]", + "niceType" : "Option of Boolean", + "description" : "Whether the resulting resource file should be made executable.", + "default" : "Empty" + }, + { + "name" : "type", + "type" : "String", + "niceType" : "String", + "description" : "Specifies the resource as a Scala script." + }, + { + "name" : "dest", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "Resulting filename of the resource. From within a script, the file can be accessed at `meta[\"resources_dir\"] + \"/\" + dest`. If unspecified, `dest` will be set to the basename of the `path` parameter.", + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "NextflowDirectives", + "niceType" : "NextflowDirectives", + "hierarchy" : [ + "io.viash.platforms.nextflow.NextflowDirectives" + ], + "description" : "Directives are optional settings that affect the execution of the process.\n", + "example" : [ + { + "example" : "directives:\n container: rocker/r-ver:4.1\n label: highcpu\n cpus: 4\n memory: 16 GB", + "format" : "yaml" + } + ] + }, + { + "name" : "beforeScript", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The `beforeScript` directive allows you to execute a custom (Bash) snippet before the main process script is run. This may be useful to initialise the underlying cluster environment or for other custom initialisation.\n\nSee [`beforeScript`](https://www.nextflow.io/docs/latest/process.html#beforeScript).\n", + "example" : [ + { + "example" : "source /cluster/bin/setup", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "module", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Environment Modules is a package manager that allows you to dynamically configure your execution environment and easily switch between multiple versions of the same software tool.\n\nIf it is available in your system you can use it with Nextflow in order to configure the processes execution environment in your pipeline.\n\nIn a process definition you can use the `module` directive to load a specific module version to be used in the process execution environment.\n\nSee [`module`](https://www.nextflow.io/docs/latest/process.html#module).\n", + "example" : [ + { + "example" : "\"ncbi-blast/2.2.27\"", + "format" : "yaml" + }, + { + "example" : "\"ncbi-blast/2.2.27:t_coffee/10.0\"", + "format" : "yaml" + }, + { + "example" : "[\"ncbi-blast/2.2.27\", \"t_coffee/10.0\"]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "queue", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "The `queue` directory allows you to set the queue where jobs are scheduled when using a grid based executor in your pipeline.\n\nSee [`queue`](https://www.nextflow.io/docs/latest/process.html#queue).\n", + "example" : [ + { + "example" : "\"long\"", + "format" : "yaml" + }, + { + "example" : "\"short,long\"", + "format" : "yaml" + }, + { + "example" : "[\"short\", \"long\"]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "label", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "The `label` directive allows the annotation of processes with mnemonic identifier of your choice.\n\nSee [`label`](https://www.nextflow.io/docs/latest/process.html#label).\n", + "example" : [ + { + "example" : "\"big_mem\"", + "format" : "yaml" + }, + { + "example" : "\"big_cpu\"", + "format" : "yaml" + }, + { + "example" : "[\"big_mem\", \"big_cpu\"]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "container", + "type" : "Option[Either[Map[String,String],String]]", + "niceType" : "Option of Either\n - Map of String to String\n - String", + "description" : "The `container` directive allows you to execute the process script in a Docker container.\n\nIt requires the Docker daemon to be running in machine where the pipeline is executed, i.e. the local machine when using the local executor or the cluster nodes when the pipeline is deployed through a grid executor.\n\nViash implements allows either a string value or a map. In case a map is used, the allowed keys are: `registry`, `image`, and `tag`. The `image` value must be specified.\n\nSee [`container`](https://www.nextflow.io/docs/latest/process.html#container).\n", + "example" : [ + { + "example" : "\"foo/bar:tag\"", + "format" : "yaml" + }, + { + "example" : "[ registry: \"reg\", image: \"im\", tag: \"ta\" ]", + "format" : "yaml", + "description" : "This is transformed to `\"reg/im:ta\"`:" + }, + { + "example" : "[ image: \"im\" ]", + "format" : "yaml", + "description" : "This is transformed to `\"im:latest\"`:" + } + ], + "default" : "Empty" + }, + { + "name" : "publishDir", + "type" : "OneOrMore[Either[String,Map[String,String]]]", + "niceType" : "OneOrMore of Either\n - String\n - Map of String to String", + "description" : "The `publishDir` directive allows you to publish the process output files to a specified folder.\n\nViash implements this directive as a plain string or a map. The allowed keywords for the map are: `path`, `mode`, `overwrite`, `pattern`, `saveAs`, `enabled`. The `path` key and value are required.\nThe allowed values for `mode` are: `symlink`, `rellink`, `link`, `copy`, `copyNoFollow`, `move`.\n\nSee [`publishDir`](https://www.nextflow.io/docs/latest/process.html#publishdir).\n", + "example" : [ + { + "example" : "[]", + "format" : "yaml" + }, + { + "example" : "[ [ path: \"foo\", enabled: true ], [ path: \"bar\", enabled: false ] ]", + "format" : "yaml" + }, + { + "example" : "\"/path/to/dir\"", + "format" : "yaml", + "description" : "This is transformed to `[[ path: \"/path/to/dir\" ]]`:" + }, + { + "example" : "[ path: \"/path/to/dir\", mode: \"cache\" ]", + "format" : "yaml", + "description" : "This is transformed to `[[ path: \"/path/to/dir\", mode: \"cache\" ]]`:" + } + ], + "default" : "Empty" + }, + { + "name" : "maxForks", + "type" : "Option[Either[String,Int]]", + "niceType" : "Option of Either\n - String\n - Int", + "description" : "The `maxForks` directive allows you to define the maximum number of process instances that can be executed in parallel. By default this value is equals to the number of CPU cores available minus 1.\n\nIf you want to execute a process in a sequential manner, set this directive to one.\n\nSee [`maxForks`](https://www.nextflow.io/docs/latest/process.html#maxforks).\n", + "example" : [ + { + "example" : "1", + "format" : "yaml" + }, + { + "example" : "3", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "maxErrors", + "type" : "Option[Either[String,Int]]", + "niceType" : "Option of Either\n - String\n - Int", + "description" : "The `maxErrors` directive allows you to specify the maximum number of times a process can fail when using the `retry` error strategy. By default this directive is disabled.\n\nSee [`maxErrors`](https://www.nextflow.io/docs/latest/process.html#maxerrors).\n", + "example" : [ + { + "example" : "1", + "format" : "yaml" + }, + { + "example" : "3", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "cpus", + "type" : "Option[Either[Int,String]]", + "niceType" : "Option of Either\n - Int\n - String", + "description" : "The `cpus` directive allows you to define the number of (logical) CPU required by the process' task.\n\nSee [`cpus`](https://www.nextflow.io/docs/latest/process.html#cpus).\n", + "example" : [ + { + "example" : "1", + "format" : "yaml" + }, + { + "example" : "10", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "accelerator", + "type" : "Map[String,String]", + "niceType" : "Map of String to String", + "description" : "The `accelerator` directive allows you to specify the hardware accelerator requirement for the task execution e.g. GPU processor.\n\nViash implements this directive as a map with accepted keywords: `type`, `limit`, `request`, and `runtime`.\n\nSee [`accelerator`](https://www.nextflow.io/docs/latest/process.html#accelerator).\n", + "example" : [ + { + "example" : "[ limit: 4, type: \"nvidia-tesla-k80\" ]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "time", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The `time` directive allows you to define how long a process is allowed to run.\n\nSee [`time`](https://www.nextflow.io/docs/latest/process.html#time).\n", + "example" : [ + { + "example" : "\"1h\"", + "format" : "yaml" + }, + { + "example" : "\"2days\"", + "format" : "yaml" + }, + { + "example" : "\"1day 6hours 3minutes 30seconds\"", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "afterScript", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The `afterScript` directive allows you to execute a custom (Bash) snippet immediately after the main process has run. This may be useful to clean up your staging area.\n\nSee [`afterScript`](https://www.nextflow.io/docs/latest/process.html#afterscript).\n", + "example" : [ + { + "example" : "source /cluster/bin/cleanup", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "executor", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The `executor` defines the underlying system where processes are executed. By default a process uses the executor defined globally in the nextflow.config file.\n\nThe `executor` directive allows you to configure what executor has to be used by the process, overriding the default configuration. The following values can be used:\n\n| Name | Executor |\n|------|----------|\n| awsbatch | The process is executed using the AWS Batch service. | \n| azurebatch | The process is executed using the Azure Batch service. | \n| condor | The process is executed using the HTCondor job scheduler. | \n| google-lifesciences | The process is executed using the Google Genomics Pipelines service. | \n| ignite | The process is executed using the Apache Ignite cluster. | \n| k8s | The process is executed using the Kubernetes cluster. | \n| local | The process is executed in the computer where Nextflow is launched. | \n| lsf | The process is executed using the Platform LSF job scheduler. | \n| moab | The process is executed using the Moab job scheduler. | \n| nqsii | The process is executed using the NQSII job scheduler. | \n| oge | Alias for the sge executor. | \n| pbs | The process is executed using the PBS/Torque job scheduler. | \n| pbspro | The process is executed using the PBS Pro job scheduler. | \n| sge | The process is executed using the Sun Grid Engine / Open Grid Engine. | \n| slurm | The process is executed using the SLURM job scheduler. | \n| tes | The process is executed using the GA4GH TES service. | \n| uge | Alias for the sge executor. |\n\nSee [`executor`](https://www.nextflow.io/docs/latest/process.html#executor).\n", + "example" : [ + { + "example" : "\"local\"", + "format" : "yaml" + }, + { + "example" : "\"sge\"", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "containerOptions", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "The `containerOptions` directive allows you to specify any container execution option supported by the underlying container engine (ie. Docker, Singularity, etc). This can be useful to provide container settings only for a specific process e.g. mount a custom path.\n\nSee [`containerOptions`](https://www.nextflow.io/docs/latest/process.html#containeroptions).\n", + "example" : [ + { + "example" : "\"--foo bar\"", + "format" : "yaml" + }, + { + "example" : "[\"--foo bar\", \"-f b\"]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "disk", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The `disk` directive allows you to define how much local disk storage the process is allowed to use.\n\nSee [`disk`](https://www.nextflow.io/docs/latest/process.html#disk).\n", + "example" : [ + { + "example" : "\"1 GB\"", + "format" : "yaml" + }, + { + "example" : "\"2TB\"", + "format" : "yaml" + }, + { + "example" : "\"3.2KB\"", + "format" : "yaml" + }, + { + "example" : "\"10.B\"", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "tag", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The `tag` directive allows you to associate each process execution with a custom label, so that it will be easier to identify them in the log file or in the trace execution report.\n\nSee [`tag`](https://www.nextflow.io/docs/latest/process.html#tag).\n", + "example" : [ + { + "example" : "\"foo\"", + "format" : "yaml" + }, + { + "example" : "'$id'", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "conda", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "The `conda` directive allows for the definition of the process dependencies using the Conda package manager.\n\nNextflow automatically sets up an environment for the given package names listed by in the `conda` directive.\n\nSee [`conda`](https://www.nextflow.io/docs/latest/process.html#conda).\n", + "example" : [ + { + "example" : "\"bwa=0.7.15\"", + "format" : "yaml" + }, + { + "example" : "\"bwa=0.7.15 fastqc=0.11.5\"", + "format" : "yaml" + }, + { + "example" : "[\"bwa=0.7.15\", \"fastqc=0.11.5\"]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "machineType", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : " The `machineType` can be used to specify a predefined Google Compute Platform machine type when running using the Google Life Sciences executor.\n\nSee [`machineType`](https://www.nextflow.io/docs/latest/process.html#machinetype).\n", + "example" : [ + { + "example" : "\"n1-highmem-8\"", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "stageInMode", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The `stageInMode` directive defines how input files are staged-in to the process work directory. The following values are allowed:\n\n| Value | Description |\n|-------|-------------| \n| copy | Input files are staged in the process work directory by creating a copy. | \n| link | Input files are staged in the process work directory by creating an (hard) link for each of them. | \n| symlink | Input files are staged in the process work directory by creating a symbolic link with an absolute path for each of them (default). | \n| rellink | Input files are staged in the process work directory by creating a symbolic link with a relative path for each of them. | \n\nSee [`stageInMode`](https://www.nextflow.io/docs/latest/process.html#stageinmode).\n", + "example" : [ + { + "example" : "\"copy\"", + "format" : "yaml" + }, + { + "example" : "\"link\"", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "cache", + "type" : "Option[Either[Boolean,String]]", + "niceType" : "Option of Either\n - Boolean\n - String", + "description" : "The `cache` directive allows you to store the process results to a local cache. When the cache is enabled and the pipeline is launched with the resume option, any following attempt to execute the process, along with the same inputs, will cause the process execution to be skipped, producing the stored data as the actual results.\n\nThe caching feature generates a unique key by indexing the process script and inputs. This key is used to identify univocally the outputs produced by the process execution.\n\nThe `cache` is enabled by default, you can disable it for a specific process by setting the cache directive to `false`.\n\nAccepted values are: `true`, `false`, `\"deep\"`, and `\"lenient\"`.\n\nSee [`cache`](https://www.nextflow.io/docs/latest/process.html#cache).\n", + "example" : [ + { + "example" : "true", + "format" : "yaml" + }, + { + "example" : "false", + "format" : "yaml" + }, + { + "example" : "\"deep\"", + "format" : "yaml" + }, + { + "example" : "\"lenient\"", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "pod", + "type" : "OneOrMore[Map[String,String]]", + "niceType" : "OneOrMore of Map of String to String", + "description" : "The `pod` directive allows the definition of pods specific settings, such as environment variables, secrets and config maps when using the Kubernetes executor.\n\nSee [`pod`](https://www.nextflow.io/docs/latest/process.html#pod).\n", + "example" : [ + { + "example" : "[ label: \"key\", value: \"val\" ]", + "format" : "yaml" + }, + { + "example" : "[ annotation: \"key\", value: \"val\" ]", + "format" : "yaml" + }, + { + "example" : "[ env: \"key\", value: \"val\" ]", + "format" : "yaml" + }, + { + "example" : "[ [label: \"l\", value: \"v\"], [env: \"e\", value: \"v\"]]", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "penv", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The `penv` directive allows you to define the parallel environment to be used when submitting a parallel task to the SGE resource manager.\n\nSee [`penv`](https://www.nextflow.io/docs/latest/process.html#penv).\n", + "example" : [ + { + "example" : "\"smp\"", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "scratch", + "type" : "Option[Either[Boolean,String]]", + "niceType" : "Option of Either\n - Boolean\n - String", + "description" : "The `scratch` directive allows you to execute the process in a temporary folder that is local to the execution node.\n\nSee [`scratch`](https://www.nextflow.io/docs/latest/process.html#scratch).\n", + "example" : [ + { + "example" : "true", + "format" : "yaml" + }, + { + "example" : "\"/path/to/scratch\"", + "format" : "yaml" + }, + { + "example" : "'$MY_PATH_TO_SCRATCH'", + "format" : "yaml" + }, + { + "example" : "\"ram-disk\"", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "storeDir", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The `storeDir` directive allows you to define a directory that is used as a permanent cache for your process results.\n\nSee [`storeDir`](https://www.nextflow.io/docs/latest/process.html#storeDir).\n", + "example" : [ + { + "example" : "\"/path/to/storeDir\"", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "maxRetries", + "type" : "Option[Either[String,Int]]", + "niceType" : "Option of Either\n - String\n - Int", + "description" : "The `maxRetries` directive allows you to define the maximum number of times a process instance can be re-submitted in case of failure. This value is applied only when using the retry error strategy. By default only one retry is allowed.\n\nSee [`maxRetries`](https://www.nextflow.io/docs/latest/process.html#maxretries).\n", + "example" : [ + { + "example" : "1", + "format" : "yaml" + }, + { + "example" : "3", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "echo", + "type" : "Option[Either[Boolean,String]]", + "niceType" : "Option of Either\n - Boolean\n - String", + "description" : "By default the stdout produced by the commands executed in all processes is ignored. By setting the `echo` directive to true, you can forward the process stdout to the current top running process stdout file, showing it in the shell terminal.\n \nSee [`echo`](https://www.nextflow.io/docs/latest/process.html#echo).\n", + "example" : [ + { + "example" : "true", + "format" : "yaml" + }, + { + "example" : "false", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "errorStrategy", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The `errorStrategy` directive allows you to define how an error condition is managed by the process. By default when an error status is returned by the executed script, the process stops immediately. This in turn forces the entire pipeline to terminate.\n\nTable of available error strategies:\n| Name | Executor |\n|------|----------|\n| `terminate` | Terminates the execution as soon as an error condition is reported. Pending jobs are killed (default) |\n| `finish` | Initiates an orderly pipeline shutdown when an error condition is raised, waiting the completion of any submitted job. |\n| `ignore` | Ignores processes execution errors. |\n| `retry` | Re-submit for execution a process returning an error condition. |\n\nSee [`errorStrategy`](https://www.nextflow.io/docs/latest/process.html#errorstrategy).\n", + "example" : [ + { + "example" : "\"terminate\"", + "format" : "yaml" + }, + { + "example" : "\"finish\"", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "memory", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The `memory` directive allows you to define how much memory the process is allowed to use.\n\nSee [`memory`](https://www.nextflow.io/docs/latest/process.html#memory).\n", + "example" : [ + { + "example" : "\"1 GB\"", + "format" : "yaml" + }, + { + "example" : "\"2TB\"", + "format" : "yaml" + }, + { + "example" : "\"3.2KB\"", + "format" : "yaml" + }, + { + "example" : "\"10.B\"", + "format" : "yaml" + } + ], + "default" : "Empty" + }, + { + "name" : "stageOutMode", + "type" : "Option[String]", + "niceType" : "Option of String", + "description" : "The `stageOutMode` directive defines how output files are staged-out from the scratch directory to the process work directory. The following values are allowed:\n\n| Value | Description |\n|-------|-------------| \n| copy | Output files are copied from the scratch directory to the work directory. | \n| move | Output files are moved from the scratch directory to the work directory. | \n| rsync | Output files are copied from the scratch directory to the work directory by using the rsync utility. |\n\nSee [`stageOutMode`](https://www.nextflow.io/docs/latest/process.html#stageoutmode).\n", + "example" : [ + { + "example" : "\"copy\"", + "format" : "yaml" + }, + { + "example" : "\"link\"", + "format" : "yaml" + } + ], + "default" : "Empty" + } + ], + [ + { + "name" : "__this__", + "type" : "NextflowAuto", + "niceType" : "NextflowAuto", + "hierarchy" : [ + "io.viash.platforms.nextflow.NextflowAuto" + ], + "description" : "Automated processing flags which can be toggled on or off." + }, + { + "name" : "simplifyInput", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "If `true`, an input tuple only containing only a single File (e.g. `[\"foo\", file(\"in.h5ad\")]`) is automatically transformed to a map (i.e. `[\"foo\", [ input: file(\"in.h5ad\") ] ]`).\n\nDefault: `true`.\n", + "default" : "True" + }, + { + "name" : "simplifyOutput", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "If `true`, an output tuple containing a map with a File (e.g. `[\"foo\", [ output: file(\"out.h5ad\") ] ]`) is automatically transformed to a map (i.e. `[\"foo\", file(\"out.h5ad\")]`).\n\nDefault: `true`.\n", + "default" : "True" + }, + { + "name" : "publish", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "If `true`, the module's outputs are automatically published to `params.publishDir`.\nWill throw an error if `params.publishDir` is not defined.\n\nDefault: `false`.\n", + "default" : "False" + }, + { + "name" : "transcript", + "type" : "Boolean", + "niceType" : "Boolean", + "description" : "If `true`, the module's transcripts from `work/` are automatically published to `params.transcriptDir`.\nIf not defined, `params.publishDir + \"/_transcripts\"` will be used.\nWill throw an error if neither are defined.\n\nDefault: `false`.\n", + "default" : "False" + } + ], + [ + { + "name" : "__this__", + "type" : "NextflowConfig", + "niceType" : "NextflowConfig", + "hierarchy" : [ + "io.viash.platforms.nextflow.NextflowConfig" + ], + "description" : "Allows tweaking how the Nextflow Config file is generated.", + "since" : "Viash 0.7.4" + }, + { + "name" : "labels", + "type" : "ListMap[String,String]", + "niceType" : "Map of String to String", + "description" : "A series of default labels to specify memory and cpu constraints.\n\nThe default memory labels are defined as \"mem1gb\", \"mem2gb\", \"mem4gb\", ... upto \"mem512tb\" and follows powers of 2.\nThe default cpu labels are defined as \"cpu1\", \"cpu2\", \"cpu5\", \"cpu10\", ... upto \"cpu1000\" and follows a semi logarithmic scale (1, 2, 5 per decade).\n\nConceptually it is possible for a Viash Config to overwrite the full labels parameter, however likely it is more efficient to add additional labels\nin the Viash Project with a config mod.\n", + "example" : [ + { + "example" : "labels:\n lowmem: \"memory = 4.GB\"\n lowcpu: \"cpus = 4\"\n midmem: \"memory = 25.GB\"\n midcpu: \"cpus = 10\"\n highmem: \"memory = 50.GB\"\n highcpu: \"cpus = 20\"\n vhighmem: \"memory = 100.GB\"\n vhighcpu: \"cpus = 40\"\n", + "format" : "yaml", + "description" : "Replace the default labels with a different set of labels" + }, + { + "example" : "-c '.platforms[.type == \"nextflow\"].config.labels.lowmem := \"memory = 4.GB\";.platforms[.type == \"nextflow\"].config.labels.lowcpu := \"cpus = 4\"'", + "format" : "viash_config_mod", + "description" : "Add 'lowmem' and 'lowcpu' to the default labels by using a config mod" + }, + { + "example" : "config_mods: |\n .platforms[.type == \"nextflow\"].config.labels.lowmem := \"memory = 4.GB\"\n .platforms[.type == \"nextflow\"].config.labels.lowcpu := \"cpus = 4\"\n", + "format" : "viash_project_file", + "description" : "Add 'lowmem' and 'lowcpu' to the default labels by using the Viash Project file" + }, + { + "example" : "config_mods: |\n .platforms[.type == \"nextflow\"].config.labels := { lowmem: \"memory = 4.GB\", lowcpu: \"cpus = 4\", midmem: \"memory = 25.GB\", midcpu: \"cpus = 10\", highmem: \"memory = 50.GB\", highcpu: \"cpus = 20\", vhighmem: \"memory = 100.GB\", vhighcpu: \"cpus = 40\" }\n", + "format" : "viash_project_file", + "description" : "Replace the default labels with a different set of labels by using the Viash Project file" + } + ], + "default" : "A series of default labels to specify memory and cpu constraints" + }, + { + "name" : "script", + "type" : "OneOrMore[String]", + "niceType" : "OneOrMore of String", + "description" : "Includes a single string or list of strings into the nextflow.config file.\nThis can be used to add custom profiles or include an additional config file.\n", + "example" : [ + { + "example" : "script:\n - |\n profiles {\n ...\n }\n", + "format" : "yaml" + }, + { + "example" : "script: includeConfig(\"config.config\")", + "format" : "yaml" + } + ], + "default" : "Empty" + } + ] +] \ No newline at end of file diff --git a/reference/nextflow_vdsl3/import_module.qmd b/reference/nextflow_vdsl3/import_module.qmd new file mode 100644 index 00000000..044433f6 --- /dev/null +++ b/reference/nextflow_vdsl3/import_module.qmd @@ -0,0 +1,107 @@ +--- +title: Import a VDSL3 module +--- + +A VDSL3 module is a Nextflow module generated by Viash. See the [guide](/guide/nextflow_vdsl3/index.qmd) for a more in-depth explanation on how to create Nextflow workflows with VDSL3 modules. + +## Importing a VDSL3 module + + +After building a VDSL3 module from a component, the VDSL3 module can be imported just like any other Nextflow module. + +**Example:** + +```groovy +include { mymodule } from 'target/nextflow/mymodule/main.nf' +``` + +## VDSL3 module interface + +VDSL3 modules are actually workflows which take one channel and emit one channel. It expects the channel events to be tuples containing an 'id' and a 'state': `[id, state]`, where `id` is a unique String and `state` is a `Map[String, Object]`. The resulting channel then consists of tuples `[id, new_state]`. + +**Example:** + +```groovy +workflow { + Channel.fromList([ + ["myid", [input: file("in.txt")]] + ]) + | mymodule +} +``` + +:::{.callout-note} +If the input tuple has more than two elements, the elements after the second element are passed through to the output tuple. +That is, an input tuple `[id, input, ...]` will result in a tuple `[id, output, ...]` after running the module. +For example, an input tuple `["foo", [input: file("in.txt")], "bar"]` will result in an output tuple `["foo", [output: file("out.txt")], "bar"]`. +::: + +## Customizing VDSL3 modules on the fly + +Usually, Nextflow processes are quite static objects. For example, changing its directives can be quite tricky. + +The `.run()` function is a unique feature for every VDSL3 module which allows dynamically altering the behaviour of a module from within the pipeline. For example, we use it to set the `publishDir` directive to `"output/"` so the output of that step in the pipeline will be stored as output. + +**Example:** + +```groovy +workflow { + Channel.fromList([ + ["myid", [input: file("in.txt")]] + ]) + | mymodule.run( + args: [k: 10], + directives: [cpus: 4, memory: "16 GB"] + ) +} +``` + +### Arguments of `.run()` + +- `key` (`String`): A unique key used to trace the process and help make names of output files unique. Default: the name of the Viash component. + +- `args` (`Map[String, Object]`): Argument overrides to be passed to the module. + +- `directives` (`Map[String, Object]`): Custom directives overrides. See the Nextflow documentation for a list of available directives. + +- `auto` (`Map[String, Boolean]`): Whether to apply certain automated processing steps. Default values are inherited from the [Viash config](/reference/config/platforms/nextflow/auto.qmd). + +- `auto.simplifyInput`: If `true`, if the input tuple is a single file and if the module only has a single input file, the input file will be passed the module accordingly. Default: `true` (inherited from Viash config). + +- `auto.simplifyOutput`: If `true`, if the output tuple is a single file and if the module only has a single output file, the output map will be transformed into a single file. Default: `true` (inherited from Viash config). + +- `auto.publish`: If `true`, the output files will be published to the `params.publishDir` folder. Default: `false` (inherited from Viash config). + +- `auto.transcript`: If `true`, the module's transcript will be published to the `params.transcriptDir` folder. Default: `false` (inherited from Viash config). + +- `map` (`Function`): Apply a map over the incoming tuple. Example: `{ tup -> [ tup[0], [input: tup[1].output] ] + tup.drop(2) }`. Default: `null`. + +- `mapId` (`Function`): Apply a map over the ID element of a tuple (i.e. the first element). Example: `{ id -> id + "_foo" }`. Default: `null`. + +- `mapData` (`Function`): Apply a map over the data element of a tuple (i.e. the second element). Example: `{ data -> [ input: data.output ] }`. Default: `null`. + +- `mapPassthrough` (`Function`): Apply a map over the passthrough elements of a tuple (i.e. the tuple excl. the first two elements). Example: `{ pt -> pt.drop(1) }`. Default: `null`. + +- `filter` (`Function`): Filter the channel. Example: `{ tup -> tup[0] == "foo" }`. Default: `null`. + +- `fromState`: Fetch data from the state and pass it to the module without altering the current state. `fromState` should be `null`, `List[String]`, `Map[String, String]` or a function. + + - If it is `null`, the state will be passed to the module as is. + - If it is a `List[String]`, the data will be the values of the state at the given keys. + - If it is a `Map[String, String]`, the data will be the values of the state at the given keys, with the keys renamed according to the map. + - If it is a function, the tuple (`[id, state]`) in the channel will be passed to the function, and the result will be used as the data. + + Example: `{ id, state -> [input: state.fastq_file] }` + Default: `null` + +- `toState`: Determine how the state should be updated after the module has been run. `toState` should be `null`, `List[String]`, `Map[String, String]` or a function. + + - If it is `null`, the state will be replaced with the output of the module. + - If it is a `List[String]`, the state will be updated with the values of the data at the given keys. + - If it is a `Map[String, String]`, the state will be updated with the values of the data at the given keys, with the keys renamed according to the map. + - If it is a function, a tuple (`[id, output, state]`) will be passed to the function, and the result will be used as the new state. + + Example: `{ id, output, state -> state + [counts: state.output] }` + Default: `{ id, output, state -> output }` + +- `debug`: Whether or not to print debug messages. Default: `false`. diff --git a/reference/nextflow_vdsl3/index.qmd b/reference/nextflow_vdsl3/index.qmd new file mode 100644 index 00000000..c33c9a06 --- /dev/null +++ b/reference/nextflow_vdsl3/index.qmd @@ -0,0 +1,10 @@ +--- +title: Nextflow VDSL3 +order: 35 +--- + +Viash supports creating Nextflow workflows in multiple ways. + +* [Run a module as a standaline pipeline](run_module.qmd) +* [Import a VDSL3 module](import_module.qmd) +* Create a Nextflow workflow with dependencies \ No newline at end of file diff --git a/reference/nextflow_vdsl3/run_module.qmd b/reference/nextflow_vdsl3/run_module.qmd new file mode 100644 index 00000000..27ce9ce8 --- /dev/null +++ b/reference/nextflow_vdsl3/run_module.qmd @@ -0,0 +1,61 @@ +--- +title: Run a VDSL3 module +--- + + +Unlike typical Nextflow modules, VDSL3 modules can actually be used as a standalone pipeline. + +To run a VDSL3 module as a standalone pipeline, you need to specify the input parameters and a `--publish_dir` parameter, as Nextflow will automatically choose the parameter names of the output files. + +## Viewing the help message + +More information regarding a modules arguments can be shown by passing the `--help` parameter. + +**Example:** + +```bash +nextflow run target/nextflow/mycomponent/main.nf --help +``` + + +## Running a module as a standalone pipeline +You can run the executable by providing a value for each of the required arguments and `--publish_dir` (where output files are published). + +**Example:** + +```bash +nextflow run target/nextflow/mycomponent/main.nf \ + --input config.vsh.yaml \ + --publish_dir output/ +``` + + +## Passing a parameter list + +Every VDSL3 can accept a list of parameters to populate a Nextflow channel with. Assuming we want to process a set of input files in parallel, we can create a yaml file `params.yaml` containing the following information. + + +```yaml +param_list: + - id: sample1 + input: data/sample1.txt + - id: sample2 + input: data/sample2.txt + - id: sample3 + input: data/sample3.txt + - id: sample4 + input: data/sample4.txt +arg1: 10 +arg2: 5 +``` + +You can run the pipeline on the list of parameters using the `-params-file` parameter. + +```bash +nextflow run target/main.nf -params-file params.yaml --publish_dir output2 +``` + + +:::{.callout-tip} +You can also pass a YAML, CSV or JSON file to the `param_list` parameter. +::: \ No newline at end of file diff --git a/reference/project/_index.yaml b/reference/project/_index.yaml deleted file mode 100644 index 9fdd491c..00000000 --- a/reference/project/_index.yaml +++ /dev/null @@ -1,40 +0,0 @@ -data: -- description: A Viash project configuration file. It's name should be `_viash.yaml`. - example: - - example: "viash_version: 0.6.4\nsource: src\ntarget: target\nconfig_mods: |\n\ - \ .platforms[.type == 'docker'].target_registry := 'ghcr.io'\n .platforms[.type\ - \ == 'docker'].target_organization := 'viash-io'\n .platforms[.type == 'docker'].namespace_separator\ - \ := '/'\n .platforms[.type == 'docker'].target_image_source := 'https://github.com/viash-io/viash'\n" - format: yaml - hierarchy: - - io.viash.project.ViashProject - name: __this__ - since: Viash 0.6.4 - type: ViashProject -- description: Which source directory to use for the `viash ns` commands. - example: - - example: 'source: src' - format: yaml - name: source - type: Option of String -- description: Which version of Viash to use. - example: - - example: 'viash_versions: 0.6.4' - format: yaml - name: viash_version - type: Option of String -- description: Which config mods to apply. - example: - - example: 'config_mods: ".functionality.name := ''foo''"' - format: yaml - name: config_mods - type: OneOrMore of String -- description: Which target directory to use for `viash ns build`. - example: - - example: 'target: target' - format: yaml - name: target - type: Option of String -order: 30 -title: Project -topic: config diff --git a/reference/project/index.qmd b/reference/project/index.qmd index 34239144..be945bc4 100644 --- a/reference/project/index.qmd +++ b/reference/project/index.qmd @@ -24,6 +24,8 @@ config_mods: | **Type**: `String` / `List of String` +**Default**: `Empty` + Which config mods to apply. **Example:** @@ -36,6 +38,8 @@ config_mods: ".functionality.name := 'foo'" **Type**: `String` +**Default**: `Empty` + Which source directory to use for the `viash ns` commands. **Example:** @@ -48,6 +52,8 @@ source: src **Type**: `String` +**Default**: `Empty` + Which target directory to use for `viash ns build`. **Example:** @@ -60,6 +66,8 @@ target: target **Type**: `String` +**Default**: `Empty` + Which version of Viash to use. **Example:** @@ -67,4 +75,3 @@ Which version of Viash to use. ```yaml viash_versions: 0.6.4 ``` - diff --git a/reference/reference.yml b/reference/reference.yml index ab22a29a..7af0a3c9 100644 --- a/reference/reference.yml +++ b/reference/reference.yml @@ -34,9 +34,7 @@ href: /reference/config/platforms/native/index.html - text: Docker Platform href: /reference/config/platforms/docker/index.html - - text: Nextflow Legacy Platform - href: /reference/config/platforms/nextflowLegacy/index.html - - text: Nextflow VDSL3 Platform + - text: Nextflow Platform href: /reference/config/platforms/nextflow/index.html - title: "Argument Types"