From 486823bf3327d6c4d0aae14a73c0635d44ebfba6 Mon Sep 17 00:00:00 2001 From: Chris Ostrouchov Date: Mon, 14 Aug 2023 22:41:27 -0400 Subject: [PATCH 1/4] Adding initial commit for extension-system docs --- docs/docs/how-tos/nebari-extension-system.md | 9 +++++++++ 1 file changed, 9 insertions(+) create mode 100644 docs/docs/how-tos/nebari-extension-system.md diff --git a/docs/docs/how-tos/nebari-extension-system.md b/docs/docs/how-tos/nebari-extension-system.md new file mode 100644 index 000000000..9bd167dd1 --- /dev/null +++ b/docs/docs/how-tos/nebari-extension-system.md @@ -0,0 +1,9 @@ +--- +id: nebari-extension-system +title: Nebari Extension System +description: An overview of the pluggy extension system for Nebari +--- + +## Introduction + +This guide is to help developers extend and modify the behavior of nebari. From fce96505e30037896acafaea087bc796a7d0f0ec Mon Sep 17 00:00:00 2001 From: Chris Ostrouchov Date: Tue, 15 Aug 2023 07:52:39 -0400 Subject: [PATCH 2/4] Adding docs on user, development, and subcommands --- docs/docs/how-tos/nebari-extension-system.md | 78 +++++++++++++++++++- 1 file changed, 77 insertions(+), 1 deletion(-) diff --git a/docs/docs/how-tos/nebari-extension-system.md b/docs/docs/how-tos/nebari-extension-system.md index 9bd167dd1..b15275a4c 100644 --- a/docs/docs/how-tos/nebari-extension-system.md +++ b/docs/docs/how-tos/nebari-extension-system.md @@ -6,4 +6,80 @@ description: An overview of the pluggy extension system for Nebari ## Introduction -This guide is to help developers extend and modify the behavior of nebari. +This guide is to help developers extend and modify the behavior of +nebari. We leverage the plugin system +[pluggy](https://pluggy.readthedocs.io/en/stable/) to enable easy +extensibility. Currently Nebari supports: + + - overriding a given stages in a deployment + - adding additional stages before, after and between existing stages + - arbitrary subcommands + +We maintain an [examples +repository](https://github.com/nebari-dev/nebari-plugin-examples) +which contains up to date usable examples. + +## Installing a plugin + +Registering a plugin should be easy for the end user. Pluggins are +either installed into your existing environment e.g. + +```shell +pip install my-nebari-plugin +``` + +Alternatively if you only want to temporarily add an extension. + +:::note +`--import-plugin` does not work for loading subcommands due to +cli already being constructed before plugin imports take place. + +```shell +nebari --import-plugin path/to/plugin.py ... +nebari --import-plugin import.module.plugin.path ... +``` + +## Developing an extension + +The most important step to developing a plugin is ensuring that the +setuptools entrypoint is set. + +```toml:pyproject.toml + +... + +[project.entry-points.nebari] +my-subcommand = "path.to.subcommand.module" + +... + +``` + +Adding this one line to your `pyproject.toml` will ensure that upon +installation of the package the nebari plugins are registered. + +### Subcommands + +Nebari exposes a hook `nebari_subcommand` which exposes the +[typer](https://github.com/tiangolo/typer) cli instance. This allows +the developer to attach and arbitrary number of subcommands. + +```python +from nebari.hookspecs import hookimpl + +import typer + + +@hookimpl +def nebari_subcommand(cli): + @cli.command() + def hello( + name: str = typer.Option( + "Nebari", help="Who to say hello to" + ) + ): + print(f"Hello {name}") +``` + + + From 5e2209fc1c9858ead4bae8d9e78ce8f6d4e2388a Mon Sep 17 00:00:00 2001 From: Chris Ostrouchov Date: Tue, 15 Aug 2023 07:58:37 -0400 Subject: [PATCH 3/4] Adding extension system --- docs/docs/how-tos/nebari-extension-system.md | 1 + docs/sidebars.js | 1 + 2 files changed, 2 insertions(+) diff --git a/docs/docs/how-tos/nebari-extension-system.md b/docs/docs/how-tos/nebari-extension-system.md index b15275a4c..6c7a57661 100644 --- a/docs/docs/how-tos/nebari-extension-system.md +++ b/docs/docs/how-tos/nebari-extension-system.md @@ -33,6 +33,7 @@ Alternatively if you only want to temporarily add an extension. :::note `--import-plugin` does not work for loading subcommands due to cli already being constructed before plugin imports take place. +::: ```shell nebari --import-plugin path/to/plugin.py ... diff --git a/docs/sidebars.js b/docs/sidebars.js index ce91f8e04..245cfae02 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -74,6 +74,7 @@ module.exports = { "how-tos/setup-argo", "how-tos/using-argo", "how-tos/idle-culling", + "how-tos/nebari-extension-system", ], }, { From 09368ea52e615299fd88bc5367927554e3f6ab98 Mon Sep 17 00:00:00 2001 From: Chris Ostrouchov Date: Tue, 15 Aug 2023 08:16:54 -0400 Subject: [PATCH 4/4] Adding documentation on stages --- docs/docs/how-tos/nebari-extension-system.md | 56 ++++++++++++++++++++ 1 file changed, 56 insertions(+) diff --git a/docs/docs/how-tos/nebari-extension-system.md b/docs/docs/how-tos/nebari-extension-system.md index 6c7a57661..6069ca0e1 100644 --- a/docs/docs/how-tos/nebari-extension-system.md +++ b/docs/docs/how-tos/nebari-extension-system.md @@ -82,5 +82,61 @@ def nebari_subcommand(cli): print(f"Hello {name}") ``` +There is a dedicated working example in [nebari-plugin-examples](https://github.com/nebari-dev/nebari-plugin-examples/tree/main/examples/nebari_subcommand_hello_world). +### Stages +Nebari exposes a hook `nebari_stage` which uses the `NebariStage` +class. The `NebriStage` exposes `render`, `deploy`, `destroy` as +arbitrary functions called. See below for a complete example. + +Nebari decides the order of stages based on two things the `name: str` +attribute and `priority: int` attribute. The rules are as follows: + - stages are ordered by `priority` + - stages which have the same `name` the one with highest priority + number is chosen. + +```python +import contextlib +import os +from typing import Dict, Any + +from nebari.hookspecs import hookimpl, NebariStage + + +class HelloWorldStage(NebariStage): + name = "hello_world" + priority = 100 + + def render(self): + return { + "hello_world.txt": "File that says hello world" + } + + @contextlib.contextmanager + def deploy(self, stage_outputs: Dict[str, Dict[str, Any]]): + print("I ran deploy") + # set environment variables for stages that run after + os.environ['HELLO'] = 'WORLD' + # set output state for future stages to use + stage_outputs[self.name] = {'hello': 'world'} + yield + # cleanup after deployment (rarely needed) + os.environ.pop('HELLO') + + def check(self, stage_outputs: Dict[str, Dict[str, Any]]): + if 'HELLO' not in os.environ: + raise ValueError('stage did not deploy successfully since HELLO environment variable not set') + + @contextlib.contextmanager + def destroy(self, stage_outputs: Dict[str, Dict[str, Any]], status: Dict[str, bool]): + print('faking to destroy things for hello world stage') + yield + + +@hookimpl +def nebari_stage(): + return [HelloWorldStage] +``` + +There is a dedicated working example in [nebari-plugin-examples](https://github.com/nebari-dev/nebari-plugin-examples/tree/main/examples/nebari_stage_hello_world).