diff --git a/.github/ISSUE_TEMPLATE/RFD.md b/.github/ISSUE_TEMPLATE/RFD.md index b1af18c82..f1b94c79d 100644 --- a/.github/ISSUE_TEMPLATE/RFD.md +++ b/.github/ISSUE_TEMPLATE/RFD.md @@ -12,11 +12,11 @@ title: "RFD - Title" | Status | Draft 🚧 / Open for comments πŸ’¬/ Accepted βœ… /Implemented πŸš€/ Obsolete πŸ—ƒ | -| ----------------- | ------------------------------------------------------------------------ | -| Author(s) | GitHub handle | -| Date Created | dd-MM-YYY | -| Date Last updated | dd-MM-YYY | -| Decision deadline | dd-MM-YYY | +| ----------------- | ------------------------------------------------------------------------- | +| Author(s) | GitHub handle | +| Date Created | dd-MM-YYY | +| Date Last updated | dd-MM-YYY | +| Decision deadline | dd-MM-YYY | # Title diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml index db8ec1502..83ccc9e9f 100644 --- a/.pre-commit-config.yaml +++ b/.pre-commit-config.yaml @@ -17,7 +17,7 @@ ci: repos: # Codespell: Spell checks the code and documentation - repo: https://github.com/codespell-project/codespell - rev: v2.2.4 + rev: v2.2.5 hooks: - id: codespell entry: codespell @@ -28,7 +28,7 @@ repos: # Markdown, json, toml - repo: https://github.com/pre-commit/mirrors-prettier - rev: v3.0.0-alpha.9-for-vscode + rev: v3.0.0 hooks: - id: prettier types_or: [markdown, mdx, json, toml] diff --git a/docs/docs/community/code-contributions.mdx b/docs/docs/community/code-contributions.mdx index 200a2132a..ab6a78b65 100644 --- a/docs/docs/community/code-contributions.mdx +++ b/docs/docs/community/code-contributions.mdx @@ -158,8 +158,8 @@ Once installed, the pre-commit hooks will run automatically when you make a comm 1. Before you start, make sure to pull the latest changes from upstream. ```bash - git checkout main - git pull upstream main + git checkout develop + git pull upstream develop ``` 2. Create a branch for the bug or feature you want to work on. The branch name will appear in the merge message, so use a sensible, self-explanatory name: diff --git a/docs/docs/community/index.mdx b/docs/docs/community/index.mdx index c948bea96..4567b7b74 100644 --- a/docs/docs/community/index.mdx +++ b/docs/docs/community/index.mdx @@ -109,7 +109,7 @@ We have a: The embedded Google Calendar below shows the meeting schedules. Click on the meeting event in the calendar to find more details like the [Google Meet link](https://meet.google.com/ijt-fveo-nvx), or to copy the event to your personal calendar. - + [Read past community meeting notes β†’](https://github.com/nebari-dev/nebari/discussions/categories/community-meetings) diff --git a/docs/docs/references/RELEASE.md b/docs/docs/references/RELEASE.md index b41c9e21a..9eb17a1cd 100644 --- a/docs/docs/references/RELEASE.md +++ b/docs/docs/references/RELEASE.md @@ -11,7 +11,68 @@ This file is copied to nebari-dev/nebari-docs using a GitHub Action. --> ## Upcoming Release -## Release 2023.5.1 - May 5, 2023 +## Release 2023.7.2 - August 3, 2023 + +This is a hot-fix release that resolves an issue whereby users in the `analyst` group are unable to launch their JupyterLab server because the name of the viewer-specific `ARGO_TOKEN` was mislabeled; see [PR 1881](https://github.com/nebari-dev/nebari/pull/1881) for more details. + +### What's Changed +* Fix argo-viewer service account reference by @iameskild in https://github.com/nebari-dev/nebari/pull/1881 +* Add release notes for 2023.7.2, update release notes for 2023.7.1 by @iameskild in https://github.com/nebari-dev/nebari/pull/1886 + + +## Release 2023.7.1 - July 21, 2023 + +> WARNING: CDS Dashboards will be deprecated soon. Nebari `2023.7.1` will be the last release with support for CDS Dashboards integration. A new dashboard sharing mechanism added in the near future, but some releases in the interim will not have dashboard sharing capabilities.. + +> WARNING: For those running on AWS, upgrading from previous versions to `2023.7.1` requires a [backup](https://www.nebari.dev/docs/how-tos/manual-backup). Due to changes made to the VPC (See [issue 1884](https://github.com/nebari-dev/nebari/issues/1884) for details), Terraform thinks it needs to destroy and reprovision a new VPC which causes the entire cluster to be destroyed and rebuilt. + +### Feature changes and enhancements + +* Addition of Nebari-Workflow-Controller in [PR 1741](https://github.com/nebari-dev/nebari/pull/1741) +* Addition of Argo-Jupyter-Scheduler in [PR 1832](https://github.com/nebari-dev/nebari/pull/1832) +* Make most of the API private + +### Breaking Changes + +* As mentioned in the above WARNING, clusters running on AWS should perform a [manual backup](https://www.nebari.dev/docs/how-tos/manual-backup) before running the upgrade to the latest version as changes to the AWS VPC will cause the cluster to be destroyed and redeployed. + + +### What's Changed +* use conda forge explicitly in conda build test by @pmeier in https://github.com/nebari-dev/nebari/pull/1771 +* document that the upgrade command is for all nebari upgrades by @Adam-D-Lewis in https://github.com/nebari-dev/nebari/pull/1794 +* don't fail CI matrices fast by @pmeier in https://github.com/nebari-dev/nebari/pull/1804 +* unvendor keycloak_metrics_spi by @Adam-D-Lewis in https://github.com/nebari-dev/nebari/pull/1810 +* Dedent fail-fast by @iameskild in https://github.com/nebari-dev/nebari/pull/1815 +* support deploying on existing vpc on aws by @Adam-D-Lewis in https://github.com/nebari-dev/nebari/pull/1807 +* purge most danlging qhub references by @pmeier in https://github.com/nebari-dev/nebari/pull/1802 +* Add Argo Workflow Admission controller by @Adam-D-Lewis in https://github.com/nebari-dev/nebari/pull/1741 +* purge infracost CLI command / CI jobs by @pmeier in https://github.com/nebari-dev/nebari/pull/1820 +* remove unused function parameters and CLI flags by @pmeier in https://github.com/nebari-dev/nebari/pull/1725 +* purge docs and nox by @pmeier in https://github.com/nebari-dev/nebari/pull/1801 +* Add Helm chart lint tool by @viniciusdc in https://github.com/nebari-dev/nebari/pull/1679 +* don't set /etc/hosts in CI by @pmeier in https://github.com/nebari-dev/nebari/pull/1729 +* remove execute permissions on templates by @pmeier in https://github.com/nebari-dev/nebari/pull/1798 +* fix deprecated file deletion by @pmeier in https://github.com/nebari-dev/nebari/pull/1799 +* make nebari API private by @pmeier in https://github.com/nebari-dev/nebari/pull/1778 +* [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/nebari-dev/nebari/pull/1831 +* Simplify CI by @iameskild in https://github.com/nebari-dev/nebari/pull/1819 +* Fix edge-case where k8s_version is equal to HIGHEST_SUPPORTED_K8S_VER… by @iameskild in https://github.com/nebari-dev/nebari/pull/1842 +* add more configuration to enable private clusters on AWS by @Adam-D-Lewis in https://github.com/nebari-dev/nebari/pull/1841 +* [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in https://github.com/nebari-dev/nebari/pull/1851 +* AWS gov cloud support by @sblair-metrostar in https://github.com/nebari-dev/nebari/pull/1857 +* Pathlib everywhere by @pmeier in https://github.com/nebari-dev/nebari/pull/1773 +* Initial playwright setup by @kcpevey in https://github.com/nebari-dev/nebari/pull/1665 +* Changes required for Jupyter-Scheduler integration by @iameskild in https://github.com/nebari-dev/nebari/pull/1832 +* Update upgrade command in preparation for release by @iameskild in https://github.com/nebari-dev/nebari/pull/1868 +* Add release notes by @iameskild in https://github.com/nebari-dev/nebari/issues/1869 + +### New Contributors +* @sblair-metrostar made their first contribution in https://github.com/nebari-dev/nebari/pull/1857 + +**Full Changelog**: https://github.com/nebari-dev/nebari/compare/2023.5.1...2023.7.1 + + +### Release 2023.5.1 - May 5, 2023 ### Feature changes and enhancements diff --git a/docs/docs/tutorials/creating-cds-dashboard.md b/docs/docs/tutorials/creating-cds-dashboard.md index e55efa60a..ec82adfd2 100644 --- a/docs/docs/tutorials/creating-cds-dashboard.md +++ b/docs/docs/tutorials/creating-cds-dashboard.md @@ -6,6 +6,14 @@ description: Quickly build and deploy a panel and CDS Dashboards # Create and deploy a dashboard with `Panel` and `CDS Dashboards` +:::danger + +CDS Dashboards will be deprecated soon. +Nebari 2023.7.1 will be the last release with support for CDS Dashboards integration. +A new dashboard sharing mechanism added in the near future, but some releases in the interim will not have dashboard sharing capabilities. + +::: + ## Introduction Analyzing data provides insights, and a dashboard stitches these insights into a meaningful story. Fortunately, there are many great open source dashboards tools out there that you can use to organize and display your data in an engaging and digestible way. diff --git a/docs/docs/tutorials/run-notebooks-as-a-job.md b/docs/docs/tutorials/run-notebooks-as-a-job.md new file mode 100644 index 000000000..cfdaa0186 --- /dev/null +++ b/docs/docs/tutorials/run-notebooks-as-a-job.md @@ -0,0 +1,131 @@ +--- +id: jupyter-scheduler +title: Submit notebooks jobs with jupyter-scheduler +description: Using jupyter-scheduler to submit batch jobs +--- + +# Submit Jupyter Notebooks as Jobs + +:::warning +This is a new feature still in beta so please [leave some feedback](https://github.com/nebari-dev/nebari/discussions) and [report any issues](https://github.com/nebari-dev/nebari/issues). + +There is one known issue with the `Update Job Definition` and `Resume` job definition which is related to Nebari-Workflow-Controller issue, [captured here](https://github.com/nebari-dev/nebari-workflow-controller/issues/18). The current workaround for those who need to update (or pause) your job definitions, is simply to delete the current job definition and create a new one as and when needed. +::: + + +A common task that many Nebari users have is submitting their notebooks to run as a script or to run on a predefined schedule. This is now possible with [Jupyter-Scheduler](https://jupyter-scheduler.readthedocs.io/en/latest/index.html#), a JupyterLab extension that has been expanded and integrated into Nebari. This also means that you can view the status of the jobs by vising the `/argo` endpoint. + +The idea of notebook jobs is useful in situations where you need no human interaction, besides submitting it as a job, and the results can be efficiently saved to your home directory, the cloud or other similar storage locations. It is also useful in situations where the notebook might run for a long time and the user needs to shut down their JupyterLab server. + +:::info +If you're curious why shutting down your JupyterLab server (and associated JupyterLab extensions) doesn't kill or stop your running Jupyter-Scheduler jobs, this is because the Nebari team extended Jupyter-Scheduler to use Argo-Workflows as it's backend; details can be f[ound here](https://github.com/nebari-dev/argo-workflows-executor). + +::: + +Jupyter-Scheduler is included by default in the base Nebari JupyterLab image and can be used with any conda-store environment available to the notebook author. + +:::note +When using a conda-store environment, please ensure that the [`papermill` package](https://papermill.readthedocs.io/en/latest/) is included. +::: + + +## Submitting a Notebook as a Jupyter-Scheduler Job + +To submit your notebook as a Jupyter-Scheduler Job, simply click the `Jupyter-Scheduler` icon on the top of your notebook toolbar. + + +![Jupyter-Scheduler UI - location of the icon on the notebook toolbar](/img/tutorials/jupyter-scheduler-icon.png) + +This will open the Jupyter-Scheduler UI. From here you can specify: +- the notebook **job name** +- the **input file** the use (this will default to the file from which the icon was clicked) +- the **environment** to run the notebook with +- add environment variables via the `Parameters` section + +
+ Jupyter-Scheduler UI - layout of Jupyter-Scheduler UI. +
+ +Once created, the status and output of the notebook job can be viewed from the Jupyter-Scheduler UI: + + +![Jupyter-Scheduler UI - view the notebook job status](/img/tutorials/jupyter-scheduler-job-status.png) + +Click on the notebook job name to view more information about the job. From here, you can view: +- the **job ID** +- when the job was **created** +- the **start time** and **end time** + +![Jupyter-Scheduler UI - view the notebook job details](/img/tutorials/jupyter-scheduler-job-details.png) + + +:::info +As mentioned above, the notebook job will run as an Argo-Workflows workflow. This means these jobs (workflows) are viewable from the Argo-Workflows UI at `/argo`. The name of the workflow is prefixed with `job-`. + +![Argo UI - view the notebook job status from the argo UI](/img/tutorials/jupyter-scheduler-argo-ui.png) + +::: + +:::note +Only users in the `admin` or `developer` groups will have access to create notebook jobs. For more information regarding users and groups, please visit [How to configure Keycloak](/how-tos/configure-keycloak-howto.md) +::: + +All associated workflows have a TTL (time-to-live) set to 600 seconds. This means that regardless of whether or not your workflow was successful, it will be deleted after 10 mins. The downside is that those logs are now gone, that said, the upside is that the workflow no longer consumes compute resources once deleted. + +## Submit Jupyter Notebooks to run on a schedule + +The other way notebooks can be run is on a specified schedule, Jupyter-Scheduler refers to these are "Job Definitions". After selecting the Jupyter-Scheduler icon at the top of the notebook, select "Run on a schedule" and then define the schedule the notebook should be run on. There is also an option to specify the time zone. + +:::tip +You can check [crontab.guru](https://crontab.guru) which is a nifty tool that tries to translate the schedule syntax into plain English. +::: + +
+ Jupyter-Scheduler UI - create a job definition to run on a specified schedule. +
+ +When a job definition is created, a new job is created at each time interval specified by the schedule. These created jobs can be inspected like a regular notebook job. From here you can: +- **delete** the job definition +- **pause** the job job definition +- view details such as the **status** of the job definition +- view the list of notebook jobs created from this job definition +- view the **tags** (by default all job definitions are tagged with `Cron-Workflow`) + +
+ Jupyter-Scheduler UI - view a job definition details. +
+ +:::info +Unlike a regular notebook job, job definitions create Argo-Workflows cron-workflows (workflows that run on a schedule). This cron-workflow in turn creates workflows at each time interval specified by the schedule. Another difference is that the workflow names are prefixed with `job-def-`. + +![Argo UI - view the job definition from the argo UI](/img/tutorials/jupyter-scheduler-argo-cron-workflow.png) +::: + +:::warning +Notebook jobs that run on a schedule will run indefinitely so it's the responsibility of the job creator to either delete or pause the job if then they are no longer needed. +::: + + +## Debugging failed jobs + +Occasionally notebook jobs will fail to run and it's helpful to understand why. + +
+ Jupyter-Scheduler UI - view the details of a failed job. +
+ + +If there is an issue with the notebook code itself, viewing the notebook job logs will help the user get a better idea of what went wrong. These are the steps to get the logs: + +1. Document the job ID +2. Launch a terminal session +3. Navigate to `~/.local/share/jupyter/scheduler_staging_area` +4. Find the folder that corresponds to the job ID that failed +5. Open (or `cat`) the `output.ipynb` to view the notebook as it was run + + +:::info +Notebook job details can also be viewed from the `/argo` UI. +::: + +Lastly, if the job fails without writing to this `scheduler_staging_area`, or the job status is stuck in `In progress` mode for an extended period of time, have an administrator try and view the specific logs on the user's JupyterLab server pod or on the workflow pod itself. \ No newline at end of file diff --git a/docs/sidebars.js b/docs/sidebars.js index 5feb8f178..b09bae619 100644 --- a/docs/sidebars.js +++ b/docs/sidebars.js @@ -47,6 +47,7 @@ module.exports = { "tutorials/creating-new-environments", "tutorials/kbatch", "tutorials/cost-estimate-report", + "tutorials/jupyter-scheduler", ], }, { diff --git a/docs/static/img/tutorials/jupyter-scheduler-argo-cron-workflow.png b/docs/static/img/tutorials/jupyter-scheduler-argo-cron-workflow.png new file mode 100644 index 000000000..4d72cce44 Binary files /dev/null and b/docs/static/img/tutorials/jupyter-scheduler-argo-cron-workflow.png differ diff --git a/docs/static/img/tutorials/jupyter-scheduler-argo-ui.png b/docs/static/img/tutorials/jupyter-scheduler-argo-ui.png new file mode 100644 index 000000000..1d8216cb2 Binary files /dev/null and b/docs/static/img/tutorials/jupyter-scheduler-argo-ui.png differ diff --git a/docs/static/img/tutorials/jupyter-scheduler-icon.png b/docs/static/img/tutorials/jupyter-scheduler-icon.png new file mode 100644 index 000000000..a4a2658cc Binary files /dev/null and b/docs/static/img/tutorials/jupyter-scheduler-icon.png differ diff --git a/docs/static/img/tutorials/jupyter-scheduler-job-def-details.png b/docs/static/img/tutorials/jupyter-scheduler-job-def-details.png new file mode 100644 index 000000000..0d1eae005 Binary files /dev/null and b/docs/static/img/tutorials/jupyter-scheduler-job-def-details.png differ diff --git a/docs/static/img/tutorials/jupyter-scheduler-job-definition.png b/docs/static/img/tutorials/jupyter-scheduler-job-definition.png new file mode 100644 index 000000000..650250fa6 Binary files /dev/null and b/docs/static/img/tutorials/jupyter-scheduler-job-definition.png differ diff --git a/docs/static/img/tutorials/jupyter-scheduler-job-details.png b/docs/static/img/tutorials/jupyter-scheduler-job-details.png new file mode 100644 index 000000000..0461e9a51 Binary files /dev/null and b/docs/static/img/tutorials/jupyter-scheduler-job-details.png differ diff --git a/docs/static/img/tutorials/jupyter-scheduler-job-failed.png b/docs/static/img/tutorials/jupyter-scheduler-job-failed.png new file mode 100644 index 000000000..69ff60026 Binary files /dev/null and b/docs/static/img/tutorials/jupyter-scheduler-job-failed.png differ diff --git a/docs/static/img/tutorials/jupyter-scheduler-job-status.png b/docs/static/img/tutorials/jupyter-scheduler-job-status.png new file mode 100644 index 000000000..ef96d3604 Binary files /dev/null and b/docs/static/img/tutorials/jupyter-scheduler-job-status.png differ diff --git a/docs/static/img/tutorials/jupyter-scheduler-run-now.png b/docs/static/img/tutorials/jupyter-scheduler-run-now.png new file mode 100644 index 000000000..a04fdac72 Binary files /dev/null and b/docs/static/img/tutorials/jupyter-scheduler-run-now.png differ