From 651d7b0da4397b30e9bba31da6353cd7a124805d Mon Sep 17 00:00:00 2001 From: willGraham01 <1willgraham@gmail.com> Date: Thu, 4 Jan 2024 12:05:47 +0000 Subject: [PATCH 01/11] Add blog, changelog, and install instructions --- .../blog/version1/version_1_released.md | 53 +++++++++++++++++++ docs/source/community/releases/v1/index.md | 20 +++++++ docs/source/documentation/index.md | 49 ++++++++++++++++- 3 files changed, 121 insertions(+), 1 deletion(-) create mode 100644 docs/source/blog/version1/version_1_released.md diff --git a/docs/source/blog/version1/version_1_released.md b/docs/source/blog/version1/version_1_released.md new file mode 100644 index 00000000..70b140cf --- /dev/null +++ b/docs/source/blog/version1/version_1_released.md @@ -0,0 +1,53 @@ +--- +blogpost: true +date: Jan 8th, 2024 +author: Will Graham +location: London, England +category: BrainGlobe-v1 +language: English +--- + +# BrainGlobe version 1 is here! :partying_face: + +Following our series of incremental updates to a number of BrainGlobe tools, we are pleased to announce that BrainGlobe version 1 has been released today! +Users can now enjoy: + +- A one-line installation of _all_ BrainGlobe tools, automatically handling dependency inconsistencies and providing the _entire_ BrainGlobe tool suite. Individual tools no longer need to be manually installed as-and-when they're needed, and users do not need to worry about the behind-the-scenes organisation of the packages providing the tools. +- A clearer and more consistent naming convention for the tools that are provided. +- Access to example analysis workflows, installable on top of the BrainGlobe tool suite. + +BrainGlobe version 1 is hallmarked by the release of (version 1 of) the [`brainglobe` package on PyPI](https://pypi.org/project/brainglobe/), and the new [`brainglobe-workflows` package](https://pypi.org/project/brainglobe-workflows/). + +## Users: what you need to know + +The individual tools that you have been using are still available, and are largely still using the same name (with the exception of the old `cellfinder` command-line-tool, see below). +Regardless of how many of our tools you use in your analysis, we recommend that you create a new environment and (re)install any BrainGlobe tools you have been using following the instructions on the installation page. + +For the most part, the effects of moving to version 1 should be limited to the cases where a tool has been merged with its napari plugin, or the relocation of the (old) cellfinder command-line tool. +The previous blog posts in this series outline these changes: + +- [Blog post](./brainreg_update_live.md) | `brainreg`, `brainreg-napari` have been merged, whilst the somewhat unrelated `brainreg-segment` package has been renamed to distinguish it. +- [Blog post](./cellfinder_migration_live.md) | The old command-line tool (which confusingly was called "cellfinder" despite being a combination of several BrainGlobe tools) has been moved to `brainmapper` in the `brainglobe-workflows` package. +- [Blog post](./core_and_napari_merge.md) | `cellfinder-core` and `cellfinder-napari` have been merged into just "`cellfinder`". + +The version 1 release of `brainglobe` on PyPI will provide you with the entire BrainGlobe tool suite and ensure that the tools are consistent with one another. +You will no longer need to worry about manually installing individual BrainGlobe tools and checking how they interface with other tools. + +Our vision for the new `brainglobe-workflows` package is to take this one step further and provide a selection of pre-written data analysis pipelines for neuroscientific data. +Currently the `brainglobe-workflows` package contains the successor to the old cellfinder command-line tool, now called `brainmapper`, but additional workflows will be added in the future. + +## Developers: what you need to know + +The `brainglobe` package allows us to ensure that tools cannot be installed in a manner which is inconsistent with each other. +If tool `A` necessitates a change in tool `B`, it was possible to update `A` without knowing about the knock-on effect on `B`, which would cause bugs further down the line. +Now the `brainglobe` package can be re-released in a manner which pins consistent versions of `A` and `B`, preventing this from happening. + +The `brainglobe-workflows` package allows us to do two further things: + +- It provides a distinct location for workflows that utilise BrainGlobe tools, so these workflows are not bundled together with their backends. +- It provides us with a set of analysis pipelines that we expect users to frequently run, which we can use to benchmark our tools and identify areas for improvement in terms of performance or speed. +- It builds on top of the `brainglobe` package, rather than having packages be interdependent on each other due to a workflow utilising multiple tools but being provided with its backend. + +## Full Changelog + +The full changelog is [available here](/community/releases/v1/index.md). diff --git a/docs/source/community/releases/v1/index.md b/docs/source/community/releases/v1/index.md index 41460bb0..d5f92b84 100644 --- a/docs/source/community/releases/v1/index.md +++ b/docs/source/community/releases/v1/index.md @@ -1,5 +1,24 @@ # BrainGlobe version 1 overview +BrainGlobe version 1 provides users with the `brainglobe` "meta"-package: a single package that will fetch all the tools in the BrainGlobe suite and install them onto your machine, ready for use. +It also provides the `brainglobe-workflows` package, which builds on top of `brainglobe` and provides some common analysis pipelines to users, as well as a benchmarking suite for developers working on code performance improvements. + +## Updating to version 1 + +Our recommendation for updating to BrainGlobe version 1 is to follow the [installation instructions on the main documentation page](/documentation/index.md#installing-brainglobe) in a clean virtual environment, and delete or archive your old environment. +This will ensure that when you install `brainglobe`, the tools are fetched in a consistent manner and do not pick up any manually installed BrainGlobe tools that might be in your environment or python path. +Likewise, it also avoids potential issues when tools have been installed from both `conda-forge` and `PyPI`. + +If you don't want to install the entire BrainGlobe suite and are happy to manage the tooling interdependencies yourself, you can still manually install the individual BrainGlobe tools that you wish to use. +If you choose to do so; as a general rule of thumb you should not mix `conda-forge` and `PyPI` installs, and avoid installing BrainGlobe tools of versions less than 1 in the same environment as those with version greater than 1 at all costs. + +### Previous installs with conda + +If you have previously installed any of our tools from conda, we strongly recommend that you create a new environment and follow the `pip` installation instructions provided on the [installation page](/documentation/index.md#installing-brainglobe). +The new "version 1" of BrainGlobe provides a PyPI package that comes with the complete suite of tools, and ensures that these are all consistent with one another. +These are fetched by `pip` and so may cause conflicts if they are already present in your environment, having been fetched from `conda-forge`. +Mixing `conda-forge` installed packages with the `brainglobe` package from PyPI will likely throw versioning issues as we can't guarantee that `conda` will resolve the tool dependencies correctly. + ## Changes at-a-glance Below is a high-level overview of the changes to the brainglobe suite of tools that we offer. @@ -13,6 +32,7 @@ The `cellfinder` command-line-interface has been moved into `brainglobe-workflow The cellfinder package is deprecated - it will later be recycled to merge some backend functionality | [Further info](cellfinder-migration.md#cellfinder-repository) | The cellfinder Docker image is discontinued | [Further info](cellfinder-migration.md#cellfinder-docker-image) | cellfinder-core and cellfinder-napari merged into "new cellfinder" | [Further info](cellfinder-core-and-plugin-merge.md) | +The cellfinder command-line tool has been replaced with `brainmapper` | [Further info](cellfinder-core-and-plugin-merge.md) | ## Complete index diff --git a/docs/source/documentation/index.md b/docs/source/documentation/index.md index b28cee14..7aeb42bf 100644 --- a/docs/source/documentation/index.md +++ b/docs/source/documentation/index.md @@ -1,5 +1,52 @@ # Documentation +## Installing BrainGlobe + +We always recommend that you install BrainGlobe tools into a virtual environment (managed by software such as `conda` or `venv`). +Your environment should run Python 3.9 or later. + +Once you have created and activated your desired environment, you can install all BrainGlobe tools using `pip`: + +```bash +pip install brainglobe +``` + +This will fetch all BrainGlobe tools, their Python APIs, and [`napari`](https://napari.org) plugins if they possess one. +It will also install `napari` into your environment so that you can make use of the plugins. + +If you also want to install our command-line tools and/or example workflows, you will also need to install the `brainglobe-workflows` package. +This package builds on top of the `brainglobe` tools, and provides you with some commonly-used data analysis pipelines to save you from having to write your own scripts for standard workflows. +You can read about the workflows that are provided [on the documentation page](./brainglobe-workflows/index.md). +To install `brainglobe-workflows`, run + +```bash +pip install brainglobe-workflows>=1.0.0 +``` + +in your environment. +If you didn't previously install `brainglobe` into this environment, `brainglobe-workflows` will do so when you run the command above. +Otherwise, it will use the version of `brainglobe` that you already have installed. + +### Updating + +You can update your BrainGlobe tools by running + +```bash +pip install brainglobe --update +``` + +in your virtual environment. + +### Installing Individual Tools + +We recommend installing BrainGlobe in the manner detailed above, however each of the tools in the suite can be individually installed if you so wish. +This can be done by following the install instructions for the particular tool in question. + +Please note that, if you choose to install the tools individually, we cannot guarantee that you will have a consistent and interoperable set of BrainGlobe tools - you may encounter version conflicts or dependency issues. +If you're not confident in resolving these yourself, we recommend installing BrainGlobe in the manner above; and updating your BrainGlobe tools by updating the overarching `brainglobe` package. + +## Individual Tool Documentation + ```{toctree} :maxdepth: 1 setting-up/index @@ -12,4 +59,4 @@ brainrender/index brainglobe-heatmap/index cellfinder/index morphapi/index -``` \ No newline at end of file +``` From 76337279997dc34b3294decaf576ffb5f8b3f9cb Mon Sep 17 00:00:00 2001 From: willGraham01 <1willgraham@gmail.com> Date: Fri, 5 Jan 2024 09:52:55 +0000 Subject: [PATCH 02/11] Add full changelog in one place with key overviews --- .../blog/version1/version_1_released.md | 11 +++++-- docs/source/community/releases/v1/index.md | 32 +++++++++++++++++-- 2 files changed, 38 insertions(+), 5 deletions(-) diff --git a/docs/source/blog/version1/version_1_released.md b/docs/source/blog/version1/version_1_released.md index 70b140cf..ecdeb4d4 100644 --- a/docs/source/blog/version1/version_1_released.md +++ b/docs/source/blog/version1/version_1_released.md @@ -20,6 +20,14 @@ BrainGlobe version 1 is hallmarked by the release of (version 1 of) the [`braing ## Users: what you need to know +The version 1 release of `brainglobe` on PyPI will provide you with the entire BrainGlobe tool suite and ensure that the tools are consistent with one another. +You will no longer need to worry about manually installing individual BrainGlobe tools and checking how they interface with other tools. +You can also easily install the complete BrainGlobe suite of tools on a new machine or into a fresh environment in a single `pip` command: + +```bash +pip install brainglobe>=1.0.0 +``` + The individual tools that you have been using are still available, and are largely still using the same name (with the exception of the old `cellfinder` command-line-tool, see below). Regardless of how many of our tools you use in your analysis, we recommend that you create a new environment and (re)install any BrainGlobe tools you have been using following the instructions on the installation page. @@ -30,9 +38,6 @@ The previous blog posts in this series outline these changes: - [Blog post](./cellfinder_migration_live.md) | The old command-line tool (which confusingly was called "cellfinder" despite being a combination of several BrainGlobe tools) has been moved to `brainmapper` in the `brainglobe-workflows` package. - [Blog post](./core_and_napari_merge.md) | `cellfinder-core` and `cellfinder-napari` have been merged into just "`cellfinder`". -The version 1 release of `brainglobe` on PyPI will provide you with the entire BrainGlobe tool suite and ensure that the tools are consistent with one another. -You will no longer need to worry about manually installing individual BrainGlobe tools and checking how they interface with other tools. - Our vision for the new `brainglobe-workflows` package is to take this one step further and provide a selection of pre-written data analysis pipelines for neuroscientific data. Currently the `brainglobe-workflows` package contains the successor to the old cellfinder command-line tool, now called `brainmapper`, but additional workflows will be added in the future. diff --git a/docs/source/community/releases/v1/index.md b/docs/source/community/releases/v1/index.md index d5f92b84..4b228253 100644 --- a/docs/source/community/releases/v1/index.md +++ b/docs/source/community/releases/v1/index.md @@ -33,8 +33,7 @@ The cellfinder package is deprecated - it will later be recycled to merge some b The cellfinder Docker image is discontinued | [Further info](cellfinder-migration.md#cellfinder-docker-image) | cellfinder-core and cellfinder-napari merged into "new cellfinder" | [Further info](cellfinder-core-and-plugin-merge.md) | The cellfinder command-line tool has been replaced with `brainmapper` | [Further info](cellfinder-core-and-plugin-merge.md) | - -## Complete index +`brainrender` and `brainrender-napari` are now available through the `brainglobe` meta-package | [Further info](/blog/version1/version_1_released.md) ```{toctree} :maxdepth: 1 @@ -44,3 +43,32 @@ brainreg cellfinder-migration cellfinder-core-and-plugin-merge ``` + +## Full Changelog + +- Recommended BrainGlobe install is via pip, into a clean environment. + - `pip install brainglobe` to install BrainGlobe tools. Recommended even if you only want to use one or two tools. + - `pip install brainglobe-workflows` if you also want to use the example analysis workflows that we provide. +- `brainreg-napari` is now shipped with `brainreg` + - `brainreg` functionality is now available in a submodule, `brainreg.core`. If you previously used the Python API provided in `brainreg`, you will need to change your `from brainreg import X` statements to `from brainreg.core import X`. + - The `brainreg-napari` plugin is now just called `brainreg` when using `napari`. The plugin's backend is in the submodule `brainreg.napari` rather than `brainreg-napari`. + - The new `brainreg.core` and `brainreg.napari` submodules retain the old internal structure of the respective deprecated packages. +- `brainreg-segment` has been renamed `brainglobe-segmentation`. + - Beyond the name change, package internals have not changed. +- `cellfinder` command-line-interface (CLI) has been renamed and moved. + - The command-line interface is now provided by `brainglobe-workflows`. + - The command-line tool is now called `brainmapper`. + - The name "cellfinder" is now reserved for the merged `cellfinder-core` and `cellfinder-napari` packages. +- `cellfinder-core` and `cellfinder-napari` have been merged into a single package, called `cellfinder`. + - The `cellfinder` package should not be confused with the old cellfinder command-line tool, described above. Going forward, "cellfinder" will refer exclusively to this new, merged package created from `cellfinder-core` and `cellfinder-napari`. The command-line tool with use the name `brianmapper`. + - The `core` and `napari` features are now available as submodules of the `cellfinder` package. If you previously used `from cellfinder_core import X`, you'll have to use `from cellfinder.core import X` instead. + - Internal function names and locations have otherwise not changed, beyond their conversion into the `core` and `napari` submodules. + - The napari plugin is now referred to as the `cellfinder` plugin, and shows up with the name `cellfinder` when viewed in napari (as opposed to the old `cellfinder-napari` name it used to have). +- `brainglobe-workflows` has been released. + - This package provides some exemplar data analysis workflows for users to modify or customise to suit their needs. + - It will also provide command-line interfaces for particularly common workflows. Currently it provides `brainmapper`, the successor to the _old_ cellfinder command-line tool. +- `pip install brainglobe` is now the recommended way to install all BrainGlobe tools. + - This will fetch the `brainglobe` meta-package, which includes the latest stable versions of all BrainGlobe tools and napari plugins. + - All tools will be accessible in the install environment as if they had been pip-installed directly. + - If you are updating to `brainglobe` version 1 having previously installed BrainGlobe tools individually, we [recommend you create](#updating-to-version-1) a clean Python environment and install `brainglobe` into this fresh environment to avoid potential package conflicts. + - `brainglobe` package now ships the fixed `brainrender`, as well as `brainrender-napari` which we plan to transition to in the future. From 4603ea1ac346f8b09e54978196918546e1d93d0e Mon Sep 17 00:00:00 2001 From: willGraham01 <1willgraham@gmail.com> Date: Fri, 5 Jan 2024 10:09:08 +0000 Subject: [PATCH 03/11] Add full changelog in one place with key overviews --- docs/source/blog/version1/version_1_released.md | 10 ++++++++++ 1 file changed, 10 insertions(+) diff --git a/docs/source/blog/version1/version_1_released.md b/docs/source/blog/version1/version_1_released.md index ecdeb4d4..eafa392c 100644 --- a/docs/source/blog/version1/version_1_released.md +++ b/docs/source/blog/version1/version_1_released.md @@ -28,6 +28,16 @@ You can also easily install the complete BrainGlobe suite of tools on a new mach pip install brainglobe>=1.0.0 ``` +The `brainglobe` package, or "meta-package" as we like to refer to it, ensures that the BrainGlobe tools installed onto your machine are stable and consistent with each other. +Key packages provided include: + +- `atlasapi` +- `brainreg` +- `brainrender` and `brainrender-napari` (new with version 1) +- `brainglobe-napari-io` +- `brainglobe-segmentation` +- `cellfinder` + The individual tools that you have been using are still available, and are largely still using the same name (with the exception of the old `cellfinder` command-line-tool, see below). Regardless of how many of our tools you use in your analysis, we recommend that you create a new environment and (re)install any BrainGlobe tools you have been using following the instructions on the installation page. From 6fd247c4bb5f6be02a92239097650ec3cf653500 Mon Sep 17 00:00:00 2001 From: Will Graham <32364977+willGraham01@users.noreply.github.com> Date: Fri, 5 Jan 2024 10:37:11 +0000 Subject: [PATCH 04/11] Update docs/source/blog/version1/version_1_released.md Co-authored-by: Adam Tyson --- docs/source/blog/version1/version_1_released.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/source/blog/version1/version_1_released.md b/docs/source/blog/version1/version_1_released.md index eafa392c..c8585bc8 100644 --- a/docs/source/blog/version1/version_1_released.md +++ b/docs/source/blog/version1/version_1_released.md @@ -31,7 +31,7 @@ pip install brainglobe>=1.0.0 The `brainglobe` package, or "meta-package" as we like to refer to it, ensures that the BrainGlobe tools installed onto your machine are stable and consistent with each other. Key packages provided include: -- `atlasapi` +- `bg-atlasapi` - `brainreg` - `brainrender` and `brainrender-napari` (new with version 1) - `brainglobe-napari-io` From 9e57224e5f84409e0d5fa5b9760dbe612e77f8ef Mon Sep 17 00:00:00 2001 From: Will Graham <32364977+willGraham01@users.noreply.github.com> Date: Fri, 5 Jan 2024 13:25:39 +0000 Subject: [PATCH 05/11] Apply suggestions from code review Co-authored-by: Adam Tyson --- docs/source/blog/version1/version_1_released.md | 4 ++-- docs/source/community/releases/v1/index.md | 4 ++-- docs/source/documentation/index.md | 6 +++--- 3 files changed, 7 insertions(+), 7 deletions(-) diff --git a/docs/source/blog/version1/version_1_released.md b/docs/source/blog/version1/version_1_released.md index c8585bc8..264e7932 100644 --- a/docs/source/blog/version1/version_1_released.md +++ b/docs/source/blog/version1/version_1_released.md @@ -7,7 +7,7 @@ category: BrainGlobe-v1 language: English --- -# BrainGlobe version 1 is here! :partying_face: +# BrainGlobe version 1 is here! Following our series of incremental updates to a number of BrainGlobe tools, we are pleased to announce that BrainGlobe version 1 has been released today! Users can now enjoy: @@ -46,7 +46,7 @@ The previous blog posts in this series outline these changes: - [Blog post](./brainreg_update_live.md) | `brainreg`, `brainreg-napari` have been merged, whilst the somewhat unrelated `brainreg-segment` package has been renamed to distinguish it. - [Blog post](./cellfinder_migration_live.md) | The old command-line tool (which confusingly was called "cellfinder" despite being a combination of several BrainGlobe tools) has been moved to `brainmapper` in the `brainglobe-workflows` package. -- [Blog post](./core_and_napari_merge.md) | `cellfinder-core` and `cellfinder-napari` have been merged into just "`cellfinder`". +- [Blog post](./core_and_napari_merge.md) | `cellfinder-core` and `cellfinder-napari` have been merged into just `cellfinder`. Our vision for the new `brainglobe-workflows` package is to take this one step further and provide a selection of pre-written data analysis pipelines for neuroscientific data. Currently the `brainglobe-workflows` package contains the successor to the old cellfinder command-line tool, now called `brainmapper`, but additional workflows will be added in the future. diff --git a/docs/source/community/releases/v1/index.md b/docs/source/community/releases/v1/index.md index 4b228253..d8a1acd9 100644 --- a/docs/source/community/releases/v1/index.md +++ b/docs/source/community/releases/v1/index.md @@ -10,7 +10,7 @@ This will ensure that when you install `brainglobe`, the tools are fetched in a Likewise, it also avoids potential issues when tools have been installed from both `conda-forge` and `PyPI`. If you don't want to install the entire BrainGlobe suite and are happy to manage the tooling interdependencies yourself, you can still manually install the individual BrainGlobe tools that you wish to use. -If you choose to do so; as a general rule of thumb you should not mix `conda-forge` and `PyPI` installs, and avoid installing BrainGlobe tools of versions less than 1 in the same environment as those with version greater than 1 at all costs. +If you choose to do so; as a general rule of thumb you should not mix `conda-forge` and `PyPI` installs, and avoid installing BrainGlobe tools of versions less than 1 in the same environment as those with version greater than 1. ### Previous installs with conda @@ -60,7 +60,7 @@ cellfinder-core-and-plugin-merge - The command-line tool is now called `brainmapper`. - The name "cellfinder" is now reserved for the merged `cellfinder-core` and `cellfinder-napari` packages. - `cellfinder-core` and `cellfinder-napari` have been merged into a single package, called `cellfinder`. - - The `cellfinder` package should not be confused with the old cellfinder command-line tool, described above. Going forward, "cellfinder" will refer exclusively to this new, merged package created from `cellfinder-core` and `cellfinder-napari`. The command-line tool with use the name `brianmapper`. + - The `cellfinder` package should not be confused with the old cellfinder command-line tool, described above. Going forward, "cellfinder" will refer exclusively to this new, merged package created from `cellfinder-core` and `cellfinder-napari` which only performs cell detection. The command-line tool with use the name `brainmapper`. - The `core` and `napari` features are now available as submodules of the `cellfinder` package. If you previously used `from cellfinder_core import X`, you'll have to use `from cellfinder.core import X` instead. - Internal function names and locations have otherwise not changed, beyond their conversion into the `core` and `napari` submodules. - The napari plugin is now referred to as the `cellfinder` plugin, and shows up with the name `cellfinder` when viewed in napari (as opposed to the old `cellfinder-napari` name it used to have). diff --git a/docs/source/documentation/index.md b/docs/source/documentation/index.md index 7aeb42bf..b8a4f46d 100644 --- a/docs/source/documentation/index.md +++ b/docs/source/documentation/index.md @@ -3,7 +3,7 @@ ## Installing BrainGlobe We always recommend that you install BrainGlobe tools into a virtual environment (managed by software such as `conda` or `venv`). -Your environment should run Python 3.9 or later. +Your environment should run Python 3.9 or 3.10. Once you have created and activated your desired environment, you can install all BrainGlobe tools using `pip`: @@ -14,7 +14,7 @@ pip install brainglobe This will fetch all BrainGlobe tools, their Python APIs, and [`napari`](https://napari.org) plugins if they possess one. It will also install `napari` into your environment so that you can make use of the plugins. -If you also want to install our command-line tools and/or example workflows, you will also need to install the `brainglobe-workflows` package. +If you also want to install our analysis workflows (such as `brainmapper`, previously the `cellfinder` CLI), you will also need to install the `brainglobe-workflows` package. This package builds on top of the `brainglobe` tools, and provides you with some commonly-used data analysis pipelines to save you from having to write your own scripts for standard workflows. You can read about the workflows that are provided [on the documentation page](./brainglobe-workflows/index.md). To install `brainglobe-workflows`, run @@ -43,7 +43,7 @@ We recommend installing BrainGlobe in the manner detailed above, however each of This can be done by following the install instructions for the particular tool in question. Please note that, if you choose to install the tools individually, we cannot guarantee that you will have a consistent and interoperable set of BrainGlobe tools - you may encounter version conflicts or dependency issues. -If you're not confident in resolving these yourself, we recommend installing BrainGlobe in the manner above; and updating your BrainGlobe tools by updating the overarching `brainglobe` package. +If you're not confident in resolving these yourself, we recommend installing BrainGlobe in the manner above, and updating your BrainGlobe tools by updating the overarching `brainglobe` package. ## Individual Tool Documentation From 6829a7b364529569070cc964faf7937164d2867f Mon Sep 17 00:00:00 2001 From: willGraham01 <1willgraham@gmail.com> Date: Fri, 5 Jan 2024 14:08:54 +0000 Subject: [PATCH 06/11] Apply Adam's suggestions --- docs/source/blog/version1/version_1_released.md | 7 ++++--- docs/source/community/releases/v1/index.md | 11 +++++------ 2 files changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/source/blog/version1/version_1_released.md b/docs/source/blog/version1/version_1_released.md index 264e7932..4e2b766b 100644 --- a/docs/source/blog/version1/version_1_released.md +++ b/docs/source/blog/version1/version_1_released.md @@ -32,10 +32,11 @@ The `brainglobe` package, or "meta-package" as we like to refer to it, ensures t Key packages provided include: - `bg-atlasapi` -- `brainreg` -- `brainrender` and `brainrender-napari` (new with version 1) +- `brainglobe-heatmap` - `brainglobe-napari-io` - `brainglobe-segmentation` +- `brainreg` +- `brainrender` and `brainrender-napari` (new with version 1) - `cellfinder` The individual tools that you have been using are still available, and are largely still using the same name (with the exception of the old `cellfinder` command-line-tool, see below). @@ -45,7 +46,7 @@ For the most part, the effects of moving to version 1 should be limited to the c The previous blog posts in this series outline these changes: - [Blog post](./brainreg_update_live.md) | `brainreg`, `brainreg-napari` have been merged, whilst the somewhat unrelated `brainreg-segment` package has been renamed to distinguish it. -- [Blog post](./cellfinder_migration_live.md) | The old command-line tool (which confusingly was called "cellfinder" despite being a combination of several BrainGlobe tools) has been moved to `brainmapper` in the `brainglobe-workflows` package. +- Blog posts: [Initial move](./cellfinder_migration_live.md) and [renaming](./core_and_napari_merge.md) | The old command-line tool (which confusingly was called "cellfinder" despite being a combination of several BrainGlobe tools) has been moved to `brainmapper` in the `brainglobe-workflows` package. - [Blog post](./core_and_napari_merge.md) | `cellfinder-core` and `cellfinder-napari` have been merged into just `cellfinder`. Our vision for the new `brainglobe-workflows` package is to take this one step further and provide a selection of pre-written data analysis pipelines for neuroscientific data. diff --git a/docs/source/community/releases/v1/index.md b/docs/source/community/releases/v1/index.md index d8a1acd9..125cd582 100644 --- a/docs/source/community/releases/v1/index.md +++ b/docs/source/community/releases/v1/index.md @@ -10,14 +10,13 @@ This will ensure that when you install `brainglobe`, the tools are fetched in a Likewise, it also avoids potential issues when tools have been installed from both `conda-forge` and `PyPI`. If you don't want to install the entire BrainGlobe suite and are happy to manage the tooling interdependencies yourself, you can still manually install the individual BrainGlobe tools that you wish to use. -If you choose to do so; as a general rule of thumb you should not mix `conda-forge` and `PyPI` installs, and avoid installing BrainGlobe tools of versions less than 1 in the same environment as those with version greater than 1. -### Previous installs with conda +### Can I install from `conda-forge`? -If you have previously installed any of our tools from conda, we strongly recommend that you create a new environment and follow the `pip` installation instructions provided on the [installation page](/documentation/index.md#installing-brainglobe). -The new "version 1" of BrainGlobe provides a PyPI package that comes with the complete suite of tools, and ensures that these are all consistent with one another. -These are fetched by `pip` and so may cause conflicts if they are already present in your environment, having been fetched from `conda-forge`. -Mixing `conda-forge` installed packages with the `brainglobe` package from PyPI will likely throw versioning issues as we can't guarantee that `conda` will resolve the tool dependencies correctly. +We are working on providing a `conda-forge` install for BrainGlobe version 1, as an alternative to the `pip install brainglobe` command that we currently recommend. +However for the time being, there is no one-line installation possible using `conda`. + +There are no problems with `pip install brainglobe`-ing into a `conda` environment, we just need some time to setup all of the individual BrainGlobe tools on `conda-forge` before we can provide an all-in-one install through `conda`! ## Changes at-a-glance From c5c019da2ad59cc4ca02b726b5161ec6046bb569 Mon Sep 17 00:00:00 2001 From: willGraham01 <1willgraham@gmail.com> Date: Fri, 5 Jan 2024 15:02:27 +0000 Subject: [PATCH 07/11] Add meta-package developer docs --- .../brainglobe-dependencies.svg | 1573 +++++++++++++++++ .../repositories/brainglobe-meta/index.md | 84 + docs/source/community/releases/v1/index.md | 2 + 3 files changed, 1659 insertions(+) create mode 100644 docs/source/community/developers/repositories/brainglobe-meta/brainglobe-dependencies.svg create mode 100644 docs/source/community/developers/repositories/brainglobe-meta/index.md diff --git a/docs/source/community/developers/repositories/brainglobe-meta/brainglobe-dependencies.svg b/docs/source/community/developers/repositories/brainglobe-meta/brainglobe-dependencies.svg new file mode 100644 index 00000000..0fb14e18 --- /dev/null +++ b/docs/source/community/developers/repositories/brainglobe-meta/brainglobe-dependencies.svg @@ -0,0 +1,1573 @@ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + diff --git a/docs/source/community/developers/repositories/brainglobe-meta/index.md b/docs/source/community/developers/repositories/brainglobe-meta/index.md new file mode 100644 index 00000000..092155cc --- /dev/null +++ b/docs/source/community/developers/repositories/brainglobe-meta/index.md @@ -0,0 +1,84 @@ +# brainglobe-meta + +`pip install brainglobe` is doing what you expect - installing a package from PyPI called `brainglobe`. +This package, colloquially referred to as _the_ "meta"-package, `brainglobe`, and/or `brainglobe-meta`, doesn't actually contain any functionality beyond providing a version and some author information. +Instead, it's primary purpose is to (explicitly) depend on the other BrainGlobe tools that we want users to be able to install in a single command, through this package. +You can find the source for this package in the [`brainglobe-meta` repository](https://github.com/brainglobe/brainglobe-meta) on GitHub. + +## Installation + +The meta-package should always be installable in one line, and preferably should be installed for the first time in a clean environment. + +### From `PyPI`, via `pip` + +`brainglobe` can be installed from PyPI via + +```bash +pip install brainglobe +``` + +or + +```bash +python -m pip install brainglobe +``` + +Pass the `-U` or `--update` flag to `pip` to force an update on an existing install. + +## Conventions + +### Versioning + +The rule of thumb for versioning and releasing the meta-package (creating a "new version of `brainglobe`"): + +> Any `brainglobe` vX.0.0 release is a set of stable, compatible tools that interoperate with each other consistently. +> A `brainglobe` vX.Y.0 or vX.Y.Z release should be used when there have been several minor or patch updates to the BrainGlobe tool suite, that we want to encourage the user-base to adopt. + +As such, we typically envision major version releases of the meta-package taking place when one or more BrainGlobe tools undergo major updates. +An example would be the release of `brainglobe` version 1; which saw the various `brainreg` packages merged, the `cellfinder` package was repurposed in order to merge `cellfinder-core` and `cellfinder-napari`, and the `brainmapper` tool replace the (old) cellfinder command-line interface. +The resulting `brainglobe` version 1.0.0 release prevents users from installing incompatible versions of `brainreg` and `cellfinder`, for example, both by removing the need for users to install these tools manually and by pinning minimum versions for each. + +Minor or patch updates to `brainglobe` we envision to be cases where multiple tools have undergone performance improvements, minor patches, or under-the-hood code refactoring that we feel benefits the user experience, but does not require a change in how the user interacts with these tools. + +### Dependency Pinning + +With the above conventions on versioning in mind, meta-package dependencies are always **pinned from below, and from above by the next major version**. +As such, all BrainGlobe dependencies in the meta-package `pyproject.toml` will have the form: + +```toml +dependencies = [ + "brainglobe-tool-1>=X.Y.Z,<(X+1)", +] +``` + +for example, + +```toml +dependencies = [ + "cellfinder>=1.1.0,<2", + "brainrender>=2.1.3,<3", +] +``` + +This ensures that: + +- Minor and patch updates to BrainGlobe tools should be picked up automatically by package installer when updating the metapackage. For example, if `cellfinder` bumps from `1.1.3` to `1.2.0`, updating the meta-package will result in `cellfinder` being updated to `1.2.0`. +- Major or breaking changes to BrainGlobe tools are **not** automatically picked up on update! +- We can force certain (combinations of) minor/patch updates to tools on users by simply releasing minor/patch updates to `brainglobe`. Changing `cellfinder>=1.1.0,<2` to `cellfinder>=1.1.4,<2` and `brainrender>=2.1.3,<3"` to `brainrender>=2.2.1,<3"`, then creating a new `brainglobe` minor/patch release, allows the user to update both of these packages with an update to `brainglobe`, not having to worry about the individual packages themselves. + +## Dependency Tree + +BrainGlobe comprises a number of tools, spread across a number of repositories. +These tools are all interlinked and interdependent, but the meta-package allows us to abstract this complexity away from the user. + +To ensure we retain explicit control over what `brainglobe` provides to users, it is important that the meta-package explicitly depends on all BrainGlobe tools that it wants to provide, including "redundant dependencies". +For example, if the meta-package is providing `brainglobe-tool-A`, which itself depends on `brainglobe-tool-B`, then the dependency list of the meta-package should include **both** `brainglobe-tool-A` and `brainglobe-tool-B`. +Both packages should also have minimum versions pinned as per the [dependency pinning guidelines](#dependency-pinning) above. + +The dependency tree of BrainGlobe tools provided by the meta-package is provided below, and should be updated periodically as new tools are added, or new versions of `brainglobe` are released. +Whilst it is not necessary for developers to remember the dependency structure, it is a useful reference guide for discerning how updates to one BrainGlobe tool might have knock-on effects on others. +The dependency tree reads top-to-bottom; that is, arrows are drawn **out** of dependencies **into** the tools that depend on them. +The tree does not include non-BrainGlobe dependencies. +Coloured arrows are only present to allow for line-tracing when arrows cross. + +![Dependency tree for tools provided by the meta-package.](./brainglobe-dependencies.svg) diff --git a/docs/source/community/releases/v1/index.md b/docs/source/community/releases/v1/index.md index 125cd582..e7a61818 100644 --- a/docs/source/community/releases/v1/index.md +++ b/docs/source/community/releases/v1/index.md @@ -3,6 +3,8 @@ BrainGlobe version 1 provides users with the `brainglobe` "meta"-package: a single package that will fetch all the tools in the BrainGlobe suite and install them onto your machine, ready for use. It also provides the `brainglobe-workflows` package, which builds on top of `brainglobe` and provides some common analysis pipelines to users, as well as a benchmarking suite for developers working on code performance improvements. +If you want to know more about the under-the-hood workings of the `brainglobe` "meta"-package, you can head to the [developer documentation](/docs/source/community/developers/repositories/brainglobe-meta/index.md). + ## Updating to version 1 Our recommendation for updating to BrainGlobe version 1 is to follow the [installation instructions on the main documentation page](/documentation/index.md#installing-brainglobe) in a clean virtual environment, and delete or archive your old environment. From 860db4ac7c22f2059bd03fd583320d66e7a69685 Mon Sep 17 00:00:00 2001 From: willGraham01 <1willgraham@gmail.com> Date: Fri, 5 Jan 2024 15:05:47 +0000 Subject: [PATCH 08/11] Add dev link to meta-package docs from main documentation page --- docs/source/documentation/index.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/docs/source/documentation/index.md b/docs/source/documentation/index.md index b8a4f46d..6ecf888f 100644 --- a/docs/source/documentation/index.md +++ b/docs/source/documentation/index.md @@ -47,6 +47,8 @@ If you're not confident in resolving these yourself, we recommend installing Bra ## Individual Tool Documentation +Once you have installed `brainglobe`, or [installed an individual tool](#installing-individual-tools) if you choose to go that route, you can lookup how to use it by following the links below. + ```{toctree} :maxdepth: 1 setting-up/index @@ -60,3 +62,5 @@ brainglobe-heatmap/index cellfinder/index morphapi/index ``` + +Developers may also wish to review the documentation [for the `brainglobe` package](/docs/source/community/developers/repositories/brainglobe-meta/index.md) itself. From 782d1e98c8cafc236f0bc2aad68df148f59dfa75 Mon Sep 17 00:00:00 2001 From: willGraham01 <1willgraham@gmail.com> Date: Fri, 5 Jan 2024 15:22:04 +0000 Subject: [PATCH 09/11] Clarify minor metapackage releases --- .../community/developers/repositories/brainglobe-meta/index.md | 3 +++ 1 file changed, 3 insertions(+) diff --git a/docs/source/community/developers/repositories/brainglobe-meta/index.md b/docs/source/community/developers/repositories/brainglobe-meta/index.md index 092155cc..d5454cad 100644 --- a/docs/source/community/developers/repositories/brainglobe-meta/index.md +++ b/docs/source/community/developers/repositories/brainglobe-meta/index.md @@ -39,6 +39,9 @@ An example would be the release of `brainglobe` version 1; which saw the various The resulting `brainglobe` version 1.0.0 release prevents users from installing incompatible versions of `brainreg` and `cellfinder`, for example, both by removing the need for users to install these tools manually and by pinning minimum versions for each. Minor or patch updates to `brainglobe` we envision to be cases where multiple tools have undergone performance improvements, minor patches, or under-the-hood code refactoring that we feel benefits the user experience, but does not require a change in how the user interacts with these tools. +A typical example of this might be when a function appears in both `brainglobe-tool-A` and `brainglobe-tool-B`, so we decide to refactor it into `brainglobe-utils` which is a common dependency. +In this case; each of `brainglobe-tool-A`, `brainglobe-tool-B`, and `brainglobe-utils` would get new releases, and furthermore both `brainglobe-tool-A` and `brainglobe-tool-B` would now depend on the new version of `brainglobe-utils`. +We would then decide to release a minor version of the meta-package, which updates the minimum version of these three packages - from the user perspective, this will come in a single update. ### Dependency Pinning From 6919911628c3cb1ec1e4c2bdc913e0f963b67cfb Mon Sep 17 00:00:00 2001 From: willGraham01 <1willgraham@gmail.com> Date: Mon, 8 Jan 2024 09:40:23 +0000 Subject: [PATCH 10/11] Add regenerating diagram text --- .../repositories/brainglobe-meta/index.md | 14 ++++++++++++++ 1 file changed, 14 insertions(+) diff --git a/docs/source/community/developers/repositories/brainglobe-meta/index.md b/docs/source/community/developers/repositories/brainglobe-meta/index.md index d5454cad..edf81415 100644 --- a/docs/source/community/developers/repositories/brainglobe-meta/index.md +++ b/docs/source/community/developers/repositories/brainglobe-meta/index.md @@ -85,3 +85,17 @@ The tree does not include non-BrainGlobe dependencies. Coloured arrows are only present to allow for line-tracing when arrows cross. ![Dependency tree for tools provided by the meta-package.](./brainglobe-dependencies.svg) + +### Regenerating the Dependency Tree + +The image file containing the dependency tree is saved under `/docs/source/community/developers/repositories/brainglobe-meta/brainglobe-dependencies.svg`. +It can be edited with any software that can open and manipulate `.svg` files, however the recommended tool is [`drawio`](https://app.diagrams.net/) which provides a GUI for editing `.svg`s. +Simply upload the current version of the file to the website, edit the flowchart as you see fit, then hit `File > Export As... > SVG`, which will open a further dialogue box with export options. +In this box, be sure to select "`Text Settings: Convert Labels to SVG`" to ensure that the exported flowchart's text is encoded correctly. +You can then export the file and save the new version to the repository. + +As a general rule of thumb when editing the dependency chart, packages/tools should be organised into levels from the top down, with arrows pointing _out of dependencies_ **into** _the dependent packages_. + +Packages on the top level depend on no other BrainGlobe tools. +Packages on the level below depend on at least one package from the level above, and any number of packages from the level(s) further up than that. +This illustrates both how BrainGlobe tools build on each other, as well as which tools can be considered ambivalent to new releases of others. From 1bb20ae0c721ad083bd7da6e4591e9f7e80e33ac Mon Sep 17 00:00:00 2001 From: Will Graham <32364977+willGraham01@users.noreply.github.com> Date: Mon, 8 Jan 2024 10:26:18 +0000 Subject: [PATCH 11/11] Apply suggestions from code review Co-authored-by: Alessandro Felder --- .../developers/repositories/brainglobe-meta/index.md | 2 +- docs/source/community/releases/v1/index.md | 2 +- docs/source/documentation/index.md | 5 ++++- 3 files changed, 6 insertions(+), 3 deletions(-) diff --git a/docs/source/community/developers/repositories/brainglobe-meta/index.md b/docs/source/community/developers/repositories/brainglobe-meta/index.md index edf81415..33d4d3ed 100644 --- a/docs/source/community/developers/repositories/brainglobe-meta/index.md +++ b/docs/source/community/developers/repositories/brainglobe-meta/index.md @@ -98,4 +98,4 @@ As a general rule of thumb when editing the dependency chart, packages/tools sho Packages on the top level depend on no other BrainGlobe tools. Packages on the level below depend on at least one package from the level above, and any number of packages from the level(s) further up than that. -This illustrates both how BrainGlobe tools build on each other, as well as which tools can be considered ambivalent to new releases of others. +This illustrates both how BrainGlobe tools build on each other, as well as which tools may be affected by new releases of others. diff --git a/docs/source/community/releases/v1/index.md b/docs/source/community/releases/v1/index.md index e7a61818..fdfc338c 100644 --- a/docs/source/community/releases/v1/index.md +++ b/docs/source/community/releases/v1/index.md @@ -61,7 +61,7 @@ cellfinder-core-and-plugin-merge - The command-line tool is now called `brainmapper`. - The name "cellfinder" is now reserved for the merged `cellfinder-core` and `cellfinder-napari` packages. - `cellfinder-core` and `cellfinder-napari` have been merged into a single package, called `cellfinder`. - - The `cellfinder` package should not be confused with the old cellfinder command-line tool, described above. Going forward, "cellfinder" will refer exclusively to this new, merged package created from `cellfinder-core` and `cellfinder-napari` which only performs cell detection. The command-line tool with use the name `brainmapper`. + - The `cellfinder` package should not be confused with the old cellfinder command-line tool, described above. Going forward, "cellfinder" will refer exclusively to this new, merged package created from `cellfinder-core` and `cellfinder-napari` which only performs cell detection. The command-line tool will use the name `brainmapper`. - The `core` and `napari` features are now available as submodules of the `cellfinder` package. If you previously used `from cellfinder_core import X`, you'll have to use `from cellfinder.core import X` instead. - Internal function names and locations have otherwise not changed, beyond their conversion into the `core` and `napari` submodules. - The napari plugin is now referred to as the `cellfinder` plugin, and shows up with the name `cellfinder` when viewed in napari (as opposed to the old `cellfinder-napari` name it used to have). diff --git a/docs/source/documentation/index.md b/docs/source/documentation/index.md index 6ecf888f..1a0a45d4 100644 --- a/docs/source/documentation/index.md +++ b/docs/source/documentation/index.md @@ -3,7 +3,10 @@ ## Installing BrainGlobe We always recommend that you install BrainGlobe tools into a virtual environment (managed by software such as `conda` or `venv`). -Your environment should run Python 3.9 or 3.10. +Your environment should run Python 3.9 or 3.10. To specify the Python version for new conda environment, add it as a parameter on creation: + +```bash +conda create -n brainglobe-env python=3.10 Once you have created and activated your desired environment, you can install all BrainGlobe tools using `pip`: