From 409af0b49597eafd5dcf3d469cf6aa55558844cc Mon Sep 17 00:00:00 2001 From: Alyssa Dai Date: Thu, 8 Aug 2024 02:19:24 -0400 Subject: [PATCH 1/5] remove deprecated non-federation query tool from docs --- docs/api.md | 4 ++-- docs/config.md | 47 ++++++++++++++++++++--------------------- docs/getting_started.md | 19 ++++++++--------- 3 files changed, 34 insertions(+), 36 deletions(-) diff --git a/docs/api.md b/docs/api.md index bcd54ded..c0971214 100644 --- a/docs/api.md +++ b/docs/api.md @@ -7,8 +7,8 @@ Neurobagel has two flavours of APIs that can be deployed: **node API** and **fed - A [Neurobagel node API (n-API)](https://github.com/neurobagel/api) formulates SPARQL queries based on a set of user-defined parameters to a single connected graph database, and processes returned query results into a user-friendly format. - A [Neurobagel federation API (f-API)](https://github.com/neurobagel/federation-api) lets the user sends a single query to _each_ node API it is aware of, and collects and combines the decentralized responses into a single set of query results. -Neurobagel's query tool provides a GUI for querying one or more Neurobagel graphs by sending requests to a Neurobagel API instance (n-API OR f-API). -However, HTTP requests can also be sent directly to a Neurobagel API (node or federation). +Neurobagel's query tool provides a GUI for querying one or more Neurobagel graphs by sending requests to a Neurobagel _federation API_ instance. +However, HTTP requests can also be sent directly to any publicly accessible Neurobagel API (node or federation). ## Public Neurobagel APIs diff --git a/docs/config.md b/docs/config.md index 11151ae3..cac02728 100644 --- a/docs/config.md +++ b/docs/config.md @@ -15,46 +15,45 @@ and coordinates them to work together: (In parentheses are the names of services within the Docker Compose stack) -- [Neurobagel node API](api.md) (`api`): The API that communicates with the graph store and determines - how detailed the response to a query should be. -- Graph store (`graph`): A third-party RDF store that stores Neurobagel-harmonized data to be queried. At the moment our recipe uses the free tier +- **[Neurobagel node API/n-API](api.md)** (`api`): The API that communicates with a single graph store and determines + how detailed the response to a query should be from that graph. +- **Graph store** (`graph`): A third-party RDF store that stores Neurobagel-harmonized data to be queried. At the moment our recipe uses the free tier of [GraphDB](https://db-engines.com/en/system/GraphDB) for this. -- Neurobagel federation API (`federation`): A special API that can federate over - multiple Neurobagel nodes to provide a single point of access to multiple nodes. +- **Neurobagel federation/f-API** (`federation`): A special API that can federate over one or more + Neurobagel nodes to provide a single point of access to multiple distributed databases. By default it will federate over all public nodes and any local nodes you specify. -- [Neurobagel query tool](query_tool.md): A graphical web tool allows users to query the federation API (or node API) - and visualize the results. Because the query tool is a static app and is run locally - in the users browser, this service simply hosts the app. +- **[Neurobagel query tool](query_tool.md)** (`query_tool`): A web app that provides a graphical interface for users to query a + federation API and view the results from one or more nodes. Because the query tool is a static app and is run locally + in the user's browser, this service simply hosts the app. ### Available profiles Neurobagel offers different deployment profiles that allow you to spin up specific combinations of services (listed below), depending on your use case. 1. `full_stack`: Best profile to get started with Neurobagel. - It includes all services you need to run a single standalone Neurobagel node, including a graphical query tool. - :information_source: By default this profile will also federate over all publicly accessible neurobagel nodes. + It includes all services you need to run a local Neurobagel node and have the ability to query public nodes, along with a graphical query tool. - `api` - `graph` - `federation` - `query_tool` + !!! info + By default this profile will also federate over all publicly accessible Neurobagel nodes, although this behaviour can be disabled in the f-API using the environment variable [`NB_FEDERATE_REMOTE_PUBLIC_NODES`](#environment-variables). + 2. `local_node`: Best profile if you want to run a standalone Neurobagel node - and rely on a different deployment for providing federation and the query tool. - :information_source: **This is the default profile** if you don't specify one. + but rely on a separate deployment for providing federation and a graphical query tool (such as Neurobagel's own hosted public instances). - `api` - `graph` + !!! info + This is the **default profile** if you don't specify one. + 3. `local_federation`: Best profile if you already have multiple standalone (local or non-publicly-accessible) Neurobagel node - deployments running and you now want to provide federation over them. - :information_source: If you only want to federate over a local node and all public Neurobagel nodes - we recommend using the `full_stack` profile to set up your node and federation in one step. If you use the `local_federation` profile, - you will have to [manually configure your `local_nb_nodes.json` file](#local_nb_nodesjson). - - `federation` - - `query_tool` -4. `local_node_query`: :warning: Deprecated profile. - This profile lets you create a local node without - federation. The query tool hosted by this deployment will talk only to the - local node. - - `api` - - `graph` + deployments running and you now want to provide federation over them. + - `federation` - `query_tool` + !!! info + If you only want to federate over a single local node and all public Neurobagel nodes, + we recommend using the `full_stack` profile to set up your node and federation in one step. + If you choose to use the `local_federation` profile, + you will have to [manually configure your `local_nb_nodes.json` file](#local_nb_nodesjson). You can then launch these profiles by using the `--profile` flag with `docker compose`, e.g.: diff --git a/docs/getting_started.md b/docs/getting_started.md index 79129073..8b5d1f7c 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -88,20 +88,19 @@ cd recipes cp template.env .env cp local_nb_nodes.template.json local_nb_nodes.json ``` -3. Change `NB_API_QUERY_URL` in the `.env` file +3. Change `NB_API_QUERY_URL` in the `.env` file You **must** replace the placeholder value for `NB_API_QUERY_URL`in the `.env` file ```bash NB_API_QUERY_URL=http://XX.XX.XX.XX ``` - with the address of the machine you are going to deploy Neurobagel on. + with the URL address where the Neurobagel federation API will be accessed: - - If you are deploying Neurobagel **for yourself** or just to try things out locally, - you can use: `NB_API_QUERY_URL=http://localhost:8080` - where `:8080` is the default port for the federation API. - - If you are deploying Neurobagel on a server for other users, - you **must** use the IP (and Port) or URL that your users will access the server with. + - If you are deploying Neurobagel for yourself or deploying and trying the services **on your local machine only**, + you can use `NB_API_QUERY_URL=http://localhost:8080`, where `8080` is the [default host port for the federation API](./config.md#environment-variables). + - If you are deploying Neurobagel **on a server for other users**, + you must use the IP (and port) or URL intended for your users to access the federation API on the server with. !!! info @@ -152,9 +151,9 @@ our [service profile documentation](config.md#available-profiles) for details. :tada: You are now the proud owner of a running Neurobagel node. Here are some things you can do now: - Try the Neurobagel node you just deployed by accessing: - - your own query tool at [http://localhost:3000](http://localhost:3000) and reading the [query tool](./query_tool.md) guide - - the interactive API docs at [http://localhost:8000/docs](http://localhost:8000/docs) and reading the [API](./api.md) guides + - your own query tool at [http://localhost:3000](http://localhost:3000), and reading the [query tool usage](./query_tool.md#usage) guide + - the interactive docs for your node API at [http://localhost:8000/docs](http://localhost:8000/docs), and reading the [API usage](./api.md) guide - [Prepare your own dataset](./data_prep.md) for annotation with Neurobagel - [Add your own data to your Neurobagel graph](maintaining.md#updating-the-data-in-your-graph) to search - Learn about the different [configuration options](config.md) for your Neurobagel node -- Hopefully all went well, but if you are experiencing issues take a look at our [getting help guide](./getting_help.md) \ No newline at end of file +- Hopefully all went well, but if you are experiencing issues, see how to [get help](./getting_help.md) \ No newline at end of file From b50599a30f7d55daec0ba7812d5a07043bb8af1f Mon Sep 17 00:00:00 2001 From: Alyssa Dai Date: Thu, 8 Aug 2024 16:29:34 -0400 Subject: [PATCH 2/5] minor grammatical / formatting edits --- docs/config.md | 15 +++++++++------ docs/getting_started.md | 2 +- 2 files changed, 10 insertions(+), 7 deletions(-) diff --git a/docs/config.md b/docs/config.md index cac02728..a13e8dc8 100644 --- a/docs/config.md +++ b/docs/config.md @@ -10,7 +10,7 @@ for the changes to take effect. ## Deployment ### Available services -The Neurobagel docker compose recipe includes several services +The Neurobagel Docker Compose recipe includes several services and coordinates them to work together: (In parentheses are the names of services within the Docker Compose stack) @@ -89,7 +89,7 @@ At minimum, we recommend reviewing and changing the values of the following vari > `NB_GRAPH_ADMIN_PASSWORD` > `NB_GRAPH_USERNAME` > `NB_GRAPH_PASSWORD` -> `NB_GRAPH_DB` +> `NB_GRAPH_DB` > `NB_RETURN_AGG` > `NB_API_QUERY_URL` @@ -114,10 +114,13 @@ openssl rand -hex 16 This file is only used by deployment profiles that include the federation API. `local_nb_nodes.json` contains the URLs and (arbitrary) names of the local nodes you wish to federate over. Each node must be denoted by a dictionary `{}` with two key-value pairs: -`"NodeName"` for the name of the node, -`"ApiURL"` for the URL of the API exposed for that node. -Multiple nodes must be wrapped in a list `[]`. +- `"NodeName"`: name of the node +- `"ApiURL"`: URL of the node API exposed for that node + +Multiple nodes must be wrapped in a list `[]`. + +Example: Let's assume there are two local nodes already running on different servers of your institutional network, and you want to set up federation across both nodes: - a node named `"My Institute"` running on your local computer (localhost), on port `8000` and @@ -158,7 +161,7 @@ To add one or more local nodes to the list of nodes known to your `f-API`, simpl ## Manually setting up a Neurobagel graph backend -The Neurobagel docker compose recipe will automatically setup and configure +The Neurobagel Docker Compose recipe will automatically set up and configure all services for you after deployment. The automated setup steps are explained in more detail below. diff --git a/docs/getting_started.md b/docs/getting_started.md index 8b5d1f7c..3f22300d 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -18,7 +18,7 @@ and are launched with Docker Compose. !!! danger "Don't install Neurobagel tools directly on your machine" - Please only use the docker images provided by Neurobagel + Please only use the Docker images provided by Neurobagel (or the third party providers Neurobagel relies on) and only launch them with our provided `docker-compose.yml` recipe. From 02e04c77166bcb72b44bf48e58dac6d6c2122169 Mon Sep 17 00:00:00 2001 From: Alyssa Dai Date: Thu, 8 Aug 2024 17:23:03 -0400 Subject: [PATCH 3/5] fix required Docker Compose version - older versions don't support environment variable secrets --- docs/getting_started.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/getting_started.md b/docs/getting_started.md index 3f22300d..3a66d8d1 100644 --- a/docs/getting_started.md +++ b/docs/getting_started.md @@ -67,7 +67,7 @@ versions on your machine: ```bash docker --version ``` -- `docker compose`: [v2.0.0](https://github.com/docker/compose/releases/tag/v2.0.0) or above +- `docker compose`: [v2.7.0](https://github.com/docker/compose/releases/tag/v2.7.0) or above ```bash docker compose version ``` From c6f5f652ee5c9eee45fcce4b80303f0a22b77f2c Mon Sep 17 00:00:00 2001 From: Alyssa Dai Date: Fri, 9 Aug 2024 13:22:28 -0400 Subject: [PATCH 4/5] Remove extra space Co-authored-by: Sebastian Urchs --- docs/config.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/config.md b/docs/config.md index a13e8dc8..6ca586cc 100644 --- a/docs/config.md +++ b/docs/config.md @@ -89,7 +89,7 @@ At minimum, we recommend reviewing and changing the values of the following vari > `NB_GRAPH_ADMIN_PASSWORD` > `NB_GRAPH_USERNAME` > `NB_GRAPH_PASSWORD` -> `NB_GRAPH_DB` +> `NB_GRAPH_DB` > `NB_RETURN_AGG` > `NB_API_QUERY_URL` From c92a4ab3819fdd00825c63c84cce06d694253ad2 Mon Sep 17 00:00:00 2001 From: Alyssa Dai Date: Fri, 9 Aug 2024 13:23:05 -0400 Subject: [PATCH 5/5] make it clear that local_nb_nodes.json contains n-API info --- docs/config.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/config.md b/docs/config.md index a13e8dc8..ffb38150 100644 --- a/docs/config.md +++ b/docs/config.md @@ -112,7 +112,7 @@ openssl rand -hex 16 ## `local_nb_nodes.json` This file is only used by deployment profiles that include the federation API. -`local_nb_nodes.json` contains the URLs and (arbitrary) names of the local nodes you wish to federate over. +`local_nb_nodes.json` contains the URLs and (arbitrary) names of the _node APIs_ of local nodes you wish to federate over. Each node must be denoted by a dictionary `{}` with two key-value pairs: - `"NodeName"`: name of the node