Update from neicnordic/sensitive-data-archive at 09:42 on 2023-12-08

docs/services/sda.md

@@ -5,7 +5,7 @@ Repository:
 [neicnordic/sensitive-data-archive](https://github.com/neicnordic/sensitive-data-archive)
 
 `sda` repository consists of a suite of services which are part of [NeIC Sensitive Data Archive](https://neic-sda.readthedocs.io/en/latest/) and implements the components required for data submission.
-It can be used as part of a [Federated EGA](https://ega-archive.org/federated) or as an isolated Sensitive Data Archive.
+It can be used as part of a [Federated EGA](https://ega-archive.org/federated) or as a stand-alone (isolated) Sensitive Data Archive.
 `sda` was built with support for both S3 and POSIX storage.
 
 The SDA submission pipeline has four main steps:
@@ -15,7 +15,9 @@ The SDA submission pipeline has four main steps:
 3. [Finalize](finalize.md) associates a stable accessionID with each archive file and backups the file.
 4. [Mapper](mapper.md) maps file accessionIDs to a datasetID.
 
-There are also three additional support services:
+There are also additional support services:
 
 1. [Intercept](intercept.md) relays messages from `CentralEGA` to the system.
 2. [s3inbox](s3inbox.md) proxies uploads to the an S3 compatible storage backend.
+3. [sync]() project.
+4. [syncapi]() project for mirroring data between two installations of SDA.

docs/services/syncapi.md

@@ -0,0 +1,69 @@
+# sync-api
+
+The sync-api service is used in the [Bigpicture](https://bigpicture.eu/) project.
+
+## Service Description
+
+The sync service facilitates replication of data and metadata between the nodes in the consortium.
+
+When enabled the service will perform the following tasks:
+
+1. Upon recieving a POST request with JSON data to the `/dataset` route.
+   1. Parse the JSON blob and validate it against the `file-sync` schema.
+   2. Build and send messages to start ingestion of files.
+   3. Build and send messages to assign stableIDs to files.
+   4. Build and send messages to map files to a dataset.
+
+## Configuration
+
+There are a number of options that can be set for the sync service.
+These settings can be set by mounting a yaml-file at `/config.yaml` with settings.
+
+ex.
+
+```yaml
+log:
+  level: "debug"
+  format: "json"
+```
+
+They may also be set using environment variables like:
+
+```bash
+export LOG_LEVEL="debug"
+export LOG_FORMAT="json"
+```
+
+### Service settings
+
+- `SYNC_API_PASSWORD`: password for the API user
+- `SYNC_API_USER`: User that will be allowed to send POST requests to the API
+
+### RabbitMQ broker settings
+
+These settings control how sync connects to the RabbitMQ message broker.
+
+- `BROKER_HOST`: hostname of the rabbitmq server
+- `BROKER_PORT`: rabbitmq broker port (commonly `5671` with TLS and `5672` without)
+- `BROKER_EXCHANGE`: exchange to send messages to
+- `BROKER_USER`: username to connect to rabbitmq
+- `BROKER_PASSWORD`: password to connect to rabbitmq
+- `BROKER_PREFETCHCOUNT`: Number of messages to pull from the message server at the time (default to 2)
+
+The default routing keys for sending ingestion, accession and maping messages can be overridden by setting the following values:
+
+- `SYNC_API_ACCESSIONROUTING`
+- `SYNC_API_INGESTROUTING`
+- `SYNC_API_MAPPINGROUTING`
+
+### Logging settings
+
+- `LOG_FORMAT` can be set to “json” to get logs in json format. All other values result in text logging
+- `LOG_LEVEL` can be set to one of the following, in increasing order of severity:
+    - `trace`
+    - `debug`
+    - `info`
+    - `warn` (or `warning`)
+    - `error`
+    - `fatal`
+    - `panic`