Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Release/v0.37.1 #389

Open
wants to merge 11 commits into
base: master
Choose a base branch
from
Open

Release/v0.37.1 #389

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
dee55b0
transactions/all: validation allows txs 18
dvshur Mar 28, 2023
866a690
ethereum transactions basic support
dvshur Apr 4, 2023
4b0c51d
package-lock.json reverted
dvshur Apr 4, 2023
7ef6524
Update openapi.json
pavel-prokhorov Apr 6, 2023
c45ca6e
Update openapi.json
pavel-prokhorov Apr 6, 2023
d710d89
0.37.0
dvshur Apr 11, 2023
85449e1
eth txs: rename db column payload to bytes
dvshur Apr 18, 2023
f4edaa2
pairs daemon: use decimals view instead of assets
dvshur Apr 20, 2023
d44b4c1
removed candles daemon
dvshur Apr 20, 2023
6bfb99a
allow empty ticker
pavel-prokhorov Aug 11, 2023
753489c
filter out empty string tickers
pavel-prokhorov Aug 11, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
72 changes: 39 additions & 33 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,15 +2,15 @@

**⚠️ This service is currently in /v0. Breaking changes are coming in /v1 (also possible, but not likely, within /v0 releases). Please use with caution.**

This is an API aimed at retrieving data from blockchain quickly and conveniently. We support public APIs for:
This is an API aimed at retrieving data from blockchain quickly and conveniently. We support public APIs for:

- Mainnet
- [https://api.wavesplatform.com/v0/](https://api.wavesplatform.com/v0/)
- Testnet
- [https://api.testnet.wavesplatform.com/v0/](https://api.testnet.wavesplatform.com/v0/)

Visit `/docs` for Swagger documentation.


## Data service on-premise

It is possible to create your own instance of this service. To do so, follow the guide below.
Expand All @@ -19,26 +19,26 @@ It is possible to create your own instance of this service. To do so, follow the

1. PostgreSQL 11 database with a table stricture found in [wavesplatform/blockchain-postgres-sync](https://github.com/wavesplatform/blockchain-postgres-sync)
2. Downloaded and continuously updated blockchain data in the database
2. NodeJS or Docker for either running the service directly, or in a container
3. NodeJS or Docker for either running the service directly, or in a container

#### Installation and start

The service uses following environment variables:

|Env variable|Default|Required|Description|
|------------|-------|--------|-----------|
|`PORT`|3000|NO|HTTP service port|
|`PGHOST`||YES|Postgres host address|
|`PGPORT`|`5432`|NO|Postgres port|
|`PGDATABASE`||YES|Postgres database name|
|`PGUSER`||YES|Postgres user name|
|`PGPASSWORD`||YES|Postgres password|
|`PGPOOLSIZE`|`20`|NO|Postgres pool size|
|`PGSTATEMENTTIMEOUT`|false|NO|Postgres `statement_timeout` number in ms. 0 disables timeout, false — use server settings; at this moment used only as default `STATEMENT_TIMEOUT`|
|`LOG_LEVEL`|`info`|NO|Log level `['info','warn','error']`|
|`DEFAULT_MATCHER`||YES|Default matcher public address|
|`MATCHER_SETTINGS_URL`||NO|Default matcher URL for getting settings|
|`DEFAULT_TIMEOUT`|30000|NO|Default timeout in ms; at this moment used only as `PG STATEMENT_TIMEOUT`|
| Env variable | Default | Required | Description |
| ---------------------- | ------- | -------- | --------------------------------------------------------------------------------------------------------------------------------------------------- |
| `PORT` | 3000 | NO | HTTP service port |
| `PGHOST` | | YES | Postgres host address |
| `PGPORT` | `5432` | NO | Postgres port |
| `PGDATABASE` | | YES | Postgres database name |
| `PGUSER` | | YES | Postgres user name |
| `PGPASSWORD` | | YES | Postgres password |
| `PGPOOLSIZE` | `20` | NO | Postgres pool size |
| `PGSTATEMENTTIMEOUT` | false | NO | Postgres `statement_timeout` number in ms. 0 disables timeout, false — use server settings; at this moment used only as default `STATEMENT_TIMEOUT` |
| `LOG_LEVEL` | `info` | NO | Log level `['info','warn','error']` |
| `DEFAULT_MATCHER` | | YES | Default matcher public address |
| `MATCHER_SETTINGS_URL` | | NO | Default matcher URL for getting settings |
| `DEFAULT_TIMEOUT` | 30000 | NO | Default timeout in ms; at this moment used only as `PG STATEMENT_TIMEOUT` |

`PGPOOLSIZE` is used by the `pg-pool` library to determine Postgres connection pool size per NodeJS process instance. A good value depends on your server and db configuration and can be found empirically. You can leave it at the default value to start with.

Expand All @@ -47,23 +47,27 @@ Set those variables to a `variables.env` file in the root of the project for con
If you would like to use some other way of setting environment variables, just replace relevant commands below with custom alternatives.

##### Docker

If you wish to build data-service image locally, run this command from the project root
```bash
docker build -t wavesplatform/data-service .
```

```bash
docker build -t wavesplatform/data-service .
```

Otherwise you can use our public image from https://hub.docker.com/r/wavesplatform/data-service

Run the container using this command:
```bash
docker run -p=<port>:3000 --env-file=variables.env wavesplatform/data-service
```

```bash
docker run -p=<port>:3000 --env-file=variables.env wavesplatform/data-service
```

A server will start at `localhost:<port>` (used in the `docker run` command). Logs will be handled by Docker. Use any other Docker options if necessary.

When using the container in production, we recommend establishing a Docker logging and restart policy.

##### NodeJS

1. Install dependencies
```bash
npm install # or `yarn install`, if you prefer
Expand All @@ -81,28 +85,30 @@ Server will start at `localhost:PORT` (defaults to 3000). Logs will be directed

If you decide to use NodeJS directly (without Docker), we recommend using a process manager, such as `pm2`.


#### Daemons
To add candles and pairs functionality the following Docker daemons must be used:
- Candles — calculate candles for exchange transactions (see [description](https://hub.docker.com/r/wavesplatform/data-service-candles/))
- Pairs — calculate last pairs for 24h exchange transactions (see [description](https://hub.docker.com/r/wavesplatform/data-service-pairs/))

To add and pairs functionality the following Docker daemons must be used:

- Pairs — calculate last pairs for 24h exchange transactions (see [description](https://hub.docker.com/r/wavesplatform/data-service-pairs/))

#### Documentation

You can run your own instance of Swagger online documentation.
To do this, you have to:

1. Build Docker image from docs/ directory:
```bash
docker build -t wavesplatform/data-service-docs docs/
```
```bash
docker build -t wavesplatform/data-service-docs docs/
```
2. Run the container
```bash
docker run --rm -d -p 8080:8080 -e SWAGGER_JSON=/app/openapi.json wavesplatform/data-service-docs
```
```bash
docker run --rm -d -p 8080:8080 -e SWAGGER_JSON=/app/openapi.json wavesplatform/data-service-docs
```

Its will start the documentation server at `localhost:8080`. Enjoy!

#### General recommendations

- Set up a dedicated web server such as Nginx in front of data-service backends (for ssl/caching/balancing);
- Implement a caching strategy. Different endpoints may need different cache time (or no cache at all);
- Run several process instances behind a load balancer per machine. `docker-compose --scale` can help with that, or it can be done manually. A good rule of thumb is to use as many instances as CPU cores available;
Expand Down
Loading