Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support more storages #77

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open

Support more storages #77

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
6303bee
Support more storages
jnmt Dec 17, 2024
816a1b4
Use include function
jnmt Dec 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .env
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
SCALARDL_VERSION=3.10.0
9 changes: 4 additions & 5 deletions docker-compose-auditor.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,6 @@
version: "3.5"
services:
scalardl-auditor-schema-loader-cassandra:
image: ghcr.io/scalar-labs/scalardl-schema-loader:3.10.0
image: ghcr.io/scalar-labs/scalardl-schema-loader:${SCALARDL_VERSION}
environment:
- SCHEMA_TYPE=auditor
volumes:
Expand All @@ -20,7 +19,7 @@ services:
restart: on-failure

scalar-ledger-as-client:
image: ghcr.io/scalar-labs/scalar-client:3.10.0
image: ghcr.io/scalar-labs/scalar-client:${SCALARDL_VERSION}
container_name: "scalardl-samples-scalar-ledger-as-client-1"
volumes:
- ./fixture/ledger.pem:/scalar/ledger.pem
Expand All @@ -46,7 +45,7 @@ services:
restart: on-failure:5

scalar-audior-as-client:
image: ghcr.io/scalar-labs/scalar-client:3.10.0
image: ghcr.io/scalar-labs/scalar-client:${SCALARDL_VERSION}
container_name: "scalardl-samples-scalar-auditor-as-client-1"
volumes:
- ./fixture/auditor.pem:/scalar/auditor.pem
Expand Down Expand Up @@ -76,7 +75,7 @@ services:
- SCALAR_DL_LEDGER_AUDITOR_ENABLED=true

scalar-auditor:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can add similar configurations for the Auditor mode, but I think preparing all of them might be too much for samples. Let's discuss it if you have opinions.

I considered some efficient ways and tried to implement some of them in my local environment. However, I was not able to good way... (I faced a bit difficult problem with the include configuration...) We might need to implement all docker-compose-auditor-xxxx.yml files one by one...

And, I agree with you. Preparing all files might be too much for testing purposes. So, at the moment, I think we don't need to provide Auditor things for all databases. At least, users can deploy Auditor with Cassandra by using the previous way.

However, sometimes, users (especially, application developers) want to test ScalarDL Auditor on their local environment for testing and developing their applications. So, We might need to continue to consider this challenge.

Anyway, I don't think we need Auditor configurations for all storage at the moment. Let's discuss this later when we have a chance!

image: ghcr.io/scalar-labs/scalardl-auditor-byol:3.10.0
image: ghcr.io/scalar-labs/scalardl-auditor-byol:${SCALARDL_VERSION}
container_name: "scalardl-samples-scalar-auditor-1"
volumes:
- ./fixture/auditor.pem:/scalar/auditor.pem
Expand Down
66 changes: 66 additions & 0 deletions docker-compose-ledger-cassandra.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
include:
- ./docker-compose-ledger-common.yml

services:
cassandra:
image: cassandra:3.11
container_name: "scalardl-samples-cassandra-1"
environment:
- CASSANDRA_DC=dc1
- CASSANDRA_ENDPOINT_SNITCH=GossipingPropertyFileSnitch
networks:
- scalar-network
healthcheck:
test: ["CMD-SHELL", "cqlsh < /dev/null || exit 1"]
interval: 1s
timeout: 10s
retries: 60
start_period: 30s

scalardl-ledger-schema-loader:
image: ghcr.io/scalar-labs/scalardl-schema-loader:${SCALARDL_VERSION}
volumes:
- ./scalardb.cassandra.properties:/scalardb.properties
depends_on:
cassandra:
condition: service_healthy
command:
- "-c"
- "/scalardb.properties"
- "--coordinator"
networks:
- scalar-network
restart: on-failure

scalar-ledger:
image: ghcr.io/scalar-labs/scalardl-ledger-byol:${SCALARDL_VERSION}
container_name: "scalardl-samples-scalar-ledger-1"
volumes:
- ./fixture/ledger-key.pem:/scalar/ledger-key.pem
- ./fixture/ledger.properties.tmpl:/scalar/ledger/ledger.properties.tmpl
- ./fixture/trial-license-cert.pem:/scalar/license-cert.pem
# If you have a commercial license key, you must use `commercial-license-cert.pem` instead of `trial-license-cert.pem`.
# - ./fixture/commercial-license-cert.pem:/scalar/license-cert.pem
depends_on:
scalardl-ledger-schema-loader:
condition: service_completed_successfully
environment:
- SCALAR_DB_CONTACT_POINTS=cassandra
- SCALAR_DB_STORAGE=cassandra
- SCALAR_DB_USERNAME=cassandra
- SCALAR_DB_PASSWORD=cassandra
- SCALAR_DL_LEDGER_PROOF_ENABLED=true
- SCALAR_DL_LEDGER_PROOF_PRIVATE_KEY_PATH=/scalar/ledger-key.pem

##### PLEASE REPLACE THIS VALUE WITH YOUR LICENSE KEY #####
- SCALAR_DL_LICENSING_LICENSE_KEY=<SET_YOUR_LICENSE_KEY>
##### PLEASE REPLACE THIS VALUE WITH YOUR LICENSE KEY #####

networks:
- scalar-network
healthcheck:
test: ["CMD-SHELL", "grpc_health_probe -addr=localhost:50051 && grpc_health_probe -addr=ledger-envoy:50051 || exit 1"]
interval: 1s
timeout: 10s
retries: 60
start_period: 10s
62 changes: 62 additions & 0 deletions docker-compose-ledger-common.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,62 @@
services:
ledger-envoy:
image: ghcr.io/scalar-labs/scalar-envoy:1.5.0
Copy link
Contributor

@kota2and3kan kota2and3kan Dec 26, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
image: ghcr.io/scalar-labs/scalar-envoy:1.5.0
image: ghcr.io/scalar-labs/scalar-envoy:1.6.1

I think it would be better to use the latest version of Scalar Envoy. For example, the latest helm chart of ScalarDL 3.10 uses Scalar Envoy 1.6.1 in its internal.
https://github.com/scalar-labs/helm-charts/blob/scalardl-4.9.1/charts/scalardl/values.yaml#L40

container_name: "scalardl-samples-ledger-envoy-1"
ports:
- "9901:9901"
- "50051:50051"
- "50052:50052"
depends_on:
scalar-ledger:
condition: service_started
environment:
- admin_access_log_path=/dev/stdout
- scalardl_address=scalar-ledger
- service_listeners=scalar-service:50051,scalar-privileged:50052
- envoy_tls=false
networks:
- scalar-network

cfssl-init:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is not directly related to this PR, but I have one question.

Is CFSSL necessary for running scalardl-samples? I think if users want to generate a private key and certificate by using this CFSSL server, I think we need to deploy it. However, now, we are using the private key and certificate under the fixture/ directory. So, I want to confirm whether this CFSSL server is necessary or not.

If it's not mandatory, I think we can make it an optional component. For example, we can set the profiles configuration as follows:

$ git diff docker-compose-ledger-common.yml
diff --git a/docker-compose-ledger-common.yml b/docker-compose-ledger-common.yml
index 6a4481c..982dc41 100644
--- a/docker-compose-ledger-common.yml
+++ b/docker-compose-ledger-common.yml
@@ -23,6 +23,8 @@ services:
     # they start up at the same time (by commands like `docker-compose up`),
     # they both attempt to generate the same missing certificate files, which
     # results in a race condition.
+    profiles:
+      - cfssl
     image: ghcr.io/scalar-labs/scalar-cfssl:1.0.0
     container_name: "scalardl-samples-cfssl-init-1"
     volumes:
@@ -32,6 +34,8 @@ services:
       - scalar-network

   cfssl-serve:
+    profiles:
+      - cfssl
     image: ghcr.io/scalar-labs/scalar-cfssl:1.0.0
     container_name: "scalardl-samples-cfssl-serve-1"
     volumes:
@@ -45,6 +49,8 @@ services:
       - scalar-network

   cfssl-ocspserve:
+    profiles:
+      - cfssl
     image: ghcr.io/scalar-labs/scalar-cfssl:1.0.0
     container_name: "scalardl-samples-cfssl-ocspserve-1"
     volumes:

In this case, the CFSSL server is not deployed by default as follows:

  • Default behavior 
    $ docker compose -f docker-compose-ledger-postgres.yml up -d

    $ docker compose -f docker-compose-ledger-postgres.yml ps
NAME                               IMAGE                                             COMMAND                  SERVICE         CREATED         STATUS                        PORTS
postgres-1                         postgres:15                                       "docker-entrypoint.s…"   postgres        2 minutes ago   Up About a minute (healthy)   0.0.0.0:5432->5432/tcp, :::5432->5432/tcp
scalardl-samples-ledger-envoy-1    ghcr.io/scalar-labs/scalar-envoy:1.5.0            "/entrypoint.sh"         ledger-envoy    8 seconds ago   Up 4 seconds                  9001/tcp, 0.0.0.0:9901->9901/tcp, :::9901->9901/tcp, 10000/tcp, 0.0.0.0:50051-50052->50051-50052/tcp, :::50051-50052->50051-50052/tcp
scalardl-samples-scalar-ledger-1   ghcr.io/scalar-labs/scalardl-ledger-byol:3.10.0   "./docker-entrypoint…"   scalar-ledger   8 seconds ago   Up 6 seconds (healthy)        50051/tcp

Also, if you want to deploy the CFSSL server, you can specify it by using the --profile option as follows:

  • Specify the profile explicitly 
    $ docker compose -f docker-compose-ledger-postgres.yml --profile cfssl up -d

    $ docker compose -f docker-compose-ledger-postgres.yml ps
NAME                                 IMAGE                                             COMMAND                  SERVICE           CREATED              STATUS                        PORTS
postgres-1                           postgres:15                                       "docker-entrypoint.s…"   postgres          3 minutes ago        Up 3 minutes (healthy)        0.0.0.0:5432->5432/tcp, :::5432->5432/tcp
scalardl-samples-cfssl-ocspserve-1   ghcr.io/scalar-labs/scalar-cfssl:1.0.0            "docker-entrypoint.s…"   cfssl-ocspserve   10 seconds ago       Up 8 seconds                  8888/tcp, 0.0.0.0:8889->8889/tcp, :::8889->8889/tcp
scalardl-samples-cfssl-serve-1       ghcr.io/scalar-labs/scalar-cfssl:1.0.0            "docker-entrypoint.s…"   cfssl-serve       10 seconds ago       Up 8 seconds                  0.0.0.0:8888->8888/tcp, :::8888->8888/tcp, 8889/tcp
scalardl-samples-ledger-envoy-1      ghcr.io/scalar-labs/scalar-envoy:1.5.0            "/entrypoint.sh"         ledger-envoy      About a minute ago   Up About a minute             9001/tcp, 0.0.0.0:9901->9901/tcp, :::9901->9901/tcp, 10000/tcp, 0.0.0.0:50051-50052->50051-50052/tcp, :::50051-50052->50051-50052/tcp
scalardl-samples-scalar-ledger-1     ghcr.io/scalar-labs/scalardl-ledger-byol:3.10.0   "./docker-entrypoint…"   scalar-ledger     About a minute ago   Up About a minute (healthy)   50051/tcp

If it's not a mandatory component, I think we can make it an optional thing as I mentioned above. This is because:

  • We can reduce deployed components.
    • It can make a system simple. Especially, for the local testing purpose of application developers, I think fewer components and resources are a better choice.
  • We can reduce unnecessary confusion and questions from actual users.
    • If there are some components that users cannot understand easily, they might confused and ask the support team What are these components? or Do we need CFSSL in the production environment? etc... To avoid such things, I think it would be better to keep the system (test environment) simple and minimal by default.

What do you think?
This comment does not prevent merging this PR, but I want to discuss it with you later!

# This service should be run separately before two services below
# (cfssl-serve and cfssl-ocspserve) with `depends_on` option, because if
# they start up at the same time (by commands like `docker-compose up`),
# they both attempt to generate the same missing certificate files, which
# results in a race condition.
image: ghcr.io/scalar-labs/scalar-cfssl:1.0.0
container_name: "scalardl-samples-cfssl-init-1"
volumes:
- ./cfssl/data:/cfssl/data
command: /bin/true
networks:
- scalar-network

cfssl-serve:
image: ghcr.io/scalar-labs/scalar-cfssl:1.0.0
container_name: "scalardl-samples-cfssl-serve-1"
volumes:
- ./cfssl/data:/cfssl/data
depends_on:
- cfssl-init
command: serve
ports:
- "8888:8888"
networks:
- scalar-network

cfssl-ocspserve:
image: ghcr.io/scalar-labs/scalar-cfssl:1.0.0
container_name: "scalardl-samples-cfssl-ocspserve-1"
volumes:
- ./cfssl/data:/cfssl/data
depends_on:
- cfssl-init
command: ocspserve
ports:
- "8889:8889"
networks:
- scalar-network

networks:
scalar-network:
name: scalar-network
47 changes: 47 additions & 0 deletions docker-compose-ledger-cosmosdb.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
include:
- ./docker-compose-ledger-common.yml

services:
scalardl-ledger-schema-loader:
image: ghcr.io/scalar-labs/scalardl-schema-loader:${SCALARDL_VERSION}
volumes:
- ./scalardb.cosmosdb.properties:/scalardb.properties
command:
- "-c"
- "/scalardb.properties"
- "--coordinator"
networks:
- scalar-network
restart: on-failure

scalar-ledger:
image: ghcr.io/scalar-labs/scalardl-ledger-byol:${SCALARDL_VERSION}
container_name: "scalardl-samples-scalar-ledger-1"
volumes:
- ./fixture/ledger-key.pem:/scalar/ledger-key.pem
- ./fixture/ledger.properties.tmpl:/scalar/ledger/ledger.properties.tmpl
- ./fixture/trial-license-cert.pem:/scalar/license-cert.pem
# If you have a commercial license key, you must use `commercial-license-cert.pem` instead of `trial-license-cert.pem`.
# - ./fixture/commercial-license-cert.pem:/scalar/license-cert.pem
depends_on:
scalardl-ledger-schema-loader:
condition: service_completed_successfully
environment:
- SCALAR_DB_CONTACT_POINTS=<COSMOS_DB_FOR_NOSQL_URI>
- SCALAR_DB_STORAGE=cosmos
- SCALAR_DB_PASSWORD=<COSMOS_DB_FOR_NOSQL_KEY>
- SCALAR_DL_LEDGER_PROOF_ENABLED=true
- SCALAR_DL_LEDGER_PROOF_PRIVATE_KEY_PATH=/scalar/ledger-key.pem

##### PLEASE REPLACE THIS VALUE WITH YOUR LICENSE KEY #####
- SCALAR_DL_LICENSING_LICENSE_KEY=<SET_YOUR_LICENSE_KEY>
##### PLEASE REPLACE THIS VALUE WITH YOUR LICENSE KEY #####

networks:
- scalar-network
healthcheck:
test: ["CMD-SHELL", "grpc_health_probe -addr=localhost:50051 && grpc_health_probe -addr=ledger-envoy:50051 || exit 1"]
interval: 1s
timeout: 10s
retries: 60
start_period: 10s
68 changes: 68 additions & 0 deletions docker-compose-ledger-dynamodb.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,68 @@
include:
- ./docker-compose-ledger-common.yml

services:
dynamodb:
image: amazon/dynamodb-local:2.5.2
container_name: "dynamodb-1"
ports:
- "8000:8000"
networks:
- scalar-network
healthcheck:
test: ["CMD-SHELL", '[ "$(curl -s -o /dev/null -I -w ''%{http_code}'' http://localhost:8000)" == "400" ] || exit 1']
interval: 1s
timeout: 10s
retries: 60
start_period: 30s

scalardl-ledger-schema-loader:
image: ghcr.io/scalar-labs/scalardl-schema-loader:${SCALARDL_VERSION}
volumes:
- ./scalardb.dynamodb.properties:/scalardb.properties
depends_on:
dynamodb:
condition: service_healthy
command:
- "-c"
- "/scalardb.properties"
- "--coordinator"
- "--no-backup"
- "--no-scaling"
networks:
- scalar-network
restart: on-failure

scalar-ledger:
image: ghcr.io/scalar-labs/scalardl-ledger-byol:${SCALARDL_VERSION}
container_name: "scalardl-samples-scalar-ledger-1"
volumes:
- ./fixture/ledger-key.pem:/scalar/ledger-key.pem
- ./fixture/ledger.properties.tmpl:/scalar/ledger/ledger.properties.tmpl
- ./fixture/trial-license-cert.pem:/scalar/license-cert.pem
# If you have a commercial license key, you must use `commercial-license-cert.pem` instead of `trial-license-cert.pem`.
# - ./fixture/commercial-license-cert.pem:/scalar/license-cert.pem
depends_on:
scalardl-ledger-schema-loader:
condition: service_completed_successfully
environment:
- SCALAR_DB_CONTACT_POINTS=sample
- SCALAR_DB_STORAGE=dynamo
- SCALAR_DB_USERNAME=sample
- SCALAR_DB_PASSWORD=sample
- SCALAR_DB_DYNAMO_ENDPOINT_OVERRIDE=http://dynamodb:8000
- SCALAR_DL_LEDGER_PROOF_ENABLED=true
- SCALAR_DL_LEDGER_PROOF_PRIVATE_KEY_PATH=/scalar/ledger-key.pem

##### PLEASE REPLACE THIS VALUE WITH YOUR LICENSE KEY #####
- SCALAR_DL_LICENSING_LICENSE_KEY=<SET_YOUR_LICENSE_KEY>
##### PLEASE REPLACE THIS VALUE WITH YOUR LICENSE KEY #####

networks:
- scalar-network
healthcheck:
test: ["CMD-SHELL", "grpc_health_probe -addr=localhost:50051 && grpc_health_probe -addr=ledger-envoy:50051 || exit 1"]
interval: 1s
timeout: 10s
retries: 60
start_period: 10s
67 changes: 67 additions & 0 deletions docker-compose-ledger-mysql.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,67 @@
include:
- ./docker-compose-ledger-common.yml

services:
mysql:
image: mysql:8.1
container_name: "mysql-1"
environment:
MYSQL_ROOT_PASSWORD: mysql
ports:
- "3306:3306"
networks:
- scalar-network
healthcheck:
test: ["CMD-SHELL", "mysql -u root -pmysql -h 127.0.0.1 < /dev/null || exit 1"]
interval: 1s
timeout: 10s
retries: 60
start_period: 30s

scalardl-ledger-schema-loader:
image: ghcr.io/scalar-labs/scalardl-schema-loader:${SCALARDL_VERSION}
volumes:
- ./scalardb.mysql.properties:/scalardb.properties
depends_on:
mysql:
condition: service_healthy
command:
- "-c"
- "/scalardb.properties"
- "--coordinator"
networks:
- scalar-network
restart: on-failure

scalar-ledger:
image: ghcr.io/scalar-labs/scalardl-ledger-byol:${SCALARDL_VERSION}
container_name: "scalardl-samples-scalar-ledger-1"
volumes:
- ./fixture/ledger-key.pem:/scalar/ledger-key.pem
- ./fixture/ledger.properties.tmpl:/scalar/ledger/ledger.properties.tmpl
- ./fixture/trial-license-cert.pem:/scalar/license-cert.pem
# If you have a commercial license key, you must use `commercial-license-cert.pem` instead of `trial-license-cert.pem`.
# - ./fixture/commercial-license-cert.pem:/scalar/license-cert.pem
depends_on:
scalardl-ledger-schema-loader:
condition: service_completed_successfully
environment:
- SCALAR_DB_CONTACT_POINTS=jdbc:mysql://mysql/
- SCALAR_DB_STORAGE=jdbc
- SCALAR_DB_USERNAME=root
- SCALAR_DB_PASSWORD=mysql
- SCALAR_DL_LEDGER_PROOF_ENABLED=true
- SCALAR_DL_LEDGER_PROOF_PRIVATE_KEY_PATH=/scalar/ledger-key.pem

##### PLEASE REPLACE THIS VALUE WITH YOUR LICENSE KEY #####
- SCALAR_DL_LICENSING_LICENSE_KEY=<SET_YOUR_LICENSE_KEY>
##### PLEASE REPLACE THIS VALUE WITH YOUR LICENSE KEY #####

networks:
- scalar-network
healthcheck:
test: ["CMD-SHELL", "grpc_health_probe -addr=localhost:50051 && grpc_health_probe -addr=ledger-envoy:50051 || exit 1"]
interval: 1s
timeout: 10s
retries: 60
start_period: 10s
Loading