Skip to content

Latest commit

 

History

History
113 lines (87 loc) · 5.53 KB

README.md

File metadata and controls

113 lines (87 loc) · 5.53 KB
Raw

Caution

The docs folder has been moved to the centralized documentation repository, docs-internal. Please update this documentation in that repository instead.

To view the ScalarDL documentation, visit ScalarDL Documentation.

ScalarDL Benchmarks

This repository contains benchmark programs for ScalarDL.

Available workloads

  • SmallBank
  • TPC-C (New-Order and Payment transactions only)
  • YCSB (Workload A, C and F)

Prerequisites

  • Java (OpenJDK 8 or higher)
  • Gradle
  • Kelpie

The benchmark uses Kelpie, which is a simple yet general framework for performing end-to-end testing such as system benchmarking and verification. Get the latest version of Kelpie from here and unzip the archive.

Usage

Set up an environment

This benchmark requires the followings:

  • A client to execute this benchmark
  • A target Ledger server
  • A target Auditor server (optional)

Set up the above components and then properly configure Client, Ledger and Auditor properties by following the getting-started guides. Note that you do not need to download Client SDK and manually register your certificate. As described later, this benchmark suite will automatically register the required certificate and contracts.

Build

./gradlew shadowJar

Load and run

  1. Prepare a configuration file
    • A configuration file requires at least the locations of workload modules to run and the client configuration. The following example shows the case for running TPC-C benchmark. The client configuration should be matched with the benchmark environment set up above. You can use the client.properties file instead of specifying each configuration item. If the config_file is specified, all other configuration items will be ignored. 
      [modules]
[modules.preprocessor]
name = "com.scalar.dl.benchmarks.tpcc.TpccLoader"
path = "./build/libs/scalardl-benchmarks-all.jar"
[modules.processor]
name = "com.scalar.dl.benchmarks.tpcc.TpccBench"
path = "./build/libs/scalardl-benchmarks-all.jar"
[modules.postprocessor]
name = "com.scalar.dl.benchmarks.tpcc.TpccReporter"
path = "./build/libs/scalardl-benchmarks-all.jar"

[client_config]
ledger_host = "localhost"
auditor_host = "localhost"
auditor_enabled = "true"
cert_holder_id = "test_holder"
certificate = "/path/to/client.pem"
private_key = "/path/to/client-key.pem"
#config_file = "/path/to/client.properties"
    • You can define static parameters to pass to modules in the file. For details, see example configuration files such as tpcc-benchmark-config.toml and available parameters in the following section.
  2. Run a benchmark 
    ${kelpie}/bin/kelpie --config your_config.toml
    • ${kelpie} is a Kelpie directory, which is extracted from the archive you downloaded above.
    • There are other options such as --only-pre (i.e., registering certificates and contracts and loading data) and --only-process (i.e., running benchmark), which run only the specified process. --except-pre and --except-process run a job without the specified process.

Common parameters

name description default
concurrency Number of threads for benchmarking. 1
run_for_sec Duration of benchmark (in seconds). 60
ramp_for_sec Duration of ramp up time before benchmark (in seconds). 0

Workload-specific parameters

SmallBank

name description default
num_accounts Number of bank accounts for benchmarking. 100000
load_concurrency Number of threads for loading. 1
load_batch_size Number of accounts in a single loading transaction. 1

TPC-C

name description default
num_warehouses Number of warehouses (scale factor) for benchmarking. 1
rate_payment Percentage of payment transaction. 50
load_concurrency Number of threads for loading. 1

YCSB

name description default
record_count Number of records for benchmarking. 1000
payload_size Payload size (in bytes) of each record. 1000
ops_per_tx Number of operations in a single transaction 2
workload Workload type (A, C or F). A
load_concurrency Number of threads for loading. 1
load_batch_size Number of records in a single loading transaction. 1