Skip to content

Commit

Permalink
TPC-DS: More tests
Browse files Browse the repository at this point in the history
  • Loading branch information
@perdelt
perdelt committed Nov 1, 2024
1 parent a3f32e5 commit a776336
Show file tree
Hide file tree
Showing 11 changed files with 7,232 additions and 4 deletions.
268 changes: 264 additions & 4 deletions docs/Example-TPC-DS.md
View file

Large diffs are not rendered by default.

897 changes: 897 additions & 0 deletions logs_tests/doc_tpcds_monetdb_1.log
View file

Large diffs are not rendered by default.

199 changes: 199 additions & 0 deletions logs_tests/doc_tpcds_monetdb_1_summary.txt
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,199 @@
## Show Summary

### Workload
TPC-DS Queries SF=100
Type: tpcds
Duration: 9730s
Code: 1730405979
This includes the reading queries of TPC-DS.
This experiment compares run time and resource consumption of TPC-DS queries in different DBMS.
TPC-DS (SF=100) data is loaded and benchmark is executed.
Query ordering is Q1 - Q99.
All instances use the same query parameters.
Timeout per query is 1200.
Import sets indexes and constraints after loading and recomputes statistics.
System metrics are monitored by a cluster-wide installation.
Benchmark is limited to DBMS ['MonetDB'].
Import is handled by 8 processes (pods).
Loading is fixed to cl-worker19.
Benchmarking is fixed to cl-worker19.
SUT is fixed to cl-worker11.
Database is persisted to disk of type shared and size 300Gi.
Loading is tested with [8] threads, split into [8] pods.
Benchmarking is tested with [1] threads, split into [1] pods.
Benchmarking is run as [1] times the number of benchmarking pods.
Experiment is run once.

### Connections
MonetDB-BHT-8-1-1 uses docker image monetdb/monetdb:Dec2023
RAM:541008605184
CPU:AMD Opteron(tm) Processor 6378
Cores:64
host:5.15.0-116-generic
node:cl-worker11
disk:254913816
datadisk:153205046
volume_size:300G
volume_used:147G
requests_cpu:4
requests_memory:16Gi

### Errors (failed queries)
No errors

### Warnings (result mismatch)
No warnings

### Latency of Timer Execution [ms]
DBMS MonetDB-BHT-8-1-1
TPC-DS Q1 8487.85
TPC-DS Q2 151923.88
TPC-DS Q3 905.48
TPC-DS Q4 254499.90
TPC-DS Q5 62567.55
TPC-DS Q6 11580.69
TPC-DS Q7 1386.00
TPC-DS Q8 4412.06
TPC-DS Q9 1997.43
TPC-DS Q10 2133.16
TPC-DS Q11 82401.02
TPC-DS Q12 898.62
TPC-DS Q13 2191.91
TPC-DS Q14a+b 324098.77
TPC-DS Q15 1846.92
TPC-DS Q16 7825.79
TPC-DS Q17 54326.28
TPC-DS Q18 9313.78
TPC-DS Q19 3102.17
TPC-DS Q20 668.31
TPC-DS Q21 924.07
TPC-DS Q22 67826.67
TPC-DS Q23a+b 3093870.15
TPC-DS Q24a+b 10940.12
TPC-DS Q25 17420.76
TPC-DS Q26 973.29
TPC-DS Q27 13031.49
TPC-DS Q28 7427.71
TPC-DS Q29 14588.75
TPC-DS Q30 792.23
TPC-DS Q31 19072.92
TPC-DS Q32 641.79
TPC-DS Q33 6272.47
TPC-DS Q34 1787.43
TPC-DS Q35 7229.82
TPC-DS Q37 7075.80
TPC-DS Q38 27739.19
TPC-DS Q39a+b 72942.27
TPC-DS Q40 3317.84
TPC-DS Q41 176.37
TPC-DS Q42 2202.22
TPC-DS Q43 667.64
TPC-DS Q44 1391.02
TPC-DS Q45 737.27
TPC-DS Q46 1637.75
TPC-DS Q47 6753.22
TPC-DS Q48 2715.50
TPC-DS Q49 10936.67
TPC-DS Q50 1829.45
TPC-DS Q51 46890.66
TPC-DS Q52 2276.80
TPC-DS Q53 2243.95
TPC-DS Q54 4364.62
TPC-DS Q55 377.10
TPC-DS Q56 1807.07
TPC-DS Q57 1117.86
TPC-DS Q58 4742.87
TPC-DS Q59 11487.65
TPC-DS Q60 1415.10
TPC-DS Q61 717.52
TPC-DS Q62 755.90
TPC-DS Q63 955.24
TPC-DS Q64 46849.52
TPC-DS Q65 23937.75
TPC-DS Q66 4750.16
TPC-DS Q67 105741.06
TPC-DS Q68 5980.62
TPC-DS Q69 2460.54
TPC-DS Q71 1322.11
TPC-DS Q72 25515.56
TPC-DS Q73 325.28
TPC-DS Q74 24674.21
TPC-DS Q75 123299.04
TPC-DS Q76 14956.14
TPC-DS Q77 7376.96
TPC-DS Q78 176943.68
TPC-DS Q79 5530.19
TPC-DS Q80 124497.51
TPC-DS Q81 784.88
TPC-DS Q82 57358.76
TPC-DS Q83 952.67
TPC-DS Q84 293.56
TPC-DS Q85 1128.16
TPC-DS Q87 53821.10
TPC-DS Q88 5310.08
TPC-DS Q89 1958.66
TPC-DS Q90 308.85
TPC-DS Q91 80.71
TPC-DS Q92 1433.28
TPC-DS Q93 18714.89
TPC-DS Q94 1680.08
TPC-DS Q95 18150.04
TPC-DS Q96 356.39
TPC-DS Q97 41533.27
TPC-DS Q98 2601.72
TPC-DS Q99 1411.17

### Loading [s]
timeGenerate timeIngesting timeSchema timeIndex timeLoad
MonetDB-BHT-8-1-1 0.0 1435.0 7.0 2616.0 4066.0

### Geometric Mean of Medians of Timer Run [s]
Geo Times [s]
DBMS
MonetDB-BHT-8-1-1 5.36

### Power@Size
Power@Size [~Q/h]
DBMS
MonetDB-BHT-8-1-1 68546.5

### Throughput@Size
time [s] count SF Throughput@Size [~GB/h]
DBMS SF num_experiment num_client
MonetDB-BHT-8-1 100 1 1 5413 1 100 1463.14

### Workflow

#### Actual
DBMS MonetDB-BHT-8 - Pods [[1]]

#### Planned
DBMS MonetDB-BHT-8 - Pods [[1]]

### Ingestion - SUT
CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb]
MonetDB-BHT-8-1 20953.14 16.82 22.09 126.44

### Ingestion - Loader
CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb]
MonetDB-BHT-8-1 2329.5 1.03 41.0 84.3

### Execution - SUT
CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb]
MonetDB-BHT-8-1 20866.68 24.86 148.0 315.74

### Execution - Benchmarker
CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb]
MonetDB-BHT-8-1 50.35 0.09 0.36 0.37

### Tests
TEST passed: Geo Times [s] contains no 0 or NaN
TEST passed: Power@Size [~Q/h] contains no 0 or NaN
TEST passed: Throughput@Size [~GB/h] contains no 0 or NaN
TEST passed: No SQL errors
TEST passed: No SQL warnings
TEST passed: Ingestion SUT contains no 0 or NaN in CPU [CPUs]
TEST passed: Ingestion Loader contains no 0 or NaN in CPU [CPUs]
TEST passed: Execution SUT contains no 0 or NaN in CPU [CPUs]
TEST passed: Execution Benchmarker contains no 0 or NaN in CPU [CPUs]
TEST passed: Workflow as planned
Loading

0 comments on commit a776336

Please sign in to comment.