diff --git a/docs/Example-CloudDatabase.md b/docs/Example-CloudDatabase.md index ef67d785..b49dc0b4 100644 --- a/docs/Example-CloudDatabase.md +++ b/docs/Example-CloudDatabase.md @@ -180,13 +180,14 @@ If something goes wrong, you can also clean up manually with `bexperiment stop` At the end of a benchmark you will see a summary like ```bash + ## Show Summary ### Workload YCSB SF=1 Type: ycsb Duration: 354s - Code: 1734449423 + Code: 1734663459 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 1000000. Batch size is ''. @@ -208,18 +209,18 @@ DatabaseService-64-8-65536-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236006380 + disk:249256012 datadisk:39348 requests_cpu:4 requests_memory:16Gi ### Loading experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [INSERT].Return=OK [INSERT].99thPercentileLatency(us) -DatabaseService-64-8-65536 1 64 65536 8 50279.137219 20168.0 1000000 25355.0 +DatabaseService-64-8-65536 1 64 65536 8 49973.150502 20251.0 1000000 25397.0 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -DatabaseService-64-8-65536-1 1 64 65536 1 55263.89 18095.0 499843 2201.0 500157 39103.0 +DatabaseService-64-8-65536-1 1 64 65536 1 55202.87 18115.0 499487 2095.0 500513 42239.0 ### Workflow @@ -278,6 +279,114 @@ This skips loading (`-sl`), as data is already present in the database. ```bash ## Show Summary +## Show Summary + +### Workload +TPC-H Queries SF=3 + Type: tpch + Duration: 262s + Code: 1734667671 + This includes the reading queries of TPC-H. + This experiment compares run time and resource consumption of TPC-H queries in different DBMS. + TPC-H (SF=3) data is loaded and benchmark is executed. + Query ordering is Q1 - Q22. + All instances use the same query parameters. + Timeout per query is 1200. + Import sets indexes and constraints after loading and recomputes statistics. + System metrics are monitored by a cluster-wide installation. + Benchmark is limited to DBMS ['DatabaseService']. + Import is handled by 8 processes (pods). + Loading is fixed to cl-worker19. + Benchmarking is fixed to cl-worker19. + SUT is fixed to cl-worker11. + Loading is skipped. + Loading is tested with [8] threads, split into [8] pods. + Benchmarking is tested with [1] threads, split into [1] pods. + Benchmarking is run as [1] times the number of benchmarking pods. + Experiment is run once. + +### Connections +DatabaseService-BHT-8-1-1 uses docker image postgres:16.1 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:249256216 + datadisk:39348 + requests_cpu:4 + requests_memory:16Gi + +### Errors (failed queries) +No errors + +### Warnings (result mismatch) +No warnings + +### Latency of Timer Execution [ms] +DBMS DatabaseService-BHT-8-1-1 +Pricing Summary Report (TPC-H Q1) 6149.40 +Minimum Cost Supplier Query (TPC-H Q2) 2110.09 +Shipping Priority (TPC-H Q3) 2435.87 +Order Priority Checking Query (TPC-H Q4) 3075.44 +Local Supplier Volume (TPC-H Q5) 2234.95 +Forecasting Revenue Change (TPC-H Q6) 1171.11 +Forecasting Revenue Change (TPC-H Q7) 2288.60 +National Market Share (TPC-H Q8) 1388.84 +Product Type Profit Measure (TPC-H Q9) 3168.23 +Forecasting Revenue Change (TPC-H Q10) 3075.63 +Important Stock Identification (TPC-H Q11) 563.29 +Shipping Modes and Order Priority (TPC-H Q12) 2453.85 +Customer Distribution (TPC-H Q13) 6242.59 +Forecasting Revenue Change (TPC-H Q14) 1271.74 +Top Supplier Query (TPC-H Q15) 1382.80 +Parts/Supplier Relationship (TPC-H Q16) 1349.50 +Small-Quantity-Order Revenue (TPC-H Q17) 5621.15 +Large Volume Customer (TPC-H Q18) 18750.06 +Discounted Revenue (TPC-H Q19) 1919.85 +Potential Part Promotion (TPC-H Q20) 1131.92 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2704.33 +Global Sales Opportunity Query (TPC-H Q22) 444.20 + +### Loading [s] + timeGenerate timeIngesting timeSchema timeIndex timeLoad +DatabaseService-BHT-8-1-1 0 0 0 0 0 + +### Geometric Mean of Medians of Timer Run [s] + Geo Times [s] +DBMS +DatabaseService-BHT-8-1-1 2.29 + +### Power@Size + Power@Size [~Q/h] +DBMS +DatabaseService-BHT-8-1-1 4850.83 + +### Throughput@Size + time [s] count SF Throughput@Size [~GB/h] +DBMS SF num_experiment num_client +DatabaseService-BHT-8-1 3 1 1 76 1 3 3126.32 + +### Workflow + +#### Actual +DBMS DatabaseService-BHT-8 - Pods [[1]] + +#### Planned +DBMS DatabaseService-BHT-8 - Pods [[1]] + +### Execution - Benchmarker + CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] +DatabaseService-BHT-8-1 11.36 0 0.22 0.24 + +### Tests +TEST passed: Geo Times [s] contains no 0 or NaN +TEST passed: Power@Size [~Q/h] contains no 0 or NaN +TEST passed: Throughput@Size [~GB/h] contains no 0 or NaN +TEST passed: No SQL errors +TEST passed: No SQL warnings +TEST passed: Execution Benchmarker contains no 0 or NaN in CPU [CPUs] +TEST passed: Workflow as planned ``` @@ -338,7 +447,75 @@ If data is delete somehow, this PVC information will be outdated and wrong. This approach helps bexhoma to persist status information, but it does not persist data inside the Cloud database. +```bash +## Show Summary + +### Workload +YCSB SF=5 + Type: ycsb + Duration: 3718s + Code: 1734700853 + YCSB tool runs the benchmark. + This experiment compares run time and resource consumption of YCSB queries. + Workload is 'A'. Number of rows to insert is 5000000. Number of operations is 10000000. Batch size is ''. + YCSB is performed using several threads and processes. Target is based on multiples of '16384'. Factors for loading are [4]. Factors for benchmarking are [4]. + System metrics are monitored by a cluster-wide installation. + Benchmark is limited to DBMS ['DatabaseService']. + Import is handled by 8 processes (pods). + Loading is fixed to cl-worker19. + Benchmarking is fixed to cl-worker19. + SUT is fixed to cl-worker11. + Database is persisted to disk of type shared and size 1Gi. + Loading is tested with [64] threads, split into [8] pods. + Benchmarking is tested with [64] threads, split into [1] pods. + Benchmarking is run as [1] times the number of benchmarking pods. + Experiment is run once. + +### Connections +DatabaseService-64-8-65536-1 uses docker image postgres:16.1 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:250060572 + datadisk:39192 + volume_size:1.0G + volume_used:36M + requests_cpu:4 + requests_memory:16Gi + +### Loading + experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [INSERT].Return=OK [INSERT].99thPercentileLatency(us) +DatabaseService-64-8-65536 1 64 65536 8 34656.857878 145263.0 5000000 6815.5 + +### Execution + experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) +DatabaseService-64-8-65536-1 1 64 65536 1 65353.5 153014.0 4997965 653.0 5002035 1296.0 + +### Workflow +#### Actual +DBMS DatabaseService-64-8-65536 - Pods [[1]] + +#### Planned +DBMS DatabaseService-64-8-65536 - Pods [[1]] + +### Ingestion - Loader + CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] +DatabaseService-64-8-65536-1 397.12 0.88 4.6 4.62 + +### Execution - Benchmarker + CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] +DatabaseService-64-8-65536-1 743.75 5.11 0.6 0.61 + +### Tests +TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN +TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN +TEST passed: Ingestion Loader contains no 0 or NaN in CPU [CPUs] +TEST passed: Execution Benchmarker contains no 0 or NaN in CPU [CPUs] +TEST passed: Workflow as planned +``` ## YCSB Example Explained @@ -444,8 +621,8 @@ yields ### Workload Benchbase Workload SF=16 (warehouses for TPC-C) Type: benchbase - Duration: 1148s - Code: 1734425508 + Duration: 1091s + Code: 1734664810 Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. @@ -467,7 +644,7 @@ DatabaseService-1-1-1024-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005104 + disk:249256016 datadisk:39348 requests_cpu:4 requests_memory:16Gi @@ -477,30 +654,30 @@ DatabaseService-1-1-1024-2 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005108 + disk:249256020 datadisk:39348 requests_cpu:4 requests_memory:16Gi ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -DatabaseService-1-1-1024-1 1 16 16384 1 300.0 1922.22 18733.0 8317.0 -DatabaseService-1-1-1024-2 1 16 16384 2 300.0 1742.44 21860.0 9174.5 +DatabaseService-1-1-1024-1 1 16 16384 1 300.0 1873.16 19246.0 8535.0 +DatabaseService-1-1-1024-2 1 16 16384 2 300.0 1820.81 21236.0 8782.5 Warehouses: 16 ### Workflow #### Actual -DBMS DatabaseService-1-1-1024 - Pods [[2, 1]] +DBMS DatabaseService-1-1-1024 - Pods [[1, 2]] #### Planned DBMS DatabaseService-1-1-1024 - Pods [[1, 2]] ### Loading time_load terminals pods Imported warehouses [1/h] -DatabaseService-1-1-1024-1 149.0 1.0 1.0 386.577181 -DatabaseService-1-1-1024-2 149.0 1.0 2.0 386.577181 +DatabaseService-1-1-1024-1 150.0 1.0 1.0 384.0 +DatabaseService-1-1-1024-2 150.0 1.0 2.0 384.0 ### Tests TEST passed: Throughput (requests/second) contains no 0 or NaN @@ -538,14 +715,14 @@ nohup python benchbase.py -ms 2 -tr \ yields -``` +```bash ## Show Summary ### Workload Benchbase Workload SF=16 (warehouses for TPC-C) Type: benchbase - Duration: 817s - Code: 1734426709 + Duration: 814s + Code: 1734665950 Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. @@ -568,7 +745,7 @@ DatabaseService-1-1-1024-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005112 + disk:249256028 datadisk:39348 requests_cpu:4 requests_memory:16Gi @@ -578,22 +755,22 @@ DatabaseService-1-1-1024-2 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005116 + disk:249256028 datadisk:39348 requests_cpu:4 requests_memory:16Gi ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -DatabaseService-1-1-1024-1 1 16 16384 1 300.0 1934.14 18523.0 8265.0 -DatabaseService-1-1-1024-2 1 16 16384 2 300.0 1805.90 21119.0 8852.0 +DatabaseService-1-1-1024-1 1 16 16384 1 300.0 1948.43 18329.0 8206.0 +DatabaseService-1-1-1024-2 1 16 16384 2 300.0 1774.67 21116.0 9008.5 Warehouses: 16 ### Workflow #### Actual -DBMS DatabaseService-1-1-1024 - Pods [[2, 1]] +DBMS DatabaseService-1-1-1024 - Pods [[1, 2]] #### Planned DBMS DatabaseService-1-1-1024 - Pods [[1, 2]] @@ -642,8 +819,8 @@ yields ### Workload TPC-H Queries SF=3 Type: tpch - Duration: 743s - Code: 1734427906 + Duration: 765s + Code: 1734666830 This includes the reading queries of TPC-H. This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. @@ -669,7 +846,7 @@ DatabaseService-BHT-8-1-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005324 + disk:249256232 datadisk:39348 requests_cpu:4 requests_memory:16Gi @@ -682,47 +859,47 @@ No warnings ### Latency of Timer Execution [ms] DBMS DatabaseService-BHT-8-1-1 -Pricing Summary Report (TPC-H Q1) 5931.50 -Minimum Cost Supplier Query (TPC-H Q2) 1926.02 -Shipping Priority (TPC-H Q3) 2296.96 -Order Priority Checking Query (TPC-H Q4) 2970.31 -Local Supplier Volume (TPC-H Q5) 2109.38 -Forecasting Revenue Change (TPC-H Q6) 1111.81 -Forecasting Revenue Change (TPC-H Q7) 2121.34 -National Market Share (TPC-H Q8) 1342.81 -Product Type Profit Measure (TPC-H Q9) 2880.66 -Forecasting Revenue Change (TPC-H Q10) 2864.12 -Important Stock Identification (TPC-H Q11) 572.12 -Shipping Modes and Order Priority (TPC-H Q12) 2198.12 -Customer Distribution (TPC-H Q13) 5642.77 -Forecasting Revenue Change (TPC-H Q14) 1207.98 -Top Supplier Query (TPC-H Q15) 1347.68 -Parts/Supplier Relationship (TPC-H Q16) 1227.46 -Small-Quantity-Order Revenue (TPC-H Q17) 4907.25 -Large Volume Customer (TPC-H Q18) 19409.20 -Discounted Revenue (TPC-H Q19) 1747.85 -Potential Part Promotion (TPC-H Q20) 1089.56 -Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2588.70 -Global Sales Opportunity Query (TPC-H Q22) 461.17 +Pricing Summary Report (TPC-H Q1) 6184.88 +Minimum Cost Supplier Query (TPC-H Q2) 2121.70 +Shipping Priority (TPC-H Q3) 2477.89 +Order Priority Checking Query (TPC-H Q4) 3120.41 +Local Supplier Volume (TPC-H Q5) 2263.59 +Forecasting Revenue Change (TPC-H Q6) 1158.38 +Forecasting Revenue Change (TPC-H Q7) 2326.20 +National Market Share (TPC-H Q8) 1410.37 +Product Type Profit Measure (TPC-H Q9) 3187.17 +Forecasting Revenue Change (TPC-H Q10) 3063.93 +Important Stock Identification (TPC-H Q11) 561.36 +Shipping Modes and Order Priority (TPC-H Q12) 2457.12 +Customer Distribution (TPC-H Q13) 6562.69 +Forecasting Revenue Change (TPC-H Q14) 1280.33 +Top Supplier Query (TPC-H Q15) 1396.18 +Parts/Supplier Relationship (TPC-H Q16) 1346.35 +Small-Quantity-Order Revenue (TPC-H Q17) 5626.89 +Large Volume Customer (TPC-H Q18) 19220.56 +Discounted Revenue (TPC-H Q19) 1909.47 +Potential Part Promotion (TPC-H Q20) 1216.54 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2784.81 +Global Sales Opportunity Query (TPC-H Q22) 465.76 ### Loading [s] timeGenerate timeIngesting timeSchema timeIndex timeLoad -DatabaseService-BHT-8-1-1 1.0 101.0 1.0 193.0 303.0 +DatabaseService-BHT-8-1-1 1.0 97.0 1.0 216.0 322.0 ### Geometric Mean of Medians of Timer Run [s] Geo Times [s] DBMS -DatabaseService-BHT-8-1-1 2.16 +DatabaseService-BHT-8-1-1 2.32 ### Power@Size Power@Size [~Q/h] DBMS -DatabaseService-BHT-8-1-1 5125.94 +DatabaseService-BHT-8-1-1 4783.05 ### Throughput@Size time [s] count SF Throughput@Size [~GB/h] DBMS SF num_experiment num_client -DatabaseService-BHT-8-1 3 1 1 76 1 3 3126.32 +DatabaseService-BHT-8-1 3 1 1 77 1 3 3085.71 ### Workflow @@ -734,11 +911,11 @@ DBMS DatabaseService-BHT-8 - Pods [[1]] ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 30.75 0.14 0.02 2.27 +DatabaseService-BHT-8-1 30.9 0.21 0.03 2.27 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 15.79 0 0.25 0.26 +DatabaseService-BHT-8-1 11.41 0 0.23 0.24 ### Tests TEST passed: Geo Times [s] contains no 0 or NaN @@ -778,7 +955,7 @@ yields TPC-H Queries SF=3 Type: tpch Duration: 262s - Code: 1734428687 + Code: 1734667671 This includes the reading queries of TPC-H. This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. @@ -805,7 +982,7 @@ DatabaseService-BHT-8-1-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005304 + disk:249256216 datadisk:39348 requests_cpu:4 requests_memory:16Gi @@ -818,28 +995,28 @@ No warnings ### Latency of Timer Execution [ms] DBMS DatabaseService-BHT-8-1-1 -Pricing Summary Report (TPC-H Q1) 5799.63 -Minimum Cost Supplier Query (TPC-H Q2) 1945.49 -Shipping Priority (TPC-H Q3) 2259.57 -Order Priority Checking Query (TPC-H Q4) 2853.45 -Local Supplier Volume (TPC-H Q5) 2054.87 -Forecasting Revenue Change (TPC-H Q6) 1110.50 -Forecasting Revenue Change (TPC-H Q7) 2097.45 -National Market Share (TPC-H Q8) 1330.24 -Product Type Profit Measure (TPC-H Q9) 2838.46 -Forecasting Revenue Change (TPC-H Q10) 2848.10 -Important Stock Identification (TPC-H Q11) 571.67 -Shipping Modes and Order Priority (TPC-H Q12) 2222.90 -Customer Distribution (TPC-H Q13) 5663.70 -Forecasting Revenue Change (TPC-H Q14) 1178.77 -Top Supplier Query (TPC-H Q15) 1409.79 -Parts/Supplier Relationship (TPC-H Q16) 1229.45 -Small-Quantity-Order Revenue (TPC-H Q17) 5089.27 -Large Volume Customer (TPC-H Q18) 20425.52 -Discounted Revenue (TPC-H Q19) 1768.51 -Potential Part Promotion (TPC-H Q20) 1041.16 -Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2462.41 -Global Sales Opportunity Query (TPC-H Q22) 446.75 +Pricing Summary Report (TPC-H Q1) 6149.40 +Minimum Cost Supplier Query (TPC-H Q2) 2110.09 +Shipping Priority (TPC-H Q3) 2435.87 +Order Priority Checking Query (TPC-H Q4) 3075.44 +Local Supplier Volume (TPC-H Q5) 2234.95 +Forecasting Revenue Change (TPC-H Q6) 1171.11 +Forecasting Revenue Change (TPC-H Q7) 2288.60 +National Market Share (TPC-H Q8) 1388.84 +Product Type Profit Measure (TPC-H Q9) 3168.23 +Forecasting Revenue Change (TPC-H Q10) 3075.63 +Important Stock Identification (TPC-H Q11) 563.29 +Shipping Modes and Order Priority (TPC-H Q12) 2453.85 +Customer Distribution (TPC-H Q13) 6242.59 +Forecasting Revenue Change (TPC-H Q14) 1271.74 +Top Supplier Query (TPC-H Q15) 1382.80 +Parts/Supplier Relationship (TPC-H Q16) 1349.50 +Small-Quantity-Order Revenue (TPC-H Q17) 5621.15 +Large Volume Customer (TPC-H Q18) 18750.06 +Discounted Revenue (TPC-H Q19) 1919.85 +Potential Part Promotion (TPC-H Q20) 1131.92 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2704.33 +Global Sales Opportunity Query (TPC-H Q22) 444.20 ### Loading [s] timeGenerate timeIngesting timeSchema timeIndex timeLoad @@ -848,17 +1025,17 @@ DatabaseService-BHT-8-1-1 0 0 0 0 ### Geometric Mean of Medians of Timer Run [s] Geo Times [s] DBMS -DatabaseService-BHT-8-1-1 2.16 +DatabaseService-BHT-8-1-1 2.29 ### Power@Size Power@Size [~Q/h] DBMS -DatabaseService-BHT-8-1-1 5156.12 +DatabaseService-BHT-8-1-1 4850.83 ### Throughput@Size time [s] count SF Throughput@Size [~GB/h] DBMS SF num_experiment num_client -DatabaseService-BHT-8-1 3 1 1 74 1 3 3210.81 +DatabaseService-BHT-8-1 3 1 1 76 1 3 3126.32 ### Workflow @@ -870,7 +1047,7 @@ DBMS DatabaseService-BHT-8 - Pods [[1]] ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 15.5 0 0.26 0.28 +DatabaseService-BHT-8-1 11.36 0 0.22 0.24 ### Tests TEST passed: Geo Times [s] contains no 0 or NaN @@ -929,8 +1106,8 @@ yields ### Workload TPC-H Queries SF=3 Type: tpch - Duration: 796s - Code: 1734429037 + Duration: 798s + Code: 1734668021 This includes the reading queries of TPC-H. This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. @@ -957,7 +1134,7 @@ DatabaseService-BHT-8-1-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:235965984 + disk:249216892 datadisk:39192 volume_size:1.0G volume_used:36M @@ -972,47 +1149,47 @@ No warnings ### Latency of Timer Execution [ms] DBMS DatabaseService-BHT-8-1-1 -Pricing Summary Report (TPC-H Q1) 5744.14 -Minimum Cost Supplier Query (TPC-H Q2) 2022.89 -Shipping Priority (TPC-H Q3) 2325.90 -Order Priority Checking Query (TPC-H Q4) 2921.01 -Local Supplier Volume (TPC-H Q5) 2085.64 -Forecasting Revenue Change (TPC-H Q6) 1132.48 -Forecasting Revenue Change (TPC-H Q7) 2157.45 -National Market Share (TPC-H Q8) 1333.06 -Product Type Profit Measure (TPC-H Q9) 2966.91 -Forecasting Revenue Change (TPC-H Q10) 2810.67 -Important Stock Identification (TPC-H Q11) 589.44 -Shipping Modes and Order Priority (TPC-H Q12) 2227.10 -Customer Distribution (TPC-H Q13) 5779.52 -Forecasting Revenue Change (TPC-H Q14) 1224.41 -Top Supplier Query (TPC-H Q15) 1354.01 -Parts/Supplier Relationship (TPC-H Q16) 1196.48 -Small-Quantity-Order Revenue (TPC-H Q17) 5286.03 -Large Volume Customer (TPC-H Q18) 17941.22 -Discounted Revenue (TPC-H Q19) 1796.70 -Potential Part Promotion (TPC-H Q20) 1085.69 -Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2513.56 -Global Sales Opportunity Query (TPC-H Q22) 480.80 +Pricing Summary Report (TPC-H Q1) 5793.75 +Minimum Cost Supplier Query (TPC-H Q2) 1922.31 +Shipping Priority (TPC-H Q3) 2231.66 +Order Priority Checking Query (TPC-H Q4) 2835.42 +Local Supplier Volume (TPC-H Q5) 2072.36 +Forecasting Revenue Change (TPC-H Q6) 1066.68 +Forecasting Revenue Change (TPC-H Q7) 2155.03 +National Market Share (TPC-H Q8) 1301.26 +Product Type Profit Measure (TPC-H Q9) 2787.60 +Forecasting Revenue Change (TPC-H Q10) 2791.42 +Important Stock Identification (TPC-H Q11) 559.95 +Shipping Modes and Order Priority (TPC-H Q12) 2143.46 +Customer Distribution (TPC-H Q13) 5364.32 +Forecasting Revenue Change (TPC-H Q14) 1190.47 +Top Supplier Query (TPC-H Q15) 1291.81 +Parts/Supplier Relationship (TPC-H Q16) 1127.44 +Small-Quantity-Order Revenue (TPC-H Q17) 4912.21 +Large Volume Customer (TPC-H Q18) 16174.10 +Discounted Revenue (TPC-H Q19) 1735.84 +Potential Part Promotion (TPC-H Q20) 1088.33 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2474.25 +Global Sales Opportunity Query (TPC-H Q22) 488.84 ### Loading [s] timeGenerate timeIngesting timeSchema timeIndex timeLoad -DatabaseService-BHT-8-1-1 1.0 122.0 1.0 198.0 330.0 +DatabaseService-BHT-8-1-1 1.0 123.0 1.0 196.0 326.0 ### Geometric Mean of Medians of Timer Run [s] Geo Times [s] DBMS -DatabaseService-BHT-8-1-1 2.17 +DatabaseService-BHT-8-1-1 2.11 ### Power@Size Power@Size [~Q/h] DBMS -DatabaseService-BHT-8-1-1 5095.38 +DatabaseService-BHT-8-1-1 5279.56 ### Throughput@Size time [s] count SF Throughput@Size [~GB/h] DBMS SF num_experiment num_client -DatabaseService-BHT-8-1 3 1 1 73 1 3 3254.79 +DatabaseService-BHT-8-1 3 1 1 70 1 3 3394.29 ### Workflow @@ -1024,11 +1201,11 @@ DBMS DatabaseService-BHT-8 - Pods [[1]] ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 40.99 0.07 0.03 2.8 +DatabaseService-BHT-8-1 41.77 0.04 0.02 2.8 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 11.61 0 0.22 0.23 +DatabaseService-BHT-8-1 15.95 0 0.26 0.27 ### Tests TEST passed: Geo Times [s] contains no 0 or NaN @@ -1043,6 +1220,10 @@ TEST passed: Workflow as planned ### Execution Only with Persistent Storage +Data is now present in the database. +The persistent volume helps to memorize this. +We can rerun the experiment without explicitly skipping ingestion (no `-sl`). + ```bash nohup python tpch.py -ms 2 -dt -tr \ -dbms DatabaseService \ @@ -1065,8 +1246,8 @@ yields ### Workload TPC-H Queries SF=3 Type: tpch - Duration: 262s - Code: 1734429878 + Duration: 273s + Code: 1734668861 This includes the reading queries of TPC-H. This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. @@ -1093,7 +1274,7 @@ DatabaseService-BHT-8-1-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:235965964 + disk:249216876 datadisk:39192 volume_size:1.0G volume_used:36M @@ -1108,47 +1289,47 @@ No warnings ### Latency of Timer Execution [ms] DBMS DatabaseService-BHT-8-1-1 -Pricing Summary Report (TPC-H Q1) 5713.84 -Minimum Cost Supplier Query (TPC-H Q2) 1886.65 -Shipping Priority (TPC-H Q3) 2253.28 -Order Priority Checking Query (TPC-H Q4) 2854.45 -Local Supplier Volume (TPC-H Q5) 2075.71 -Forecasting Revenue Change (TPC-H Q6) 1101.50 -Forecasting Revenue Change (TPC-H Q7) 2145.88 -National Market Share (TPC-H Q8) 1332.73 -Product Type Profit Measure (TPC-H Q9) 2925.21 -Forecasting Revenue Change (TPC-H Q10) 2867.79 -Important Stock Identification (TPC-H Q11) 564.90 -Shipping Modes and Order Priority (TPC-H Q12) 2189.70 -Customer Distribution (TPC-H Q13) 5426.61 -Forecasting Revenue Change (TPC-H Q14) 1194.76 -Top Supplier Query (TPC-H Q15) 1364.10 -Parts/Supplier Relationship (TPC-H Q16) 1319.02 -Small-Quantity-Order Revenue (TPC-H Q17) 5020.23 -Large Volume Customer (TPC-H Q18) 16806.97 -Discounted Revenue (TPC-H Q19) 1775.99 -Potential Part Promotion (TPC-H Q20) 1037.63 -Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2513.75 -Global Sales Opportunity Query (TPC-H Q22) 475.99 +Pricing Summary Report (TPC-H Q1) 5785.36 +Minimum Cost Supplier Query (TPC-H Q2) 1938.19 +Shipping Priority (TPC-H Q3) 2255.59 +Order Priority Checking Query (TPC-H Q4) 2827.21 +Local Supplier Volume (TPC-H Q5) 2018.18 +Forecasting Revenue Change (TPC-H Q6) 1062.37 +Forecasting Revenue Change (TPC-H Q7) 2075.73 +National Market Share (TPC-H Q8) 1336.07 +Product Type Profit Measure (TPC-H Q9) 2808.84 +Forecasting Revenue Change (TPC-H Q10) 2804.71 +Important Stock Identification (TPC-H Q11) 583.68 +Shipping Modes and Order Priority (TPC-H Q12) 2137.04 +Customer Distribution (TPC-H Q13) 5569.93 +Forecasting Revenue Change (TPC-H Q14) 1130.83 +Top Supplier Query (TPC-H Q15) 1321.32 +Parts/Supplier Relationship (TPC-H Q16) 1239.32 +Small-Quantity-Order Revenue (TPC-H Q17) 5228.78 +Large Volume Customer (TPC-H Q18) 17602.90 +Discounted Revenue (TPC-H Q19) 1735.06 +Potential Part Promotion (TPC-H Q20) 1018.27 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2464.08 +Global Sales Opportunity Query (TPC-H Q22) 451.76 ### Loading [s] timeGenerate timeIngesting timeSchema timeIndex timeLoad -DatabaseService-BHT-8-1-1 1.0 122.0 1.0 198.0 330.0 +DatabaseService-BHT-8-1-1 1.0 123.0 1.0 196.0 326.0 ### Geometric Mean of Medians of Timer Run [s] Geo Times [s] DBMS -DatabaseService-BHT-8-1-1 2.14 +DatabaseService-BHT-8-1-1 2.11 ### Power@Size Power@Size [~Q/h] DBMS -DatabaseService-BHT-8-1-1 5185.98 +DatabaseService-BHT-8-1-1 5249.82 ### Throughput@Size time [s] count SF Throughput@Size [~GB/h] DBMS SF num_experiment num_client -DatabaseService-BHT-8-1 3 1 1 70 1 3 3394.29 +DatabaseService-BHT-8-1 3 1 1 71 1 3 3346.48 ### Workflow @@ -1160,7 +1341,7 @@ DBMS DatabaseService-BHT-8 - Pods [[1]] ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 16.29 0.32 0.25 0.27 +DatabaseService-BHT-8-1 16.0 0 0.25 0.27 ### Tests TEST passed: Geo Times [s] contains no 0 or NaN diff --git a/docs/Example-CockroachDB.md b/docs/Example-CockroachDB.md index 5c596d78..19874adb 100644 --- a/docs/Example-CockroachDB.md +++ b/docs/Example-CockroachDB.md @@ -121,8 +121,8 @@ At the end of a benchmark you will see a summary like ### Workload YCSB SF=1 Type: ycsb - Duration: 1055s - Code: 1734604175 + Duration: 1056s + Code: 1734645173 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. @@ -144,25 +144,25 @@ CockroachDB-64-8-65536-1 uses docker image cockroachdb/cockroach:v24.2.4 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:249207160 + disk:249215592 requests_cpu:4 requests_memory:16Gi worker 0 - RAM:1081965555712 + RAM:1081966526464 Cores:256 host:5.15.0-1067-nvidia - node:cl-worker27 - disk:560813152 - datadisk:116276744 + node:cl-worker28 + disk:446476124 + datadisk:116276640 volume_size:1000G volume_used:109G worker 1 - RAM:1081966526464 + RAM:1081965555712 Cores:256 host:5.15.0-1067-nvidia - node:cl-worker28 - disk:442719496 - datadisk:116066508 + node:cl-worker27 + disk:634371096 + datadisk:116064760 volume_size:1000G volume_used:109G worker 2 @@ -170,18 +170,18 @@ CockroachDB-64-8-65536-1 uses docker image cockroachdb/cockroach:v24.2.4 Cores:128 host:5.15.0-126-generic node:cl-worker29 - disk:153230928 - datadisk:116065692 + disk:153231576 + datadisk:116065120 volume_size:1000G volume_used:109G ### Loading experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [INSERT].Return=OK [INSERT].99thPercentileLatency(us) -CockroachDB-64-8-65536 1 64 65536 8 17071.978081 58791.0 1000000 7190.0 +CockroachDB-64-8-65536 1 64 65536 8 16211.343884 61937.0 1000000 7579.5 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -CockroachDB-64-8-65536-1 1 64 65536 1 14391.04 694877.0 5000111 5691.0 4999889 126207.0 +CockroachDB-64-8-65536-1 1 64 65536 1 14106.68 708884.0 5000094 5851.0 4999906 130879.0 ### Workflow @@ -193,19 +193,19 @@ DBMS CockroachDB-64-8-65536 - Pods [[1]] ### Ingestion - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -CockroachDB-64-8-65536-1 1295.39 6.78 3.68 6.88 +CockroachDB-64-8-65536-1 888.04 0.03 3.05 5.76 ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -CockroachDB-64-8-65536-1 22.52 0.41 0.58 0.59 +CockroachDB-64-8-65536-1 103.7 0 4.34 4.37 ### Execution - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -CockroachDB-64-8-65536-1 21640.62 21.35 12.76 28.74 +CockroachDB-64-8-65536-1 20657.12 20.74 12.51 26.94 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -CockroachDB-64-8-65536-1 1181.64 1.78 0.6 0.61 +CockroachDB-64-8-65536-1 1024.62 1.56 0.6 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN @@ -321,8 +321,8 @@ nohup python benchbase.py -ms 1 -tr \ ### Workload Benchbase Workload SF=16 (warehouses for TPC-C) Type: benchbase - Duration: 1047s - Code: 1734624013 + Duration: 1166s + Code: 1734646253 Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. @@ -343,7 +343,7 @@ CockroachDB-1-1-1024-1 uses docker image cockroachdb/cockroach:v24.2.4 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:249214372 + disk:249215596 requests_cpu:4 requests_memory:16Gi worker 0 @@ -351,26 +351,26 @@ CockroachDB-1-1-1024-1 uses docker image cockroachdb/cockroach:v24.2.4 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:439691956 - datadisk:116318604 + disk:461657896 + datadisk:116314488 volume_size:1000G volume_used:109G worker 1 - RAM:1081751019520 - Cores:128 - host:5.15.0-126-generic - node:cl-worker29 - disk:153273172 - datadisk:116107360 - volume_size:1000G - volume_used:109G - worker 2 RAM:1081965555712 Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:584549888 - datadisk:116107116 + disk:635102812 + datadisk:116104180 + volume_size:1000G + volume_used:109G + worker 2 + RAM:540587499520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:123840188 + datadisk:116091372 volume_size:1000G volume_used:109G CockroachDB-1-1-1024-2 uses docker image cockroachdb/cockroach:v24.2.4 @@ -378,7 +378,7 @@ CockroachDB-1-1-1024-2 uses docker image cockroachdb/cockroach:v24.2.4 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:249214372 + disk:249215600 requests_cpu:4 requests_memory:16Gi worker 0 @@ -386,48 +386,48 @@ CockroachDB-1-1-1024-2 uses docker image cockroachdb/cockroach:v24.2.4 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:439986660 - datadisk:116611832 + disk:461867536 + datadisk:116522308 volume_size:1000G volume_used:109G worker 1 - RAM:1081751019520 - Cores:128 - host:5.15.0-126-generic - node:cl-worker29 - disk:153570092 - datadisk:116404276 - volume_size:1000G - volume_used:109G - worker 2 RAM:1081965555712 Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:584845212 - datadisk:116401204 + disk:635488320 + datadisk:116308436 + volume_size:1000G + volume_used:109G + worker 2 + RAM:540587499520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:124062476 + datadisk:116312956 volume_size:1000G volume_used:109G ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -CockroachDB-1-1-1024-1 1 16 16384 1 300.0 481.72 84008.0 33199.0 -CockroachDB-1-1-1024-2 1 16 16384 2 300.0 448.16 95345.0 35721.0 +CockroachDB-1-1-1024-1 1 16 16384 1 300.0 312.89 95381.0 51118.0 +CockroachDB-1-1-1024-2 1 16 16384 2 300.0 241.61 142861.0 66206.0 Warehouses: 16 ### Workflow #### Actual -DBMS CockroachDB-1-1-1024 - Pods [[2, 1]] +DBMS CockroachDB-1-1-1024 - Pods [[1, 2]] #### Planned DBMS CockroachDB-1-1-1024 - Pods [[1, 2]] ### Loading time_load terminals pods Imported warehouses [1/h] -CockroachDB-1-1-1024-1 142.0 1.0 1.0 405.633803 -CockroachDB-1-1-1024-2 142.0 1.0 2.0 405.633803 +CockroachDB-1-1-1024-1 267.0 1.0 1.0 215.730337 +CockroachDB-1-1-1024-2 267.0 1.0 2.0 215.730337 ### Tests TEST passed: Throughput (requests/second) contains no 0 or NaN @@ -461,9 +461,9 @@ nohup python benchbase.py -ms 1 -tr \ ### Workload Benchbase Workload SF=128 (warehouses for TPC-C) Type: benchbase - Duration: 15418s - Code: 1730374413 - This includes no queries. Benchbase runs the benchmark + Duration: 15957s + Code: 1734647454 + Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. Benchmark is 'tpcc'. Scaling factor (e.g., number of warehouses) is 128. Benchmarking runs for 60 minutes. Target is based on multiples of '1024'. Factors for benchmarking are [16]. @@ -479,61 +479,169 @@ Benchbase Workload SF=128 (warehouses for TPC-C) ### Connections CockroachDB-1-1-1024-1 uses docker image cockroachdb/cockroach:v24.2.4 - RAM:541008605184 + RAM:541008576512 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254912048 + disk:249215616 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:469202612 + datadisk:123845016 + volume_size:1000G + volume_used:109G + worker 1 + RAM:1081751019520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker29 + disk:160789996 + datadisk:123623440 + volume_size:1000G + volume_used:109G + worker 2 + RAM:1081965555712 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:644345576 + datadisk:123626156 + volume_size:1000G + volume_used:109G CockroachDB-1-1-1024-2 uses docker image cockroachdb/cockroach:v24.2.4 - RAM:541008605184 + RAM:541008576512 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254912216 + disk:249215820 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:475115332 + datadisk:129756564 + volume_size:1000G + volume_used:109G + worker 1 + RAM:1081751019520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker29 + disk:166744188 + datadisk:129577516 + volume_size:1000G + volume_used:109G + worker 2 + RAM:1081965555712 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:654361308 + datadisk:129571596 + volume_size:1000G + volume_used:109G CockroachDB-1-1-1024-3 uses docker image cockroachdb/cockroach:v24.2.4 - RAM:541008605184 + RAM:541008576512 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254912388 + disk:249216060 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:478912840 + datadisk:133546860 + volume_size:1000G + volume_used:109G + worker 1 + RAM:1081751019520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker29 + disk:170393272 + datadisk:133226492 + volume_size:1000G + volume_used:109G + worker 2 + RAM:1081965555712 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:661848004 + datadisk:133214428 + volume_size:1000G + volume_used:109G CockroachDB-1-1-1024-4 uses docker image cockroachdb/cockroach:v24.2.4 - RAM:541008605184 + RAM:541008576512 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254912556 + disk:249216408 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:481732908 + datadisk:136364000 + volume_size:1000G + volume_used:109G + worker 1 + RAM:1081751019520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker29 + disk:173351696 + datadisk:136184808 + volume_size:1000G + volume_used:109G + worker 2 + RAM:1081965555712 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:667527460 + datadisk:136153100 + volume_size:1000G + volume_used:109G ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -CockroachDB-1-1-1024-1 1 64 16384 1 3600.0 696.70 233544.0 91853.00 -CockroachDB-1-1-1024-2 1 64 16384 2 3600.0 653.01 247449.0 97998.00 -CockroachDB-1-1-1024-3 1 64 16384 4 3600.0 599.12 258276.0 106813.50 -CockroachDB-1-1-1024-4 1 64 16384 8 3600.0 538.01 262975.0 118944.88 +CockroachDB-1-1-1024-1 1 64 16384 1 3600.0 1097.64 144074.0 58301.00 +CockroachDB-1-1-1024-2 1 64 16384 2 3600.0 1026.91 161894.0 62323.00 +CockroachDB-1-1-1024-3 1 64 16384 4 3600.0 908.92 181035.0 70443.25 +CockroachDB-1-1-1024-4 1 64 16384 8 3600.0 675.46 224333.0 94757.50 Warehouses: 128 ### Workflow #### Actual -DBMS CockroachDB-1-1-1024 - Pods [[8, 4, 2, 1]] +DBMS CockroachDB-1-1-1024 - Pods [[1, 2, 8, 4]] #### Planned DBMS CockroachDB-1-1-1024 - Pods [[1, 2, 4, 8]] ### Loading time_load terminals pods Imported warehouses [1/h] -CockroachDB-1-1-1024-1 531.0 1.0 1.0 867.79661 -CockroachDB-1-1-1024-2 531.0 1.0 2.0 867.79661 -CockroachDB-1-1-1024-3 531.0 1.0 4.0 867.79661 -CockroachDB-1-1-1024-4 531.0 1.0 8.0 867.79661 +CockroachDB-1-1-1024-1 1036.0 1.0 1.0 444.787645 +CockroachDB-1-1-1024-2 1036.0 1.0 2.0 444.787645 +CockroachDB-1-1-1024-3 1036.0 1.0 4.0 444.787645 +CockroachDB-1-1-1024-4 1036.0 1.0 8.0 444.787645 ### Tests TEST passed: Throughput (requests/second) contains no 0 or NaN diff --git a/docs/Example-YugaByteDB.md b/docs/Example-YugaByteDB.md index e929aa39..e95587ed 100644 --- a/docs/Example-YugaByteDB.md +++ b/docs/Example-YugaByteDB.md @@ -314,7 +314,7 @@ This skips loading (`-sl`), as data is already present in the database. YCSB SF=1 Type: ycsb Duration: 511s - Code: 1734598346 + Code: 1734626144 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. @@ -338,7 +338,7 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:248621276 + disk:249253676 datadisk:39268 requests_cpu:4 requests_memory:16Gi @@ -348,25 +348,25 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:444140892 + disk:441407636 worker 1 RAM:540587499520 CPU:AMD EPYC 7502 32-Core Processor Cores:128 host:5.15.0-126-generic node:cl-worker22 - disk:125097616 + disk:125219072 worker 2 RAM:1081965555712 CPU:AMD EPYC 7742 64-Core Processor Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:561947884 + disk:586475888 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -YugabyteDB-64-8-65536-1 1 64 65536 1 30985.17 322735.0 4997322 39711.0 5002678 44959.0 +YugabyteDB-64-8-65536-1 1 64 65536 1 31287.55 319616.0 4999554 39551.0 5000446 43999.0 ### Workflow @@ -378,11 +378,11 @@ DBMS YugabyteDB-64-8-65536 - Pods [[1]] ### Execution - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 14114.83 15.67 7.24 23.99 +YugabyteDB-64-8-65536-1 13772.52 16.0 7.23 23.99 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 962.1 3.24 0.61 0.61 +YugabyteDB-64-8-65536-1 982.63 3.4 0.61 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN @@ -456,8 +456,8 @@ This approach helps bexhoma to persist status information, but it does not persi ### Workload YCSB SF=1 Type: ycsb - Duration: 562s - Code: 1734599008 + Duration: 591s + Code: 1734626805 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. @@ -481,8 +481,8 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:248582020 - datadisk:39262 + disk:249214420 + datadisk:39106 volume_size:1.0G volume_used:36M requests_cpu:4 @@ -493,29 +493,29 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:441849660 + disk:439198928 worker 1 - RAM:1081965555712 - CPU:AMD EPYC 7742 64-Core Processor - Cores:256 - host:5.15.0-1067-nvidia - node:cl-worker27 - disk:559737852 - worker 2 RAM:540587499520 CPU:AMD EPYC 7502 32-Core Processor Cores:128 host:5.15.0-126-generic node:cl-worker22 - disk:122889020 + disk:122937852 + worker 2 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:584265648 ### Loading experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [INSERT].Return=OK [INSERT].99thPercentileLatency(us) -YugabyteDB-64-8-65536 1 64 65536 8 29685.991562 34115.0 1000000 11968.0 +YugabyteDB-64-8-65536 1 64 65536 8 27920.198198 36082.0 1000000 15099.0 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -YugabyteDB-64-8-65536-1 1 64 65536 1 31292.83 319562.0 4998841 26335.0 5001159 40159.0 +YugabyteDB-64-8-65536-1 1 64 65536 1 30852.87 324119.0 4999076 26703.0 5000924 44287.0 ### Workflow @@ -527,19 +527,19 @@ DBMS YugabyteDB-64-8-65536 - Pods [[1]] ### Ingestion - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 25686.07 1.15 6.32 20.86 +YugabyteDB-64-8-65536-1 14677.38 4.86 4.23 13.99 ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 36.85 0 2.05 2.06 +YugabyteDB-64-8-65536-1 0.08 0 0.01 0.01 ### Execution - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 14651.28 15.71 5.01 16.47 +YugabyteDB-64-8-65536-1 14586.01 17.96 4.88 16.69 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 1009.99 3.37 0.61 0.61 +YugabyteDB-64-8-65536-1 771.47 2.69 0.61 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN @@ -658,8 +658,8 @@ yields ### Workload Benchbase Workload SF=16 (warehouses for TPC-C) Type: benchbase - Duration: 1034s - Code: 1734601765 + Duration: 1067s + Code: 1734627587 Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. @@ -681,7 +681,7 @@ YugabyteDB-1-1-1024-1 uses docker image postgres:15.0 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:248621648 + disk:249254028 datadisk:39428 requests_cpu:4 requests_memory:16Gi @@ -691,28 +691,28 @@ YugabyteDB-1-1-1024-1 uses docker image postgres:15.0 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:443073852 + disk:440540068 worker 1 - RAM:540587499520 - CPU:AMD EPYC 7502 32-Core Processor - Cores:128 - host:5.15.0-126-generic - node:cl-worker22 - disk:124173692 - worker 2 RAM:1081965555712 CPU:AMD EPYC 7742 64-Core Processor Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:560979688 + disk:585609424 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:124275600 YugabyteDB-1-1-1024-2 uses docker image postgres:15.0 RAM:541008576512 CPU:AMD Opteron(tm) Processor 6378 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:248621648 + disk:249254032 datadisk:39428 requests_cpu:4 requests_memory:16Gi @@ -722,26 +722,26 @@ YugabyteDB-1-1-1024-2 uses docker image postgres:15.0 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:443885248 + disk:441361868 worker 1 - RAM:540587499520 - CPU:AMD EPYC 7502 32-Core Processor - Cores:128 - host:5.15.0-126-generic - node:cl-worker22 - disk:124976104 - worker 2 RAM:1081965555712 CPU:AMD EPYC 7742 64-Core Processor Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:561773616 + disk:586458512 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:125118168 ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -YugabyteDB-1-1-1024-1 1 16 16384 1 300.0 364.30 107743.0 43901.0 -YugabyteDB-1-1-1024-2 1 16 16384 2 300.0 381.44 105259.0 41945.5 +YugabyteDB-1-1-1024-1 1 16 16384 1 300.0 403.95 96762.0 39594.0 +YugabyteDB-1-1-1024-2 1 16 16384 2 300.0 362.52 111737.0 44178.0 Warehouses: 16 @@ -755,8 +755,8 @@ DBMS YugabyteDB-1-1-1024 - Pods [[1, 2]] ### Loading time_load terminals pods Imported warehouses [1/h] -YugabyteDB-1-1-1024-1 204.0 1.0 1.0 282.352941 -YugabyteDB-1-1-1024-2 204.0 1.0 2.0 282.352941 +YugabyteDB-1-1-1024-1 216.0 1.0 1.0 266.666667 +YugabyteDB-1-1-1024-2 216.0 1.0 2.0 266.666667 ### Tests TEST passed: Throughput (requests/second) contains no 0 or NaN @@ -799,5 +799,179 @@ nohup python benchbase.py -ms 1 -tr \ yields ```bash +## Show Summary + +### Workload +Benchbase Workload SF=128 (warehouses for TPC-C) + Type: benchbase + Duration: 16283s + Code: 1734628788 + Benchbase runs the benchmark. + This experiment compares run time and resource consumption of Benchbase queries in different DBMS. + Benchbase data is generated and loaded using several threads. + Benchmark is 'tpcc'. Scaling factor (e.g., number of warehouses) is 128. Benchmarking runs for 60 minutes. Target is based on multiples of '1024'. Factors for benchmarking are [16]. + Benchmark is limited to DBMS ['YugabyteDB']. + Import is handled by 1 processes (pods). + Loading is fixed to cl-worker19. + Benchmarking is fixed to cl-worker19. + SUT is fixed to cl-worker11. + Loading is tested with [1] threads, split into [1] pods. + Benchmarking is tested with [64] threads, split into [1, 2, 4, 8] pods. + Benchmarking is run as [1] times the number of benchmarking pods. + Experiment is run once. + +### Connections +YugabyteDB-1-1-1024-1 uses docker image postgres:15.0 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:249254048 + datadisk:39428 + requests_cpu:4 + requests_memory:16Gi + worker 0 + RAM:1081966526464 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:458849332 + worker 1 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:603879336 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:143144816 +YugabyteDB-1-1-1024-2 uses docker image postgres:15.0 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:249254240 + datadisk:39428 + requests_cpu:4 + requests_memory:16Gi + worker 0 + RAM:1081966526464 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:450176968 + worker 1 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:595302840 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:133934964 +YugabyteDB-1-1-1024-3 uses docker image postgres:15.0 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:249254432 + datadisk:39428 + requests_cpu:4 + requests_memory:16Gi + worker 0 + RAM:1081966526464 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:450484892 + worker 1 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:595477572 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:134049872 +YugabyteDB-1-1-1024-4 uses docker image postgres:15.0 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:249254644 + datadisk:39428 + requests_cpu:4 + requests_memory:16Gi + worker 0 + RAM:1081966526464 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:456563680 + worker 1 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:640944212 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:134098940 + +### Execution + experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) +YugabyteDB-1-1-1024-1 1 64 16384 1 3600.0 482.34 346967.0 132675.00 +YugabyteDB-1-1-1024-2 1 64 16384 2 3600.0 388.47 414796.0 164642.00 +YugabyteDB-1-1-1024-3 1 64 16384 4 3600.0 376.70 430011.0 169282.00 +YugabyteDB-1-1-1024-4 1 64 16384 8 3600.0 258.70 767035.0 244458.62 + +Warehouses: 128 + +### Workflow + +#### Actual +DBMS YugabyteDB-1-1-1024 - Pods [[8, 4, 2, 1]] + +#### Planned +DBMS YugabyteDB-1-1-1024 - Pods [[1, 2, 4, 8]] + +### Loading + time_load terminals pods Imported warehouses [1/h] +YugabyteDB-1-1-1024-1 1033.0 1.0 1.0 446.07938 +YugabyteDB-1-1-1024-2 1033.0 1.0 2.0 446.07938 +YugabyteDB-1-1-1024-3 1033.0 1.0 4.0 446.07938 +YugabyteDB-1-1-1024-4 1033.0 1.0 8.0 446.07938 + +### Tests +TEST passed: Throughput (requests/second) contains no 0 or NaN +TEST passed: Workflow as planned ``` diff --git a/logs_tests/doc_benchbase_cockroachdb_1.log b/logs_tests/doc_benchbase_cockroachdb_1.log index 66d8be43..261b43f5 100644 --- a/logs_tests/doc_benchbase_cockroachdb_1.log +++ b/logs_tests/doc_benchbase_cockroachdb_1.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734624013 -Experiment : starts at 2024-12-19 17:00:12.861793 (4308814.268927681) +Experiment : has code 1734646253 +Experiment : starts at 2024-12-19 23:10:53.358305 (4331054.765439166) Experiment : This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. Benchmark is 'tpcc'. Scaling factor (e.g., number of warehouses) is 16. Benchmarking runs for 5 minutes. Target is based on multiples of '1024'. Factors for benchmarking are [16]. @@ -23,22 +23,22 @@ done CockroachDB-1-1-1024 : will start now - waiting 30s - : done CockroachDB-1-1-1024 : is not loaded yet -CockroachDB-1-1-1024 : will start loading but not before 2024-12-19 16:03:13 (that is in 120 secs) +CockroachDB-1-1-1024 : will start loading but not before 2024-12-19 22:13:54 (that is in 120 secs) - waiting 30s - : done CockroachDB-1-1-1024 : is not loaded yet -CockroachDB-1-1-1024 : will start loading but not before 2024-12-19 16:03:13 +CockroachDB-1-1-1024 : will start loading but not before 2024-12-19 22:13:54 - waiting 30s - : done CockroachDB-1-1-1024 : is not loaded yet -CockroachDB-1-1-1024 : will start loading but not before 2024-12-19 16:03:13 +CockroachDB-1-1-1024 : will start loading but not before 2024-12-19 22:13:54 - waiting 30s - : done CockroachDB-1-1-1024 : is not loaded yet -CockroachDB-1-1-1024 : will start loading but not before 2024-12-19 16:03:13 +CockroachDB-1-1-1024 : will start loading but not before 2024-12-19 22:13:54 - waiting 30s - : done CockroachDB-1-1-1024 : is not loaded yet done -Worker pods found: ['bexhoma-worker-cockroachdb-1-1-1024-1734624013-0', 'bexhoma-worker-cockroachdb-1-1-1024-1734624013-1', 'bexhoma-worker-cockroachdb-1-1-1024-1734624013-2'] +Worker pods found: ['bexhoma-worker-cockroachdb-1-1-1024-1734646253-0', 'bexhoma-worker-cockroachdb-1-1-1024-1734646253-1', 'bexhoma-worker-cockroachdb-1-1-1024-1734646253-2'] CockroachDB-1-1-1024 Workers 3 of 3 -Worker pods found: ['bexhoma-worker-cockroachdb-1-1-1024-1734624013-0', 'bexhoma-worker-cockroachdb-1-1-1024-1734624013-1', 'bexhoma-worker-cockroachdb-1-1-1024-1734624013-2'] +Worker pods found: ['bexhoma-worker-cockroachdb-1-1-1024-1734646253-0', 'bexhoma-worker-cockroachdb-1-1-1024-1734646253-1', 'bexhoma-worker-cockroachdb-1-1-1024-1734646253-2'] CockroachDB-1-1-1024 : start asynch loading scripts of type loaded CockroachDB-1-1-1024 : is loading - waiting 30s - : done @@ -49,19 +49,27 @@ CockroachDB-1-1-1024 : is loading CockroachDB-1-1-1024 : is loading - waiting 30s - : done CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading done CockroachDB-1-1-1024 : showing loader times -CockroachDB-1-1-1024 : loader timespan (first to last [s]) = 142 +CockroachDB-1-1-1024 : loader timespan (first to last [s]) = 267 CockroachDB-1-1-1024 : benchmarks done 0 of 1. This will be client 1 CockroachDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '1', 'SF': '16', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'cockroachdb', 'BEXHOMA_DATABASE': 'defaultdb', 'BENCHBASE_TARGET': 16384, 'BENCHBASE_TERMINALS': 16, 'BENCHBASE_TIME': 300, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_USER': 'root', 'BEXHOMA_PASSWORD': ''} CockroachDB-1-1-1024-1 : start benchmarking -Worker pods found: ['bexhoma-worker-cockroachdb-1-1-1024-1734624013-0', 'bexhoma-worker-cockroachdb-1-1-1024-1734624013-1', 'bexhoma-worker-cockroachdb-1-1-1024-1734624013-2'] -CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734624013-0 -CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734624013-1 -CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734624013-2 -CockroachDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734624013 +Worker pods found: ['bexhoma-worker-cockroachdb-1-1-1024-1734646253-0', 'bexhoma-worker-cockroachdb-1-1-1024-1734646253-1', 'bexhoma-worker-cockroachdb-1-1-1024-1734646253-2'] +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734646253-0 +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734646253-1 +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734646253-2 +CockroachDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734646253 - waiting 10s - : done -CockroachDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-cockroachdb-1-1-1024-1734624013-1-1-68j7x: found +CockroachDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-cockroachdb-1-1-1024-1734646253-1-1-vbl52: found - waiting 30s - : done CockroachDB-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -83,20 +91,20 @@ CockroachDB-1-1-1024 : has running benchmarks done CockroachDB-1-1-1024 : has running benchmarks CockroachDB-1-1-1024-1 : showing benchmarker times -CockroachDB-1-1-1024-1 : benchmarker timespan (start to end single container [s]) = 313 -CockroachDB-1-1-1024-1 : benchmarker times (start/end per pod and container) = [(1734624363, 1734624667)] -CockroachDB-1-1-1024-1 : found and updated times {'benchmarker': [(1734624363, 1734624667)]} +CockroachDB-1-1-1024-1 : benchmarker timespan (start to end single container [s]) = 314 +CockroachDB-1-1-1024-1 : benchmarker times (start/end per pod and container) = [(1734646723, 1734647027)] +CockroachDB-1-1-1024-1 : found and updated times {'benchmarker': [(1734646723, 1734647027)]} done CockroachDB-1-1-1024 : benchmarks done 0 of 1. This will be client 2 CockroachDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '2', 'SF': '16', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'cockroachdb', 'BEXHOMA_DATABASE': 'defaultdb', 'BENCHBASE_TARGET': 8192, 'BENCHBASE_TERMINALS': 8, 'BENCHBASE_TIME': 300, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_USER': 'root', 'BEXHOMA_PASSWORD': ''} CockroachDB-1-1-1024-2 : start benchmarking -Worker pods found: ['bexhoma-worker-cockroachdb-1-1-1024-1734624013-0', 'bexhoma-worker-cockroachdb-1-1-1024-1734624013-1', 'bexhoma-worker-cockroachdb-1-1-1024-1734624013-2'] -CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734624013-0 -CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734624013-1 -CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734624013-2 -CockroachDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734624013 +Worker pods found: ['bexhoma-worker-cockroachdb-1-1-1024-1734646253-0', 'bexhoma-worker-cockroachdb-1-1-1024-1734646253-1', 'bexhoma-worker-cockroachdb-1-1-1024-1734646253-2'] +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734646253-0 +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734646253-1 +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734646253-2 +CockroachDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734646253 - waiting 10s - : done -CockroachDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-cockroachdb-1-1-1024-1734624013-1-2-8wmjv: found +CockroachDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-cockroachdb-1-1-1024-1734646253-1-2-2vzvp: found - waiting 30s - : done CockroachDB-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -118,24 +126,24 @@ CockroachDB-1-1-1024 : has running benchmarks done CockroachDB-1-1-1024 : has running benchmarks CockroachDB-1-1-1024-2 : showing benchmarker times -CockroachDB-1-1-1024-2 : benchmarker timespan (start to end single container [s]) = 315 -CockroachDB-1-1-1024-2 : benchmarker times (start/end per pod and container) = [(1734624714, 1734625018), (1734624714, 1734625019)] -CockroachDB-1-1-1024-2 : found and updated times {'benchmarker': [(1734624714, 1734625018), (1734624714, 1734625019)]} +CockroachDB-1-1-1024-2 : benchmarker timespan (start to end single container [s]) = 314 +CockroachDB-1-1-1024-2 : benchmarker times (start/end per pod and container) = [(1734647075, 1734647379), (1734647075, 1734647379)] +CockroachDB-1-1-1024-2 : found and updated times {'benchmarker': [(1734647075, 1734647379), (1734647075, 1734647379)]} done CockroachDB-1-1-1024 : can be stopped -Experiment ends at 2024-12-19 17:17:39.472259 (4309860.879392857): 1046.6104651754722s total +Experiment ends at 2024-12-19 23:30:19.076842 (4332220.48397571): 1165.7185365445912s total Experiment : downloading partial results Experiment : uploading full results Experiment : uploading workload file -Result workflow not complete +Result workflow complete ## Show Summary ### Workload Benchbase Workload SF=16 (warehouses for TPC-C) Type: benchbase - Duration: 1047s - Code: 1734624013 + Duration: 1166s + Code: 1734646253 Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. @@ -156,7 +164,7 @@ CockroachDB-1-1-1024-1 uses docker image cockroachdb/cockroach:v24.2.4 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:249214372 + disk:249215596 requests_cpu:4 requests_memory:16Gi worker 0 @@ -164,26 +172,26 @@ CockroachDB-1-1-1024-1 uses docker image cockroachdb/cockroach:v24.2.4 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:439691956 - datadisk:116318604 + disk:461657896 + datadisk:116314488 volume_size:1000G volume_used:109G worker 1 - RAM:1081751019520 - Cores:128 - host:5.15.0-126-generic - node:cl-worker29 - disk:153273172 - datadisk:116107360 - volume_size:1000G - volume_used:109G - worker 2 RAM:1081965555712 Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:584549888 - datadisk:116107116 + disk:635102812 + datadisk:116104180 + volume_size:1000G + volume_used:109G + worker 2 + RAM:540587499520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:123840188 + datadisk:116091372 volume_size:1000G volume_used:109G CockroachDB-1-1-1024-2 uses docker image cockroachdb/cockroach:v24.2.4 @@ -191,7 +199,7 @@ CockroachDB-1-1-1024-2 uses docker image cockroachdb/cockroach:v24.2.4 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:249214372 + disk:249215600 requests_cpu:4 requests_memory:16Gi worker 0 @@ -199,48 +207,48 @@ CockroachDB-1-1-1024-2 uses docker image cockroachdb/cockroach:v24.2.4 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:439986660 - datadisk:116611832 + disk:461867536 + datadisk:116522308 volume_size:1000G volume_used:109G worker 1 - RAM:1081751019520 - Cores:128 - host:5.15.0-126-generic - node:cl-worker29 - disk:153570092 - datadisk:116404276 - volume_size:1000G - volume_used:109G - worker 2 RAM:1081965555712 Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:584845212 - datadisk:116401204 + disk:635488320 + datadisk:116308436 + volume_size:1000G + volume_used:109G + worker 2 + RAM:540587499520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:124062476 + datadisk:116312956 volume_size:1000G volume_used:109G ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -CockroachDB-1-1-1024-1 1 16 16384 1 300.0 481.72 84008.0 33199.0 -CockroachDB-1-1-1024-2 1 16 16384 2 300.0 448.16 95345.0 35721.0 +CockroachDB-1-1-1024-1 1 16 16384 1 300.0 312.89 95381.0 51118.0 +CockroachDB-1-1-1024-2 1 16 16384 2 300.0 241.61 142861.0 66206.0 Warehouses: 16 ### Workflow #### Actual -DBMS CockroachDB-1-1-1024 - Pods [[2, 1]] +DBMS CockroachDB-1-1-1024 - Pods [[1, 2]] #### Planned DBMS CockroachDB-1-1-1024 - Pods [[1, 2]] ### Loading time_load terminals pods Imported warehouses [1/h] -CockroachDB-1-1-1024-1 142.0 1.0 1.0 405.633803 -CockroachDB-1-1-1024-2 142.0 1.0 2.0 405.633803 +CockroachDB-1-1-1024-1 267.0 1.0 1.0 215.730337 +CockroachDB-1-1-1024-2 267.0 1.0 2.0 215.730337 ### Tests TEST passed: Throughput (requests/second) contains no 0 or NaN diff --git a/logs_tests/doc_benchbase_cockroachdb_1_summary.txt b/logs_tests/doc_benchbase_cockroachdb_1_summary.txt index 592927bf..a85ba716 100644 --- a/logs_tests/doc_benchbase_cockroachdb_1_summary.txt +++ b/logs_tests/doc_benchbase_cockroachdb_1_summary.txt @@ -3,8 +3,8 @@ ### Workload Benchbase Workload SF=16 (warehouses for TPC-C) Type: benchbase - Duration: 1047s - Code: 1734624013 + Duration: 1166s + Code: 1734646253 Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. @@ -25,7 +25,7 @@ CockroachDB-1-1-1024-1 uses docker image cockroachdb/cockroach:v24.2.4 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:249214372 + disk:249215596 requests_cpu:4 requests_memory:16Gi worker 0 @@ -33,26 +33,26 @@ CockroachDB-1-1-1024-1 uses docker image cockroachdb/cockroach:v24.2.4 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:439691956 - datadisk:116318604 + disk:461657896 + datadisk:116314488 volume_size:1000G volume_used:109G worker 1 - RAM:1081751019520 - Cores:128 - host:5.15.0-126-generic - node:cl-worker29 - disk:153273172 - datadisk:116107360 - volume_size:1000G - volume_used:109G - worker 2 RAM:1081965555712 Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:584549888 - datadisk:116107116 + disk:635102812 + datadisk:116104180 + volume_size:1000G + volume_used:109G + worker 2 + RAM:540587499520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:123840188 + datadisk:116091372 volume_size:1000G volume_used:109G CockroachDB-1-1-1024-2 uses docker image cockroachdb/cockroach:v24.2.4 @@ -60,7 +60,7 @@ CockroachDB-1-1-1024-2 uses docker image cockroachdb/cockroach:v24.2.4 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:249214372 + disk:249215600 requests_cpu:4 requests_memory:16Gi worker 0 @@ -68,48 +68,48 @@ CockroachDB-1-1-1024-2 uses docker image cockroachdb/cockroach:v24.2.4 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:439986660 - datadisk:116611832 + disk:461867536 + datadisk:116522308 volume_size:1000G volume_used:109G worker 1 - RAM:1081751019520 - Cores:128 - host:5.15.0-126-generic - node:cl-worker29 - disk:153570092 - datadisk:116404276 - volume_size:1000G - volume_used:109G - worker 2 RAM:1081965555712 Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:584845212 - datadisk:116401204 + disk:635488320 + datadisk:116308436 + volume_size:1000G + volume_used:109G + worker 2 + RAM:540587499520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:124062476 + datadisk:116312956 volume_size:1000G volume_used:109G ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -CockroachDB-1-1-1024-1 1 16 16384 1 300.0 481.72 84008.0 33199.0 -CockroachDB-1-1-1024-2 1 16 16384 2 300.0 448.16 95345.0 35721.0 +CockroachDB-1-1-1024-1 1 16 16384 1 300.0 312.89 95381.0 51118.0 +CockroachDB-1-1-1024-2 1 16 16384 2 300.0 241.61 142861.0 66206.0 Warehouses: 16 ### Workflow #### Actual -DBMS CockroachDB-1-1-1024 - Pods [[2, 1]] +DBMS CockroachDB-1-1-1024 - Pods [[1, 2]] #### Planned DBMS CockroachDB-1-1-1024 - Pods [[1, 2]] ### Loading time_load terminals pods Imported warehouses [1/h] -CockroachDB-1-1-1024-1 142.0 1.0 1.0 405.633803 -CockroachDB-1-1-1024-2 142.0 1.0 2.0 405.633803 +CockroachDB-1-1-1024-1 267.0 1.0 1.0 215.730337 +CockroachDB-1-1-1024-2 267.0 1.0 2.0 215.730337 ### Tests TEST passed: Throughput (requests/second) contains no 0 or NaN diff --git a/logs_tests/doc_benchbase_cockroachdb_2.log b/logs_tests/doc_benchbase_cockroachdb_2.log index 13f3ae9b..1cdebe1e 100644 --- a/logs_tests/doc_benchbase_cockroachdb_2.log +++ b/logs_tests/doc_benchbase_cockroachdb_2.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1730374413 -Experiment : starts at 2024-10-31 12:33:33.078575 (59214.485708551) +Experiment : has code 1734647454 +Experiment : starts at 2024-12-19 23:30:53.703666 (4332255.110801013) Experiment : This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. Benchmark is 'tpcc'. Scaling factor (e.g., number of warehouses) is 128. Benchmarking runs for 60 minutes. Target is based on multiples of '1024'. Factors for benchmarking are [16]. @@ -17,26 +17,28 @@ Benchmarking is tested with [64] threads, split into [1, 2, 4, 8] pods. Benchmarking is run as [1] times the number of benchmarking pods. Experiment is run once. Cluster monitoring : is running -Warning: spec.template.spec.containers[1].env[4]: hides previous definition of "BEXHOMA_WORKER_LIST" -Warning: spec.template.spec.containers[0].env[2]: hides previous definition of "BEXHOMA_WORKER_FIRST" +Warning: spec.template.spec.containers[1].env[4]: hides previous definition of "BEXHOMA_WORKER_LIST", which may be dropped when using apply +Warning: spec.template.spec.containers[0].env[2]: hides previous definition of "BEXHOMA_WORKER_FIRST", which may be dropped when using apply done CockroachDB-1-1-1024 : will start now - waiting 30s - : done CockroachDB-1-1-1024 : is not loaded yet -CockroachDB-1-1-1024 : will start loading but not before 2024-10-31 11:36:33 (that is in 120 secs) +CockroachDB-1-1-1024 : will start loading but not before 2024-12-19 22:33:54 (that is in 120 secs) - waiting 30s - : done CockroachDB-1-1-1024 : is not loaded yet -CockroachDB-1-1-1024 : will start loading but not before 2024-10-31 11:36:33 +CockroachDB-1-1-1024 : will start loading but not before 2024-12-19 22:33:54 - waiting 30s - : done CockroachDB-1-1-1024 : is not loaded yet -CockroachDB-1-1-1024 : will start loading but not before 2024-10-31 11:36:33 +CockroachDB-1-1-1024 : will start loading but not before 2024-12-19 22:33:54 - waiting 30s - : done CockroachDB-1-1-1024 : is not loaded yet -CockroachDB-1-1-1024 : will start loading but not before 2024-10-31 11:36:33 +CockroachDB-1-1-1024 : will start loading but not before 2024-12-19 22:33:54 - waiting 30s - : done CockroachDB-1-1-1024 : is not loaded yet done -CockroachDB-1-1-1024 Workers 1 of 1 +Worker pods found: ['bexhoma-worker-cockroachdb-1-1-1024-1734647454-0', 'bexhoma-worker-cockroachdb-1-1-1024-1734647454-1', 'bexhoma-worker-cockroachdb-1-1-1024-1734647454-2'] +CockroachDB-1-1-1024 Workers 3 of 3 +Worker pods found: ['bexhoma-worker-cockroachdb-1-1-1024-1734647454-0', 'bexhoma-worker-cockroachdb-1-1-1024-1734647454-1', 'bexhoma-worker-cockroachdb-1-1-1024-1734647454-2'] CockroachDB-1-1-1024 : start asynch loading scripts of type loaded CockroachDB-1-1-1024 : is loading - waiting 30s - : done @@ -73,15 +75,53 @@ CockroachDB-1-1-1024 : is loading CockroachDB-1-1-1024 : is loading - waiting 30s - : done CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading +- waiting 30s - : done +CockroachDB-1-1-1024 : is loading done CockroachDB-1-1-1024 : showing loader times -CockroachDB-1-1-1024 : loader timespan (first to last [s]) = 531 +CockroachDB-1-1-1024 : loader timespan (first to last [s]) = 1036 CockroachDB-1-1-1024 : benchmarks done 0 of 1. This will be client 1 CockroachDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '1', 'SF': '128', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'cockroachdb', 'BEXHOMA_DATABASE': 'defaultdb', 'BENCHBASE_TARGET': 16384, 'BENCHBASE_TERMINALS': 64, 'BENCHBASE_TIME': 3600, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_USER': 'root', 'BEXHOMA_PASSWORD': ''} CockroachDB-1-1-1024-1 : start benchmarking -CockroachDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1730374413 +Worker pods found: ['bexhoma-worker-cockroachdb-1-1-1024-1734647454-0', 'bexhoma-worker-cockroachdb-1-1-1024-1734647454-1', 'bexhoma-worker-cockroachdb-1-1-1024-1734647454-2'] +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734647454-0 +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734647454-1 +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734647454-2 +CockroachDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734647454 - waiting 10s - : done -CockroachDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-cockroachdb-1-1-1024-1730374413-1-1-pzq2d: found +CockroachDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-cockroachdb-1-1-1024-1734647454-1-1-lbt66: found - waiting 30s - : done CockroachDB-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -323,16 +363,20 @@ CockroachDB-1-1-1024 : has running benchmarks done CockroachDB-1-1-1024 : has running benchmarks CockroachDB-1-1-1024-1 : showing benchmarker times -CockroachDB-1-1-1024-1 : benchmarker timespan (start to end single container [s]) = 3625 -CockroachDB-1-1-1024-1 : benchmarker times (start/end per pod and container) = [(1730375150, 1730378765)] -CockroachDB-1-1-1024-1 : found and updated times {'benchmarker': [(1730375150, 1730378765)]} +CockroachDB-1-1-1024-1 : benchmarker timespan (start to end single container [s]) = 3637 +CockroachDB-1-1-1024-1 : benchmarker times (start/end per pod and container) = [(1734648709, 1734652333)] +CockroachDB-1-1-1024-1 : found and updated times {'benchmarker': [(1734648709, 1734652333)]} done CockroachDB-1-1-1024 : benchmarks done 0 of 1. This will be client 2 CockroachDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '2', 'SF': '128', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'cockroachdb', 'BEXHOMA_DATABASE': 'defaultdb', 'BENCHBASE_TARGET': 8192, 'BENCHBASE_TERMINALS': 32, 'BENCHBASE_TIME': 3600, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_USER': 'root', 'BEXHOMA_PASSWORD': ''} CockroachDB-1-1-1024-2 : start benchmarking -CockroachDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1730374413 +Worker pods found: ['bexhoma-worker-cockroachdb-1-1-1024-1734647454-0', 'bexhoma-worker-cockroachdb-1-1-1024-1734647454-1', 'bexhoma-worker-cockroachdb-1-1-1024-1734647454-2'] +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734647454-0 +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734647454-1 +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734647454-2 +CockroachDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734647454 - waiting 10s - : done -CockroachDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-cockroachdb-1-1-1024-1730374413-1-2-h8bjm: found +CockroachDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-cockroachdb-1-1-1024-1734647454-1-2-tsf9s: found - waiting 30s - : done CockroachDB-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -574,18 +618,20 @@ CockroachDB-1-1-1024 : has running benchmarks done CockroachDB-1-1-1024 : has running benchmarks CockroachDB-1-1-1024-2 : showing benchmarker times -CockroachDB-1-1-1024-2 : benchmarker timespan (start to end single container [s]) = 3627 -CockroachDB-1-1-1024-2 : benchmarker times (start/end per pod and container) = [(1730378811, 1730382421), (1730378811, 1730382421)] -CockroachDB-1-1-1024-2 : found and updated times {'benchmarker': [(1730378811, 1730382421), (1730378811, 1730382421)]} +CockroachDB-1-1-1024-2 : benchmarker timespan (start to end single container [s]) = 3642 +CockroachDB-1-1-1024-2 : benchmarker times (start/end per pod and container) = [(1734652384, 1734655998), (1734652384, 1734655998)] +CockroachDB-1-1-1024-2 : found and updated times {'benchmarker': [(1734652384, 1734655998), (1734652384, 1734655998)]} done CockroachDB-1-1-1024 : benchmarks done 0 of 1. This will be client 3 CockroachDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '4', 'SF': '128', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'cockroachdb', 'BEXHOMA_DATABASE': 'defaultdb', 'BENCHBASE_TARGET': 4096, 'BENCHBASE_TERMINALS': 16, 'BENCHBASE_TIME': 3600, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_USER': 'root', 'BEXHOMA_PASSWORD': ''} CockroachDB-1-1-1024-3 : start benchmarking -CockroachDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1730374413 +Worker pods found: ['bexhoma-worker-cockroachdb-1-1-1024-1734647454-0', 'bexhoma-worker-cockroachdb-1-1-1024-1734647454-1', 'bexhoma-worker-cockroachdb-1-1-1024-1734647454-2'] +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734647454-0 +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734647454-1 +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734647454-2 +CockroachDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734647454 - waiting 10s - : done -CockroachDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-cockroachdb-1-1-1024-1730374413-1-3-c4rm7: found -- waiting 30s - : done -CockroachDB-1-1-1024 : has running benchmarks +CockroachDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-cockroachdb-1-1-1024-1734647454-1-3-l8x4n: found - waiting 30s - : done CockroachDB-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -825,18 +871,20 @@ CockroachDB-1-1-1024 : has running benchmarks done CockroachDB-1-1-1024 : has running benchmarks CockroachDB-1-1-1024-3 : showing benchmarker times -CockroachDB-1-1-1024-3 : benchmarker timespan (start to end single container [s]) = 3634 -CockroachDB-1-1-1024-3 : benchmarker times (start/end per pod and container) = [(1730382478, 1730386086), (1730382478, 1730386086), (1730382478, 1730386085), (1730382479, 1730386086)] -CockroachDB-1-1-1024-3 : found and updated times {'benchmarker': [(1730382478, 1730386086), (1730382478, 1730386086), (1730382478, 1730386085), (1730382479, 1730386086)]} +CockroachDB-1-1-1024-3 : benchmarker timespan (start to end single container [s]) = 3625 +CockroachDB-1-1-1024-3 : benchmarker times (start/end per pod and container) = [(1734656067, 1734659676), (1734656066, 1734659676), (1734656067, 1734659677), (1734656066, 1734659676)] +CockroachDB-1-1-1024-3 : found and updated times {'benchmarker': [(1734656067, 1734659676), (1734656066, 1734659676), (1734656067, 1734659677), (1734656066, 1734659676)]} done CockroachDB-1-1-1024 : benchmarks done 0 of 1. This will be client 4 CockroachDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '8', 'SF': '128', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'cockroachdb', 'BEXHOMA_DATABASE': 'defaultdb', 'BENCHBASE_TARGET': 2048, 'BENCHBASE_TERMINALS': 8, 'BENCHBASE_TIME': 3600, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_USER': 'root', 'BEXHOMA_PASSWORD': ''} CockroachDB-1-1-1024-4 : start benchmarking -CockroachDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1730374413 +Worker pods found: ['bexhoma-worker-cockroachdb-1-1-1024-1734647454-0', 'bexhoma-worker-cockroachdb-1-1-1024-1734647454-1', 'bexhoma-worker-cockroachdb-1-1-1024-1734647454-2'] +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734647454-0 +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734647454-1 +CockroachDB-1-1-1024 : distributed system - get host info for worker bexhoma-worker-cockroachdb-1-1-1024-1734647454-2 +CockroachDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734647454 - waiting 10s - : done -CockroachDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-cockroachdb-1-1-1024-1730374413-1-4-6n5f2: found -- waiting 30s - : done -CockroachDB-1-1-1024 : has running benchmarks +CockroachDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-cockroachdb-1-1-1024-1734647454-1-4-8n254: found - waiting 30s - : done CockroachDB-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -1076,35 +1124,15 @@ CockroachDB-1-1-1024 : has running benchmarks done CockroachDB-1-1-1024 : has running benchmarks CockroachDB-1-1-1024-4 : showing benchmarker times -CockroachDB-1-1-1024-4 : benchmarker timespan (start to end single container [s]) = 3653 -CockroachDB-1-1-1024-4 : benchmarker times (start/end per pod and container) = [(1730386153, 1730389760), (1730386154, 1730389760), (1730386153, 1730389760), (1730386153, 1730389760), (1730386153, 1730389760), (1730386154, 1730389760), (1730386153, 1730389760), (1730386153, 1730389760)] -CockroachDB-1-1-1024-4 : found and updated times {'benchmarker': [(1730386153, 1730389760), (1730386154, 1730389760), (1730386153, 1730389760), (1730386153, 1730389760), (1730386153, 1730389760), (1730386154, 1730389760), (1730386153, 1730389760), (1730386153, 1730389760)]} +CockroachDB-1-1-1024-4 : benchmarker timespan (start to end single container [s]) = 3652 +CockroachDB-1-1-1024-4 : benchmarker times (start/end per pod and container) = [(1734659733, 1734663341), (1734659732, 1734663341), (1734659732, 1734663341), (1734659733, 1734663341), (1734659732, 1734663340), (1734659733, 1734663340), (1734659732, 1734663340), (1734659733, 1734663341)] +CockroachDB-1-1-1024-4 : found and updated times {'benchmarker': [(1734659733, 1734663341), (1734659732, 1734663341), (1734659732, 1734663341), (1734659733, 1734663341), (1734659732, 1734663340), (1734659733, 1734663340), (1734659732, 1734663340), (1734659733, 1734663341)]} done CockroachDB-1-1-1024 : can be stopped -Experiment ends at 2024-10-31 16:50:30.565068 (74631.97220079): 15417.486492238997s total +Experiment ends at 2024-12-20 03:56:50.447696 (4348211.854830527): 15956.74402951356s total Experiment : downloading partial results Experiment : uploading full results Experiment : uploading workload file -Benchmarking connection ... Latency Distribution.Average Latency (microseconds) -connection_pod ... -CockroachDB-1-1-1024-4-1 CockroachDB-1-1-1024-4 ... 118664 -CockroachDB-1-1-1024-3-1 CockroachDB-1-1-1024-3 ... 106931 -CockroachDB-1-1-1024-4-2 CockroachDB-1-1-1024-4 ... 119281 -CockroachDB-1-1-1024-4-3 CockroachDB-1-1-1024-4 ... 118937 -CockroachDB-1-1-1024-4-4 CockroachDB-1-1-1024-4 ... 118812 -CockroachDB-1-1-1024-2-1 CockroachDB-1-1-1024-2 ... 98010 -CockroachDB-1-1-1024-3-2 CockroachDB-1-1-1024-3 ... 106741 -CockroachDB-1-1-1024-4-5 CockroachDB-1-1-1024-4 ... 119215 -CockroachDB-1-1-1024-3-3 CockroachDB-1-1-1024-3 ... 106713 -CockroachDB-1-1-1024-1-1 CockroachDB-1-1-1024-1 ... 91853 -CockroachDB-1-1-1024-4-6 CockroachDB-1-1-1024-4 ... 119222 -CockroachDB-1-1-1024-2-2 CockroachDB-1-1-1024-2 ... 97986 -CockroachDB-1-1-1024-4-7 CockroachDB-1-1-1024-4 ... 118708 -CockroachDB-1-1-1024-3-4 CockroachDB-1-1-1024-3 ... 106869 -CockroachDB-1-1-1024-4-8 CockroachDB-1-1-1024-4 ... 118720 - -[15 rows x 36 columns] -Workflow {'CockroachDB-1-1-1024': [[8, 4, 2, 1]]} Result workflow not complete ## Show Summary @@ -1112,9 +1140,9 @@ Result workflow not complete ### Workload Benchbase Workload SF=128 (warehouses for TPC-C) Type: benchbase - Duration: 15418s - Code: 1730374413 - This includes no queries. Benchbase runs the benchmark + Duration: 15957s + Code: 1734647454 + Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. Benchmark is 'tpcc'. Scaling factor (e.g., number of warehouses) is 128. Benchmarking runs for 60 minutes. Target is based on multiples of '1024'. Factors for benchmarking are [16]. @@ -1130,61 +1158,169 @@ Benchbase Workload SF=128 (warehouses for TPC-C) ### Connections CockroachDB-1-1-1024-1 uses docker image cockroachdb/cockroach:v24.2.4 - RAM:541008605184 + RAM:541008576512 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254912048 + disk:249215616 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:469202612 + datadisk:123845016 + volume_size:1000G + volume_used:109G + worker 1 + RAM:1081751019520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker29 + disk:160789996 + datadisk:123623440 + volume_size:1000G + volume_used:109G + worker 2 + RAM:1081965555712 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:644345576 + datadisk:123626156 + volume_size:1000G + volume_used:109G CockroachDB-1-1-1024-2 uses docker image cockroachdb/cockroach:v24.2.4 - RAM:541008605184 + RAM:541008576512 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254912216 + disk:249215820 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:475115332 + datadisk:129756564 + volume_size:1000G + volume_used:109G + worker 1 + RAM:1081751019520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker29 + disk:166744188 + datadisk:129577516 + volume_size:1000G + volume_used:109G + worker 2 + RAM:1081965555712 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:654361308 + datadisk:129571596 + volume_size:1000G + volume_used:109G CockroachDB-1-1-1024-3 uses docker image cockroachdb/cockroach:v24.2.4 - RAM:541008605184 + RAM:541008576512 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254912388 + disk:249216060 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:478912840 + datadisk:133546860 + volume_size:1000G + volume_used:109G + worker 1 + RAM:1081751019520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker29 + disk:170393272 + datadisk:133226492 + volume_size:1000G + volume_used:109G + worker 2 + RAM:1081965555712 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:661848004 + datadisk:133214428 + volume_size:1000G + volume_used:109G CockroachDB-1-1-1024-4 uses docker image cockroachdb/cockroach:v24.2.4 - RAM:541008605184 + RAM:541008576512 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254912556 + disk:249216408 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:481732908 + datadisk:136364000 + volume_size:1000G + volume_used:109G + worker 1 + RAM:1081751019520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker29 + disk:173351696 + datadisk:136184808 + volume_size:1000G + volume_used:109G + worker 2 + RAM:1081965555712 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:667527460 + datadisk:136153100 + volume_size:1000G + volume_used:109G ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -CockroachDB-1-1-1024-1 1 64 16384 1 3600.0 696.70 233544.0 91853.00 -CockroachDB-1-1-1024-2 1 64 16384 2 3600.0 653.01 247449.0 97998.00 -CockroachDB-1-1-1024-3 1 64 16384 4 3600.0 599.12 258276.0 106813.50 -CockroachDB-1-1-1024-4 1 64 16384 8 3600.0 538.01 262975.0 118944.88 +CockroachDB-1-1-1024-1 1 64 16384 1 3600.0 1097.64 144074.0 58301.00 +CockroachDB-1-1-1024-2 1 64 16384 2 3600.0 1026.91 161894.0 62323.00 +CockroachDB-1-1-1024-3 1 64 16384 4 3600.0 908.92 181035.0 70443.25 +CockroachDB-1-1-1024-4 1 64 16384 8 3600.0 675.46 224333.0 94757.50 Warehouses: 128 ### Workflow #### Actual -DBMS CockroachDB-1-1-1024 - Pods [[8, 4, 2, 1]] +DBMS CockroachDB-1-1-1024 - Pods [[1, 2, 8, 4]] #### Planned DBMS CockroachDB-1-1-1024 - Pods [[1, 2, 4, 8]] ### Loading time_load terminals pods Imported warehouses [1/h] -CockroachDB-1-1-1024-1 531.0 1.0 1.0 867.79661 -CockroachDB-1-1-1024-2 531.0 1.0 2.0 867.79661 -CockroachDB-1-1-1024-3 531.0 1.0 4.0 867.79661 -CockroachDB-1-1-1024-4 531.0 1.0 8.0 867.79661 +CockroachDB-1-1-1024-1 1036.0 1.0 1.0 444.787645 +CockroachDB-1-1-1024-2 1036.0 1.0 2.0 444.787645 +CockroachDB-1-1-1024-3 1036.0 1.0 4.0 444.787645 +CockroachDB-1-1-1024-4 1036.0 1.0 8.0 444.787645 ### Tests TEST passed: Throughput (requests/second) contains no 0 or NaN diff --git a/logs_tests/doc_benchbase_cockroachdb_2_summary.txt b/logs_tests/doc_benchbase_cockroachdb_2_summary.txt index e05d773f..78034a94 100644 --- a/logs_tests/doc_benchbase_cockroachdb_2_summary.txt +++ b/logs_tests/doc_benchbase_cockroachdb_2_summary.txt @@ -3,9 +3,9 @@ ### Workload Benchbase Workload SF=128 (warehouses for TPC-C) Type: benchbase - Duration: 15418s - Code: 1730374413 - This includes no queries. Benchbase runs the benchmark + Duration: 15957s + Code: 1734647454 + Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. Benchmark is 'tpcc'. Scaling factor (e.g., number of warehouses) is 128. Benchmarking runs for 60 minutes. Target is based on multiples of '1024'. Factors for benchmarking are [16]. @@ -21,61 +21,169 @@ Benchbase Workload SF=128 (warehouses for TPC-C) ### Connections CockroachDB-1-1-1024-1 uses docker image cockroachdb/cockroach:v24.2.4 - RAM:541008605184 + RAM:541008576512 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254912048 + disk:249215616 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:469202612 + datadisk:123845016 + volume_size:1000G + volume_used:109G + worker 1 + RAM:1081751019520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker29 + disk:160789996 + datadisk:123623440 + volume_size:1000G + volume_used:109G + worker 2 + RAM:1081965555712 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:644345576 + datadisk:123626156 + volume_size:1000G + volume_used:109G CockroachDB-1-1-1024-2 uses docker image cockroachdb/cockroach:v24.2.4 - RAM:541008605184 + RAM:541008576512 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254912216 + disk:249215820 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:475115332 + datadisk:129756564 + volume_size:1000G + volume_used:109G + worker 1 + RAM:1081751019520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker29 + disk:166744188 + datadisk:129577516 + volume_size:1000G + volume_used:109G + worker 2 + RAM:1081965555712 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:654361308 + datadisk:129571596 + volume_size:1000G + volume_used:109G CockroachDB-1-1-1024-3 uses docker image cockroachdb/cockroach:v24.2.4 - RAM:541008605184 + RAM:541008576512 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254912388 + disk:249216060 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:478912840 + datadisk:133546860 + volume_size:1000G + volume_used:109G + worker 1 + RAM:1081751019520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker29 + disk:170393272 + datadisk:133226492 + volume_size:1000G + volume_used:109G + worker 2 + RAM:1081965555712 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:661848004 + datadisk:133214428 + volume_size:1000G + volume_used:109G CockroachDB-1-1-1024-4 uses docker image cockroachdb/cockroach:v24.2.4 - RAM:541008605184 + RAM:541008576512 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254912556 + disk:249216408 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:481732908 + datadisk:136364000 + volume_size:1000G + volume_used:109G + worker 1 + RAM:1081751019520 + Cores:128 + host:5.15.0-126-generic + node:cl-worker29 + disk:173351696 + datadisk:136184808 + volume_size:1000G + volume_used:109G + worker 2 + RAM:1081965555712 + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:667527460 + datadisk:136153100 + volume_size:1000G + volume_used:109G ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -CockroachDB-1-1-1024-1 1 64 16384 1 3600.0 696.70 233544.0 91853.00 -CockroachDB-1-1-1024-2 1 64 16384 2 3600.0 653.01 247449.0 97998.00 -CockroachDB-1-1-1024-3 1 64 16384 4 3600.0 599.12 258276.0 106813.50 -CockroachDB-1-1-1024-4 1 64 16384 8 3600.0 538.01 262975.0 118944.88 +CockroachDB-1-1-1024-1 1 64 16384 1 3600.0 1097.64 144074.0 58301.00 +CockroachDB-1-1-1024-2 1 64 16384 2 3600.0 1026.91 161894.0 62323.00 +CockroachDB-1-1-1024-3 1 64 16384 4 3600.0 908.92 181035.0 70443.25 +CockroachDB-1-1-1024-4 1 64 16384 8 3600.0 675.46 224333.0 94757.50 Warehouses: 128 ### Workflow #### Actual -DBMS CockroachDB-1-1-1024 - Pods [[8, 4, 2, 1]] +DBMS CockroachDB-1-1-1024 - Pods [[1, 2, 8, 4]] #### Planned DBMS CockroachDB-1-1-1024 - Pods [[1, 2, 4, 8]] ### Loading time_load terminals pods Imported warehouses [1/h] -CockroachDB-1-1-1024-1 531.0 1.0 1.0 867.79661 -CockroachDB-1-1-1024-2 531.0 1.0 2.0 867.79661 -CockroachDB-1-1-1024-3 531.0 1.0 4.0 867.79661 -CockroachDB-1-1-1024-4 531.0 1.0 8.0 867.79661 +CockroachDB-1-1-1024-1 1036.0 1.0 1.0 444.787645 +CockroachDB-1-1-1024-2 1036.0 1.0 2.0 444.787645 +CockroachDB-1-1-1024-3 1036.0 1.0 4.0 444.787645 +CockroachDB-1-1-1024-4 1036.0 1.0 8.0 444.787645 ### Tests TEST passed: Throughput (requests/second) contains no 0 or NaN diff --git a/logs_tests/doc_benchbase_databaseservice_1.log b/logs_tests/doc_benchbase_databaseservice_1.log index 51c2eea9..5951b562 100644 --- a/logs_tests/doc_benchbase_databaseservice_1.log +++ b/logs_tests/doc_benchbase_databaseservice_1.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734425508 -Experiment : starts at 2024-12-17 09:51:48.528178 (4110309.935312608) +Experiment : has code 1734664810 +Experiment : starts at 2024-12-20 04:20:09.627530 (4349611.034664394) Experiment : This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. Benchmark is 'tpcc'. Scaling factor (e.g., number of warehouses) is 16. Benchmarking runs for 5 minutes. Target is based on multiples of '1024'. Factors for benchmarking are [16]. @@ -17,8 +17,6 @@ Benchmarking is tested with [16] threads, split into [1, 2] pods. Benchmarking is run as [1] times the number of benchmarking pods. Experiment is run once. Cluster monitoring : is running -- waiting 30s - : done -DatabaseService-1-1-1024 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster done DatabaseService-1-1-1024 : will start now - waiting 30s - : done @@ -28,13 +26,11 @@ DatabaseService-1-1-1024 : waits for health check to succeed - waiting 30s - : done DatabaseService-1-1-1024 : waits for health check to succeed - waiting 30s - : done -DatabaseService-1-1-1024 : waits for health check to succeed -- waiting 30s - : done DatabaseService-1-1-1024 : is not loaded yet -DatabaseService-1-1-1024 : will start loading but not before 2024-12-17 08:56:19 (that is in 60 secs) +DatabaseService-1-1-1024 : will start loading but not before 2024-12-20 03:23:42 (that is in 60 secs) - waiting 30s - : done DatabaseService-1-1-1024 : is not loaded yet -DatabaseService-1-1-1024 : will start loading but not before 2024-12-17 08:56:19 +DatabaseService-1-1-1024 : will start loading but not before 2024-12-20 03:23:42 done DatabaseService-1-1-1024 : is not loaded yet DatabaseService-1-1-1024 : start asynch loading scripts of type loaded @@ -51,14 +47,14 @@ DatabaseService-1-1-1024 : is loading DatabaseService-1-1-1024 : is loading done DatabaseService-1-1-1024 : showing loader times -DatabaseService-1-1-1024 : loader timespan (first to last [s]) = 149 +DatabaseService-1-1-1024 : loader timespan (first to last [s]) = 150 DatabaseService-1-1-1024 : benchmarks done 0 of 1. This will be client 1 DatabaseService-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '1', 'SF': '16', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'postgres', 'BEXHOMA_DATABASE': 'postgres', 'BEXHOMA_HOST': 'bexhoma-service.perdelt.svc.cluster.local', 'BENCHBASE_TARGET': 16384, 'BENCHBASE_TERMINALS': 16, 'BENCHBASE_TIME': 300, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED'} DatabaseService-1-1-1024-1 : start benchmarking Worker pods found: [] -DatabaseService-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734425508 +DatabaseService-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734664810 - waiting 10s - : done -DatabaseService-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-1-1-1024-1734425508-1-pm9s9: found +DatabaseService-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-1-1-1024-1734664810-1-qwfcv: found - waiting 30s - : done DatabaseService-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -88,9 +84,9 @@ DatabaseService-1-1-1024 : benchmarks done 0 of 1. This will be client 2 DatabaseService-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '2', 'SF': '16', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'postgres', 'BEXHOMA_DATABASE': 'postgres', 'BEXHOMA_HOST': 'bexhoma-service.perdelt.svc.cluster.local', 'BENCHBASE_TARGET': 8192, 'BENCHBASE_TERMINALS': 8, 'BENCHBASE_TIME': 300, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED'} DatabaseService-1-1-1024-2 : start benchmarking Worker pods found: [] -DatabaseService-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734425508 +DatabaseService-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734664810 - waiting 10s - : done -DatabaseService-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-1-1-1024-1734425508-1-9c47d: found +DatabaseService-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-1-1-1024-1734664810-1-dbz5k: found - waiting 30s - : done DatabaseService-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -112,24 +108,24 @@ DatabaseService-1-1-1024 : has running benchmarks done DatabaseService-1-1-1024 : has running benchmarks DatabaseService-1-1-1024-2 : showing benchmarker times -DatabaseService-1-1-1024-2 : benchmarker timespan (start to end single container [s]) = 313 +DatabaseService-1-1-1024-2 : benchmarker timespan (start to end single container [s]) = 312 DatabaseService-1-1-1024-2 : benchmarker times (start/end per pod and container) = [] DatabaseService-1-1-1024-2 : found and updated times {'benchmarker': []} done DatabaseService-1-1-1024 : can be stopped -Experiment ends at 2024-12-17 10:10:55.597707 (4111457.004836019): 1147.0695234113373s total +Experiment ends at 2024-12-20 04:38:19.697126 (4350701.104259551): 1090.0695951562375s total Experiment : downloading partial results Experiment : uploading full results Experiment : uploading workload file -Result workflow not complete +Result workflow complete ## Show Summary ### Workload Benchbase Workload SF=16 (warehouses for TPC-C) Type: benchbase - Duration: 1148s - Code: 1734425508 + Duration: 1091s + Code: 1734664810 Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. @@ -151,7 +147,7 @@ DatabaseService-1-1-1024-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005104 + disk:249256016 datadisk:39348 requests_cpu:4 requests_memory:16Gi @@ -161,30 +157,30 @@ DatabaseService-1-1-1024-2 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005108 + disk:249256020 datadisk:39348 requests_cpu:4 requests_memory:16Gi ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -DatabaseService-1-1-1024-1 1 16 16384 1 300.0 1922.22 18733.0 8317.0 -DatabaseService-1-1-1024-2 1 16 16384 2 300.0 1742.44 21860.0 9174.5 +DatabaseService-1-1-1024-1 1 16 16384 1 300.0 1873.16 19246.0 8535.0 +DatabaseService-1-1-1024-2 1 16 16384 2 300.0 1820.81 21236.0 8782.5 Warehouses: 16 ### Workflow #### Actual -DBMS DatabaseService-1-1-1024 - Pods [[2, 1]] +DBMS DatabaseService-1-1-1024 - Pods [[1, 2]] #### Planned DBMS DatabaseService-1-1-1024 - Pods [[1, 2]] ### Loading time_load terminals pods Imported warehouses [1/h] -DatabaseService-1-1-1024-1 149.0 1.0 1.0 386.577181 -DatabaseService-1-1-1024-2 149.0 1.0 2.0 386.577181 +DatabaseService-1-1-1024-1 150.0 1.0 1.0 384.0 +DatabaseService-1-1-1024-2 150.0 1.0 2.0 384.0 ### Tests TEST passed: Throughput (requests/second) contains no 0 or NaN diff --git a/logs_tests/doc_benchbase_databaseservice_1_summary.txt b/logs_tests/doc_benchbase_databaseservice_1_summary.txt index 6b800eb1..49f3333c 100644 --- a/logs_tests/doc_benchbase_databaseservice_1_summary.txt +++ b/logs_tests/doc_benchbase_databaseservice_1_summary.txt @@ -3,8 +3,8 @@ ### Workload Benchbase Workload SF=16 (warehouses for TPC-C) Type: benchbase - Duration: 1148s - Code: 1734425508 + Duration: 1091s + Code: 1734664810 Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. @@ -26,7 +26,7 @@ DatabaseService-1-1-1024-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005104 + disk:249256016 datadisk:39348 requests_cpu:4 requests_memory:16Gi @@ -36,30 +36,30 @@ DatabaseService-1-1-1024-2 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005108 + disk:249256020 datadisk:39348 requests_cpu:4 requests_memory:16Gi ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -DatabaseService-1-1-1024-1 1 16 16384 1 300.0 1922.22 18733.0 8317.0 -DatabaseService-1-1-1024-2 1 16 16384 2 300.0 1742.44 21860.0 9174.5 +DatabaseService-1-1-1024-1 1 16 16384 1 300.0 1873.16 19246.0 8535.0 +DatabaseService-1-1-1024-2 1 16 16384 2 300.0 1820.81 21236.0 8782.5 Warehouses: 16 ### Workflow #### Actual -DBMS DatabaseService-1-1-1024 - Pods [[2, 1]] +DBMS DatabaseService-1-1-1024 - Pods [[1, 2]] #### Planned DBMS DatabaseService-1-1-1024 - Pods [[1, 2]] ### Loading time_load terminals pods Imported warehouses [1/h] -DatabaseService-1-1-1024-1 149.0 1.0 1.0 386.577181 -DatabaseService-1-1-1024-2 149.0 1.0 2.0 386.577181 +DatabaseService-1-1-1024-1 150.0 1.0 1.0 384.0 +DatabaseService-1-1-1024-2 150.0 1.0 2.0 384.0 ### Tests TEST passed: Throughput (requests/second) contains no 0 or NaN diff --git a/logs_tests/doc_benchbase_databaseservice_2.log b/logs_tests/doc_benchbase_databaseservice_2.log index 16c16e52..dde3c540 100644 --- a/logs_tests/doc_benchbase_databaseservice_2.log +++ b/logs_tests/doc_benchbase_databaseservice_2.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734426709 -Experiment : starts at 2024-12-17 10:11:48.796934 (4111510.204068016) +Experiment : has code 1734665950 +Experiment : starts at 2024-12-20 04:39:09.834487 (4350751.241621306) Experiment : This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. Benchmark is 'tpcc'. Scaling factor (e.g., number of warehouses) is 16. Benchmarking runs for 5 minutes. Target is based on multiples of '1024'. Factors for benchmarking are [16]. @@ -21,17 +21,17 @@ Cluster monitoring : is running done DatabaseService-1-1-1024 : will start now - waiting 30s - : done -DatabaseService-1-1-1024 : will start benchmarking but not before 2024-12-17 09:13:49 (that is in 60 secs) +DatabaseService-1-1-1024 : will start benchmarking but not before 2024-12-20 03:41:10 (that is in 60 secs) - waiting 30s - : done -DatabaseService-1-1-1024 : will start benchmarking but not before 2024-12-17 09:13:49 +DatabaseService-1-1-1024 : will start benchmarking but not before 2024-12-20 03:41:10 done DatabaseService-1-1-1024 : benchmarks done 0 of 1. This will be client 1 DatabaseService-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '1', 'SF': '16', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'postgres', 'BEXHOMA_DATABASE': 'postgres', 'BEXHOMA_HOST': 'bexhoma-service.perdelt.svc.cluster.local', 'BENCHBASE_TARGET': 16384, 'BENCHBASE_TERMINALS': 16, 'BENCHBASE_TIME': 300, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED'} DatabaseService-1-1-1024-1 : start benchmarking Worker pods found: [] -DatabaseService-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734426709 +DatabaseService-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734665950 - waiting 10s - : done -DatabaseService-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-1-1-1024-1734426709-1-gh94w: found +DatabaseService-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-1-1-1024-1734665950-1-ckwjp: found - waiting 30s - : done DatabaseService-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -53,7 +53,7 @@ DatabaseService-1-1-1024 : has running benchmarks done DatabaseService-1-1-1024 : has running benchmarks DatabaseService-1-1-1024-1 : showing benchmarker times -DatabaseService-1-1-1024-1 : benchmarker timespan (start to end single container [s]) = 313 +DatabaseService-1-1-1024-1 : benchmarker timespan (start to end single container [s]) = 312 DatabaseService-1-1-1024-1 : benchmarker times (start/end per pod and container) = [] DatabaseService-1-1-1024-1 : found and updated times {'benchmarker': []} done @@ -61,9 +61,9 @@ DatabaseService-1-1-1024 : benchmarks done 0 of 1. This will be client 2 DatabaseService-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '2', 'SF': '16', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'postgres', 'BEXHOMA_DATABASE': 'postgres', 'BEXHOMA_HOST': 'bexhoma-service.perdelt.svc.cluster.local', 'BENCHBASE_TARGET': 8192, 'BENCHBASE_TERMINALS': 8, 'BENCHBASE_TIME': 300, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED'} DatabaseService-1-1-1024-2 : start benchmarking Worker pods found: [] -DatabaseService-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734426709 +DatabaseService-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734665950 - waiting 10s - : done -DatabaseService-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-1-1-1024-1734426709-1-wk2xx: found +DatabaseService-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-1-1-1024-1734665950-1-47jf6: found - waiting 30s - : done DatabaseService-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -85,24 +85,24 @@ DatabaseService-1-1-1024 : has running benchmarks done DatabaseService-1-1-1024 : has running benchmarks DatabaseService-1-1-1024-2 : showing benchmarker times -DatabaseService-1-1-1024-2 : benchmarker timespan (start to end single container [s]) = 315 +DatabaseService-1-1-1024-2 : benchmarker timespan (start to end single container [s]) = 313 DatabaseService-1-1-1024-2 : benchmarker times (start/end per pod and container) = [] DatabaseService-1-1-1024-2 : found and updated times {'benchmarker': []} done DatabaseService-1-1-1024 : can be stopped -Experiment ends at 2024-12-17 10:25:25.716050 (4112327.123183415): 816.9191153990105s total +Experiment ends at 2024-12-20 04:52:43.009699 (4351564.416832483): 813.1752111772075s total Experiment : downloading partial results Experiment : uploading full results Experiment : uploading workload file -Result workflow not complete +Result workflow complete ## Show Summary ### Workload Benchbase Workload SF=16 (warehouses for TPC-C) Type: benchbase - Duration: 817s - Code: 1734426709 + Duration: 814s + Code: 1734665950 Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. @@ -125,7 +125,7 @@ DatabaseService-1-1-1024-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005112 + disk:249256028 datadisk:39348 requests_cpu:4 requests_memory:16Gi @@ -135,22 +135,22 @@ DatabaseService-1-1-1024-2 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005116 + disk:249256028 datadisk:39348 requests_cpu:4 requests_memory:16Gi ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -DatabaseService-1-1-1024-1 1 16 16384 1 300.0 1934.14 18523.0 8265.0 -DatabaseService-1-1-1024-2 1 16 16384 2 300.0 1805.90 21119.0 8852.0 +DatabaseService-1-1-1024-1 1 16 16384 1 300.0 1948.43 18329.0 8206.0 +DatabaseService-1-1-1024-2 1 16 16384 2 300.0 1774.67 21116.0 9008.5 Warehouses: 16 ### Workflow #### Actual -DBMS DatabaseService-1-1-1024 - Pods [[2, 1]] +DBMS DatabaseService-1-1-1024 - Pods [[1, 2]] #### Planned DBMS DatabaseService-1-1-1024 - Pods [[1, 2]] diff --git a/logs_tests/doc_benchbase_databaseservice_2_summary.txt b/logs_tests/doc_benchbase_databaseservice_2_summary.txt index dbb8ff3a..64ebd43e 100644 --- a/logs_tests/doc_benchbase_databaseservice_2_summary.txt +++ b/logs_tests/doc_benchbase_databaseservice_2_summary.txt @@ -3,8 +3,8 @@ ### Workload Benchbase Workload SF=16 (warehouses for TPC-C) Type: benchbase - Duration: 817s - Code: 1734426709 + Duration: 814s + Code: 1734665950 Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. @@ -27,7 +27,7 @@ DatabaseService-1-1-1024-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005112 + disk:249256028 datadisk:39348 requests_cpu:4 requests_memory:16Gi @@ -37,22 +37,22 @@ DatabaseService-1-1-1024-2 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005116 + disk:249256028 datadisk:39348 requests_cpu:4 requests_memory:16Gi ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -DatabaseService-1-1-1024-1 1 16 16384 1 300.0 1934.14 18523.0 8265.0 -DatabaseService-1-1-1024-2 1 16 16384 2 300.0 1805.90 21119.0 8852.0 +DatabaseService-1-1-1024-1 1 16 16384 1 300.0 1948.43 18329.0 8206.0 +DatabaseService-1-1-1024-2 1 16 16384 2 300.0 1774.67 21116.0 9008.5 Warehouses: 16 ### Workflow #### Actual -DBMS DatabaseService-1-1-1024 - Pods [[2, 1]] +DBMS DatabaseService-1-1-1024 - Pods [[1, 2]] #### Planned DBMS DatabaseService-1-1-1024 - Pods [[1, 2]] diff --git a/logs_tests/doc_benchbase_yugabytedb_1.log b/logs_tests/doc_benchbase_yugabytedb_1.log index 6c053ac3..ad3e7b58 100644 --- a/logs_tests/doc_benchbase_yugabytedb_1.log +++ b/logs_tests/doc_benchbase_yugabytedb_1.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734601765 -Experiment : starts at 2024-12-19 10:49:24.690293 (4286566.097427178) +Experiment : has code 1734627587 +Experiment : starts at 2024-12-19 17:59:46.587090 (4312387.994224559) Experiment : This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. Benchmark is 'tpcc'. Scaling factor (e.g., number of warehouses) is 16. Benchmarking runs for 5 minutes. Target is based on multiples of '1024'. Factors for benchmarking are [16]. @@ -21,10 +21,10 @@ done YugabyteDB-1-1-1024 : will start now - waiting 30s - : done YugabyteDB-1-1-1024 : is not loaded yet -YugabyteDB-1-1-1024 : will start loading but not before 2024-12-19 09:51:25 (that is in 60 secs) +YugabyteDB-1-1-1024 : will start loading but not before 2024-12-19 17:01:47 (that is in 60 secs) - waiting 30s - : done YugabyteDB-1-1-1024 : is not loaded yet -YugabyteDB-1-1-1024 : will start loading but not before 2024-12-19 09:51:25 +YugabyteDB-1-1-1024 : will start loading but not before 2024-12-19 17:01:47 done YugabyteDB-1-1-1024 : is not loaded yet YugabyteDB-1-1-1024 : start asynch loading scripts of type loaded @@ -41,18 +41,20 @@ YugabyteDB-1-1-1024 : is loading YugabyteDB-1-1-1024 : is loading - waiting 30s - : done YugabyteDB-1-1-1024 : is loading +- waiting 30s - : done +YugabyteDB-1-1-1024 : is loading done YugabyteDB-1-1-1024 : showing loader times -YugabyteDB-1-1-1024 : loader timespan (first to last [s]) = 204 +YugabyteDB-1-1-1024 : loader timespan (first to last [s]) = 216 YugabyteDB-1-1-1024 : benchmarks done 0 of 1. This will be client 1 YugabyteDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '1', 'SF': '16', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'postgres', 'BEXHOMA_DATABASE': 'yugabyte', 'BENCHBASE_TARGET': 16384, 'BENCHBASE_TERMINALS': 16, 'BENCHBASE_TIME': 300, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_USER': 'yugabyte', 'BEXHOMA_PASSWORD': '', 'BEXHOMA_PORT': 5433} YugabyteDB-1-1-1024-1 : start benchmarking YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-0 YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-1 YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-2 -YugabyteDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734601765 +YugabyteDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734627587 - waiting 10s - : done -YugabyteDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-yugabytedb-1-1-1024-1734601765-1-1-qjs5z: found +YugabyteDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-yugabytedb-1-1-1024-1734627587-1-1-xxtnv: found - waiting 30s - : done YugabyteDB-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -74,9 +76,9 @@ YugabyteDB-1-1-1024 : has running benchmarks done YugabyteDB-1-1-1024 : has running benchmarks YugabyteDB-1-1-1024-1 : showing benchmarker times -YugabyteDB-1-1-1024-1 : benchmarker timespan (start to end single container [s]) = 312 -YugabyteDB-1-1-1024-1 : benchmarker times (start/end per pod and container) = [(1734602106, 1734602411)] -YugabyteDB-1-1-1024-1 : found and updated times {'benchmarker': [(1734602106, 1734602411)]} +YugabyteDB-1-1-1024-1 : benchmarker timespan (start to end single container [s]) = 313 +YugabyteDB-1-1-1024-1 : benchmarker times (start/end per pod and container) = [(1734627958, 1734628262)] +YugabyteDB-1-1-1024-1 : found and updated times {'benchmarker': [(1734627958, 1734628262)]} done YugabyteDB-1-1-1024 : benchmarks done 0 of 1. This will be client 2 YugabyteDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '2', 'SF': '16', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'postgres', 'BEXHOMA_DATABASE': 'yugabyte', 'BENCHBASE_TARGET': 8192, 'BENCHBASE_TERMINALS': 8, 'BENCHBASE_TIME': 300, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_USER': 'yugabyte', 'BEXHOMA_PASSWORD': '', 'BEXHOMA_PORT': 5433} @@ -84,9 +86,9 @@ YugabyteDB-1-1-1024-2 : start benchmarking YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-0 YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-1 YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-2 -YugabyteDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734601765 +YugabyteDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734627587 - waiting 10s - : done -YugabyteDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-yugabytedb-1-1-1024-1734601765-1-2-rpdmw: found +YugabyteDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-yugabytedb-1-1-1024-1734627587-1-2-p4jz7: found - waiting 30s - : done YugabyteDB-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -108,12 +110,12 @@ YugabyteDB-1-1-1024 : has running benchmarks done YugabyteDB-1-1-1024 : has running benchmarks YugabyteDB-1-1-1024-2 : showing benchmarker times -YugabyteDB-1-1-1024-2 : benchmarker timespan (start to end single container [s]) = 313 -YugabyteDB-1-1-1024-2 : benchmarker times (start/end per pod and container) = [(1734602457, 1734602761), (1734602457, 1734602761)] -YugabyteDB-1-1-1024-2 : found and updated times {'benchmarker': [(1734602457, 1734602761), (1734602457, 1734602761)]} +YugabyteDB-1-1-1024-2 : benchmarker timespan (start to end single container [s]) = 314 +YugabyteDB-1-1-1024-2 : benchmarker times (start/end per pod and container) = [(1734628310, 1734628613), (1734628310, 1734628613)] +YugabyteDB-1-1-1024-2 : found and updated times {'benchmarker': [(1734628310, 1734628613), (1734628310, 1734628613)]} done YugabyteDB-1-1-1024 : can be stopped -Experiment ends at 2024-12-19 11:06:38.251655 (4287599.65878731): 1033.5613601319492s total +Experiment ends at 2024-12-19 18:17:33.397073 (4313454.804205057): 1066.8099804986268s total Experiment : downloading partial results Experiment : uploading full results Experiment : uploading workload file @@ -124,8 +126,8 @@ Result workflow not complete ### Workload Benchbase Workload SF=16 (warehouses for TPC-C) Type: benchbase - Duration: 1034s - Code: 1734601765 + Duration: 1067s + Code: 1734627587 Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. @@ -147,7 +149,7 @@ YugabyteDB-1-1-1024-1 uses docker image postgres:15.0 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:248621648 + disk:249254028 datadisk:39428 requests_cpu:4 requests_memory:16Gi @@ -157,28 +159,28 @@ YugabyteDB-1-1-1024-1 uses docker image postgres:15.0 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:443073852 + disk:440540068 worker 1 - RAM:540587499520 - CPU:AMD EPYC 7502 32-Core Processor - Cores:128 - host:5.15.0-126-generic - node:cl-worker22 - disk:124173692 - worker 2 RAM:1081965555712 CPU:AMD EPYC 7742 64-Core Processor Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:560979688 + disk:585609424 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:124275600 YugabyteDB-1-1-1024-2 uses docker image postgres:15.0 RAM:541008576512 CPU:AMD Opteron(tm) Processor 6378 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:248621648 + disk:249254032 datadisk:39428 requests_cpu:4 requests_memory:16Gi @@ -188,26 +190,26 @@ YugabyteDB-1-1-1024-2 uses docker image postgres:15.0 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:443885248 + disk:441361868 worker 1 - RAM:540587499520 - CPU:AMD EPYC 7502 32-Core Processor - Cores:128 - host:5.15.0-126-generic - node:cl-worker22 - disk:124976104 - worker 2 RAM:1081965555712 CPU:AMD EPYC 7742 64-Core Processor Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:561773616 + disk:586458512 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:125118168 ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -YugabyteDB-1-1-1024-1 1 16 16384 1 300.0 364.30 107743.0 43901.0 -YugabyteDB-1-1-1024-2 1 16 16384 2 300.0 381.44 105259.0 41945.5 +YugabyteDB-1-1-1024-1 1 16 16384 1 300.0 403.95 96762.0 39594.0 +YugabyteDB-1-1-1024-2 1 16 16384 2 300.0 362.52 111737.0 44178.0 Warehouses: 16 @@ -221,8 +223,8 @@ DBMS YugabyteDB-1-1-1024 - Pods [[1, 2]] ### Loading time_load terminals pods Imported warehouses [1/h] -YugabyteDB-1-1-1024-1 204.0 1.0 1.0 282.352941 -YugabyteDB-1-1-1024-2 204.0 1.0 2.0 282.352941 +YugabyteDB-1-1-1024-1 216.0 1.0 1.0 266.666667 +YugabyteDB-1-1-1024-2 216.0 1.0 2.0 266.666667 ### Tests TEST passed: Throughput (requests/second) contains no 0 or NaN diff --git a/logs_tests/doc_benchbase_yugabytedb_1_summary.txt b/logs_tests/doc_benchbase_yugabytedb_1_summary.txt index e145292e..b39c9389 100644 --- a/logs_tests/doc_benchbase_yugabytedb_1_summary.txt +++ b/logs_tests/doc_benchbase_yugabytedb_1_summary.txt @@ -3,8 +3,8 @@ ### Workload Benchbase Workload SF=16 (warehouses for TPC-C) Type: benchbase - Duration: 1034s - Code: 1734601765 + Duration: 1067s + Code: 1734627587 Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. @@ -26,7 +26,7 @@ YugabyteDB-1-1-1024-1 uses docker image postgres:15.0 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:248621648 + disk:249254028 datadisk:39428 requests_cpu:4 requests_memory:16Gi @@ -36,28 +36,28 @@ YugabyteDB-1-1-1024-1 uses docker image postgres:15.0 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:443073852 + disk:440540068 worker 1 - RAM:540587499520 - CPU:AMD EPYC 7502 32-Core Processor - Cores:128 - host:5.15.0-126-generic - node:cl-worker22 - disk:124173692 - worker 2 RAM:1081965555712 CPU:AMD EPYC 7742 64-Core Processor Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:560979688 + disk:585609424 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:124275600 YugabyteDB-1-1-1024-2 uses docker image postgres:15.0 RAM:541008576512 CPU:AMD Opteron(tm) Processor 6378 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:248621648 + disk:249254032 datadisk:39428 requests_cpu:4 requests_memory:16Gi @@ -67,26 +67,26 @@ YugabyteDB-1-1-1024-2 uses docker image postgres:15.0 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:443885248 + disk:441361868 worker 1 - RAM:540587499520 - CPU:AMD EPYC 7502 32-Core Processor - Cores:128 - host:5.15.0-126-generic - node:cl-worker22 - disk:124976104 - worker 2 RAM:1081965555712 CPU:AMD EPYC 7742 64-Core Processor Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:561773616 + disk:586458512 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:125118168 ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -YugabyteDB-1-1-1024-1 1 16 16384 1 300.0 364.30 107743.0 43901.0 -YugabyteDB-1-1-1024-2 1 16 16384 2 300.0 381.44 105259.0 41945.5 +YugabyteDB-1-1-1024-1 1 16 16384 1 300.0 403.95 96762.0 39594.0 +YugabyteDB-1-1-1024-2 1 16 16384 2 300.0 362.52 111737.0 44178.0 Warehouses: 16 @@ -100,8 +100,8 @@ DBMS YugabyteDB-1-1-1024 - Pods [[1, 2]] ### Loading time_load terminals pods Imported warehouses [1/h] -YugabyteDB-1-1-1024-1 204.0 1.0 1.0 282.352941 -YugabyteDB-1-1-1024-2 204.0 1.0 2.0 282.352941 +YugabyteDB-1-1-1024-1 216.0 1.0 1.0 266.666667 +YugabyteDB-1-1-1024-2 216.0 1.0 2.0 266.666667 ### Tests TEST passed: Throughput (requests/second) contains no 0 or NaN diff --git a/logs_tests/doc_benchbase_yugabytedb_2.log b/logs_tests/doc_benchbase_yugabytedb_2.log index 36fc774a..4f56527c 100644 --- a/logs_tests/doc_benchbase_yugabytedb_2.log +++ b/logs_tests/doc_benchbase_yugabytedb_2.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1730226312 -Experiment : starts at 2024-10-29 19:25:12.301464 (4962252.425445554) +Experiment : has code 1734628788 +Experiment : starts at 2024-12-19 18:19:48.112043 (4313589.519176751) Experiment : This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. Benchmark is 'tpcc'. Scaling factor (e.g., number of warehouses) is 128. Benchmarking runs for 60 minutes. Target is based on multiples of '1024'. Factors for benchmarking are [16]. @@ -21,10 +21,10 @@ done YugabyteDB-1-1-1024 : will start now - waiting 30s - : done YugabyteDB-1-1-1024 : is not loaded yet -YugabyteDB-1-1-1024 : will start loading but not before 2024-10-29 18:27:12 (that is in 60 secs) +YugabyteDB-1-1-1024 : will start loading but not before 2024-12-19 17:21:49 (that is in 60 secs) - waiting 30s - : done YugabyteDB-1-1-1024 : is not loaded yet -YugabyteDB-1-1-1024 : will start loading but not before 2024-10-29 18:27:12 +YugabyteDB-1-1-1024 : will start loading but not before 2024-12-19 17:21:49 done YugabyteDB-1-1-1024 : is not loaded yet YugabyteDB-1-1-1024 : start asynch loading scripts of type loaded @@ -97,23 +97,18 @@ YugabyteDB-1-1-1024 : is loading YugabyteDB-1-1-1024 : is loading - waiting 30s - : done YugabyteDB-1-1-1024 : is loading -- waiting 30s - : done -YugabyteDB-1-1-1024 : is loading -- waiting 30s - : done -YugabyteDB-1-1-1024 : is loading -- waiting 30s - : done -YugabyteDB-1-1-1024 : is loading -- waiting 30s - : done -YugabyteDB-1-1-1024 : is loading done YugabyteDB-1-1-1024 : showing loader times -YugabyteDB-1-1-1024 : loader timespan (first to last [s]) = 1151 +YugabyteDB-1-1-1024 : loader timespan (first to last [s]) = 1033 YugabyteDB-1-1-1024 : benchmarks done 0 of 1. This will be client 1 -YugabyteDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '1', 'SF': '128', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'postgres', 'BEXHOMA_DATABASE': 'yugabyte', 'BENCHBASE_TARGET': 16384, 'BENCHBASE_TERMINALS': 64, 'BENCHBASE_TIME': 3600, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_PORT': 5433} +YugabyteDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '1', 'SF': '128', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'postgres', 'BEXHOMA_DATABASE': 'yugabyte', 'BENCHBASE_TARGET': 16384, 'BENCHBASE_TERMINALS': 64, 'BENCHBASE_TIME': 3600, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_USER': 'yugabyte', 'BEXHOMA_PASSWORD': '', 'BEXHOMA_PORT': 5433} YugabyteDB-1-1-1024-1 : start benchmarking -YugabyteDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1730226312 +YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-0 +YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-1 +YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-2 +YugabyteDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734628788 - waiting 10s - : done -YugabyteDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-yugabytedb-1-1-1024-1730226312-1-1-dk9sr: found +YugabyteDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-yugabytedb-1-1-1024-1734628788-1-1-ph9rq: found - waiting 30s - : done YugabyteDB-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -355,16 +350,19 @@ YugabyteDB-1-1-1024 : has running benchmarks done YugabyteDB-1-1-1024 : has running benchmarks YugabyteDB-1-1-1024-1 : showing benchmarker times -YugabyteDB-1-1-1024-1 : benchmarker timespan (start to end single container [s]) = 3621 -YugabyteDB-1-1-1024-1 : benchmarker times (start/end per pod and container) = [(1730227613, 1730231226)] -YugabyteDB-1-1-1024-1 : found and updated times {'benchmarker': [(1730227613, 1730231226)]} +YugabyteDB-1-1-1024-1 : benchmarker timespan (start to end single container [s]) = 3641 +YugabyteDB-1-1-1024-1 : benchmarker times (start/end per pod and container) = [(1734629981, 1734633595)] +YugabyteDB-1-1-1024-1 : found and updated times {'benchmarker': [(1734629981, 1734633595)]} done YugabyteDB-1-1-1024 : benchmarks done 0 of 1. This will be client 2 -YugabyteDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '2', 'SF': '128', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'postgres', 'BEXHOMA_DATABASE': 'yugabyte', 'BENCHBASE_TARGET': 8192, 'BENCHBASE_TERMINALS': 32, 'BENCHBASE_TIME': 3600, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_PORT': 5433} +YugabyteDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '2', 'SF': '128', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'postgres', 'BEXHOMA_DATABASE': 'yugabyte', 'BENCHBASE_TARGET': 8192, 'BENCHBASE_TERMINALS': 32, 'BENCHBASE_TIME': 3600, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_USER': 'yugabyte', 'BEXHOMA_PASSWORD': '', 'BEXHOMA_PORT': 5433} YugabyteDB-1-1-1024-2 : start benchmarking -YugabyteDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1730226312 +YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-0 +YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-1 +YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-2 +YugabyteDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734628788 - waiting 10s - : done -YugabyteDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-yugabytedb-1-1-1024-1730226312-1-2-7h7d6: found +YugabyteDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-yugabytedb-1-1-1024-1734628788-1-2-b78gk: found - waiting 30s - : done YugabyteDB-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -605,17 +603,22 @@ YugabyteDB-1-1-1024 : has running benchmarks YugabyteDB-1-1-1024 : has running benchmarks done YugabyteDB-1-1-1024 : has running benchmarks +done +YugabyteDB-1-1-1024 : has running benchmarks YugabyteDB-1-1-1024-2 : showing benchmarker times -YugabyteDB-1-1-1024-2 : benchmarker timespan (start to end single container [s]) = 3623 -YugabyteDB-1-1-1024-2 : benchmarker times (start/end per pod and container) = [(1730231269, 1730234878), (1730231269, 1730234880)] -YugabyteDB-1-1-1024-2 : found and updated times {'benchmarker': [(1730231269, 1730234878), (1730231269, 1730234880)]} +YugabyteDB-1-1-1024-2 : benchmarker timespan (start to end single container [s]) = 3675 +YugabyteDB-1-1-1024-2 : benchmarker times (start/end per pod and container) = [(1734633661, 1734637272), (1734633661, 1734637318)] +YugabyteDB-1-1-1024-2 : found and updated times {'benchmarker': [(1734633661, 1734637272), (1734633661, 1734637318)]} done YugabyteDB-1-1-1024 : benchmarks done 0 of 1. This will be client 3 -YugabyteDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '4', 'SF': '128', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'postgres', 'BEXHOMA_DATABASE': 'yugabyte', 'BENCHBASE_TARGET': 4096, 'BENCHBASE_TERMINALS': 16, 'BENCHBASE_TIME': 3600, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_PORT': 5433} +YugabyteDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '4', 'SF': '128', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'postgres', 'BEXHOMA_DATABASE': 'yugabyte', 'BENCHBASE_TARGET': 4096, 'BENCHBASE_TERMINALS': 16, 'BENCHBASE_TIME': 3600, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_USER': 'yugabyte', 'BEXHOMA_PASSWORD': '', 'BEXHOMA_PORT': 5433} YugabyteDB-1-1-1024-3 : start benchmarking -YugabyteDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1730226312 +YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-0 +YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-1 +YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-2 +YugabyteDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734628788 - waiting 10s - : done -YugabyteDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-yugabytedb-1-1-1024-1730226312-1-3-5j5rn: found +YugabyteDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-yugabytedb-1-1-1024-1734628788-1-3-pcp4c: found - waiting 30s - : done YugabyteDB-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -852,21 +855,28 @@ YugabyteDB-1-1-1024 : has running benchmarks YugabyteDB-1-1-1024 : has running benchmarks - waiting 30s - : done YugabyteDB-1-1-1024 : has running benchmarks +done +YugabyteDB-1-1-1024 : has running benchmarks +done +YugabyteDB-1-1-1024 : has running benchmarks - waiting 30s - : done YugabyteDB-1-1-1024 : has running benchmarks done YugabyteDB-1-1-1024 : has running benchmarks YugabyteDB-1-1-1024-3 : showing benchmarker times -YugabyteDB-1-1-1024-3 : benchmarker timespan (start to end single container [s]) = 3628 -YugabyteDB-1-1-1024-3 : benchmarker times (start/end per pod and container) = [(1730234929, 1730238537), (1730234928, 1730238536), (1730234929, 1730238537), (1730234929, 1730238538)] -YugabyteDB-1-1-1024-3 : found and updated times {'benchmarker': [(1730234929, 1730238537), (1730234928, 1730238536), (1730234929, 1730238537), (1730234929, 1730238538)]} +YugabyteDB-1-1-1024-3 : benchmarker timespan (start to end single container [s]) = 3719 +YugabyteDB-1-1-1024-3 : benchmarker times (start/end per pod and container) = [(1734637375, 1734640991), (1734637375, 1734641019), (1734637375, 1734641073), (1734637375, 1734640998)] +YugabyteDB-1-1-1024-3 : found and updated times {'benchmarker': [(1734637375, 1734640991), (1734637375, 1734641019), (1734637375, 1734641073), (1734637375, 1734640998)]} done YugabyteDB-1-1-1024 : benchmarks done 0 of 1. This will be client 4 -YugabyteDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '8', 'SF': '128', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'postgres', 'BEXHOMA_DATABASE': 'yugabyte', 'BENCHBASE_TARGET': 2048, 'BENCHBASE_TERMINALS': 8, 'BENCHBASE_TIME': 3600, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_PORT': 5433} +YugabyteDB-1-1-1024 : we will change parameters of benchmark as {'PARALLEL': '8', 'SF': '128', 'BENCHBASE_BENCH': 'tpcc', 'BENCHBASE_PROFILE': 'postgres', 'BEXHOMA_DATABASE': 'yugabyte', 'BENCHBASE_TARGET': 2048, 'BENCHBASE_TERMINALS': 8, 'BENCHBASE_TIME': 3600, 'BENCHBASE_ISOLATION': 'TRANSACTION_READ_COMMITTED', 'BEXHOMA_USER': 'yugabyte', 'BEXHOMA_PASSWORD': '', 'BEXHOMA_PORT': 5433} YugabyteDB-1-1-1024-4 : start benchmarking -YugabyteDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1730226312 +YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-0 +YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-1 +YugabyteDB-1-1-1024 : distributed system - get host info for worker yb-tserver-2 +YugabyteDB-1-1-1024 : benchmarking results in folder /home/perdelt/benchmarks/1734628788 - waiting 10s - : done -YugabyteDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-yugabytedb-1-1-1024-1730226312-1-4-4gqxz: found +YugabyteDB-1-1-1024 : benchmarking is waiting for job bexhoma-benchmarker-yugabytedb-1-1-1024-1734628788-1-4-7nwvg: .found - waiting 30s - : done YugabyteDB-1-1-1024 : has running benchmarks - waiting 30s - : done @@ -1107,9 +1117,9 @@ YugabyteDB-1-1-1024 : has running benchmarks YugabyteDB-1-1-1024 : has running benchmarks done YugabyteDB-1-1-1024 : has running benchmarks -done +- waiting 30s - : done YugabyteDB-1-1-1024 : has running benchmarks -done +- waiting 30s - : done YugabyteDB-1-1-1024 : has running benchmarks - waiting 30s - : done YugabyteDB-1-1-1024 : has running benchmarks @@ -1117,36 +1127,20 @@ done YugabyteDB-1-1-1024 : has running benchmarks done YugabyteDB-1-1-1024 : has running benchmarks +- waiting 30s - : done +YugabyteDB-1-1-1024 : has running benchmarks +done +YugabyteDB-1-1-1024 : has running benchmarks YugabyteDB-1-1-1024-4 : showing benchmarker times -YugabyteDB-1-1-1024-4 : benchmarker timespan (start to end single container [s]) = 3791 -YugabyteDB-1-1-1024-4 : benchmarker times (start/end per pod and container) = [(1730238595, 1730242235), (1730238595, 1730242255), (1730238595, 1730242231), (1730238595, 1730242224), (1730238595, 1730242245), (1730238595, 1730242322), (1730238595, 1730242213), (1730238596, 1730242350)] -YugabyteDB-1-1-1024-4 : found and updated times {'benchmarker': [(1730238595, 1730242235), (1730238595, 1730242255), (1730238595, 1730242231), (1730238595, 1730242224), (1730238595, 1730242245), (1730238595, 1730242322), (1730238595, 1730242213), (1730238596, 1730242350)]} +YugabyteDB-1-1-1024-4 : benchmarker timespan (start to end single container [s]) = 3910 +YugabyteDB-1-1-1024-4 : benchmarker times (start/end per pod and container) = [(1734641136, 1734644797), (1734641136, 1734644818), (1734641136, 1734644926), (1734641136, 1734644953), (1734641136, 1734644953), (1734641137, 1734645005), (1734641136, 1734644927), (1734641137, 1734644932)] +YugabyteDB-1-1-1024-4 : found and updated times {'benchmarker': [(1734641136, 1734644797), (1734641136, 1734644818), (1734641136, 1734644926), (1734641136, 1734644953), (1734641136, 1734644953), (1734641137, 1734645005), (1734641136, 1734644927), (1734641137, 1734644932)]} done YugabyteDB-1-1-1024 : can be stopped -Experiment ends at 2024-10-29 23:53:30.141071 (4978350.265050158): 16097.839604604058s total +Experiment ends at 2024-12-19 22:51:10.118243 (4329871.525375789): 16282.006199037656s total Experiment : downloading partial results Experiment : uploading full results Experiment : uploading workload file -Benchmarking connection ... Latency Distribution.Average Latency (microseconds) -connection_pod ... -YugabyteDB-1-1-1024-1-1 YugabyteDB-1-1-1024-1 ... 136271 -YugabyteDB-1-1-1024-3-1 YugabyteDB-1-1-1024-3 ... 151569 -YugabyteDB-1-1-1024-2-1 YugabyteDB-1-1-1024-2 ... 142319 -YugabyteDB-1-1-1024-4-1 YugabyteDB-1-1-1024-4 ... 240528 -YugabyteDB-1-1-1024-4-2 YugabyteDB-1-1-1024-4 ... 251836 -YugabyteDB-1-1-1024-4-3 YugabyteDB-1-1-1024-4 ... 295401 -YugabyteDB-1-1-1024-4-4 YugabyteDB-1-1-1024-4 ... 246889 -YugabyteDB-1-1-1024-3-2 YugabyteDB-1-1-1024-3 ... 164894 -YugabyteDB-1-1-1024-4-5 YugabyteDB-1-1-1024-4 ... 225020 -YugabyteDB-1-1-1024-3-3 YugabyteDB-1-1-1024-3 ... 157392 -YugabyteDB-1-1-1024-4-6 YugabyteDB-1-1-1024-4 ... 272495 -YugabyteDB-1-1-1024-4-7 YugabyteDB-1-1-1024-4 ... 268149 -YugabyteDB-1-1-1024-2-2 YugabyteDB-1-1-1024-2 ... 141678 -YugabyteDB-1-1-1024-4-8 YugabyteDB-1-1-1024-4 ... 268839 -YugabyteDB-1-1-1024-3-4 YugabyteDB-1-1-1024-3 ... 162663 - -[15 rows x 36 columns] -Workflow {'YugabyteDB-1-1-1024': [[1, 4, 2, 8]]} Result workflow not complete ## Show Summary @@ -1154,9 +1148,9 @@ Result workflow not complete ### Workload Benchbase Workload SF=128 (warehouses for TPC-C) Type: benchbase - Duration: 16098s - Code: 1730226312 - This includes no queries. Benchbase runs the benchmark + Duration: 16283s + Code: 1734628788 + Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. Benchmark is 'tpcc'. Scaling factor (e.g., number of warehouses) is 128. Benchmarking runs for 60 minutes. Target is based on multiples of '1024'. Factors for benchmarking are [16]. @@ -1172,69 +1166,153 @@ Benchbase Workload SF=128 (warehouses for TPC-C) ### Connections YugabyteDB-1-1-1024-1 uses docker image postgres:15.0 - RAM:541008605184 + RAM:541008576512 CPU:AMD Opteron(tm) Processor 6378 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254319580 + disk:249254048 datadisk:39428 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:458849332 + worker 1 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:603879336 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:143144816 YugabyteDB-1-1-1024-2 uses docker image postgres:15.0 - RAM:541008605184 + RAM:541008576512 CPU:AMD Opteron(tm) Processor 6378 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254319748 + disk:249254240 datadisk:39428 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:450176968 + worker 1 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:595302840 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:133934964 YugabyteDB-1-1-1024-3 uses docker image postgres:15.0 - RAM:541008605184 + RAM:541008576512 CPU:AMD Opteron(tm) Processor 6378 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254319920 + disk:249254432 datadisk:39428 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:450484892 + worker 1 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:595477572 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:134049872 YugabyteDB-1-1-1024-4 uses docker image postgres:15.0 - RAM:541008605184 + RAM:541008576512 CPU:AMD Opteron(tm) Processor 6378 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254320088 + disk:249254644 datadisk:39428 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:456563680 + worker 1 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:640944212 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:134098940 ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -YugabyteDB-1-1-1024-1 1 64 16384 1 3600.0 469.61 327056.0 136271.00 -YugabyteDB-1-1-1024-2 1 64 16384 2 3600.0 450.66 357886.0 141998.50 -YugabyteDB-1-1-1024-3 1 64 16384 4 3600.0 402.57 409184.0 159129.50 -YugabyteDB-1-1-1024-4 1 64 16384 8 3600.0 247.49 896527.0 258644.62 +YugabyteDB-1-1-1024-1 1 64 16384 1 3600.0 482.34 346967.0 132675.00 +YugabyteDB-1-1-1024-2 1 64 16384 2 3600.0 388.47 414796.0 164642.00 +YugabyteDB-1-1-1024-3 1 64 16384 4 3600.0 376.70 430011.0 169282.00 +YugabyteDB-1-1-1024-4 1 64 16384 8 3600.0 258.70 767035.0 244458.62 Warehouses: 128 ### Workflow #### Actual -DBMS YugabyteDB-1-1-1024 - Pods [[1, 4, 2, 8]] +DBMS YugabyteDB-1-1-1024 - Pods [[8, 4, 2, 1]] #### Planned DBMS YugabyteDB-1-1-1024 - Pods [[1, 2, 4, 8]] ### Loading time_load terminals pods Imported warehouses [1/h] -YugabyteDB-1-1-1024-1 1151.0 1.0 1.0 400.347524 -YugabyteDB-1-1-1024-2 1151.0 1.0 2.0 400.347524 -YugabyteDB-1-1-1024-3 1151.0 1.0 4.0 400.347524 -YugabyteDB-1-1-1024-4 1151.0 1.0 8.0 400.347524 +YugabyteDB-1-1-1024-1 1033.0 1.0 1.0 446.07938 +YugabyteDB-1-1-1024-2 1033.0 1.0 2.0 446.07938 +YugabyteDB-1-1-1024-3 1033.0 1.0 4.0 446.07938 +YugabyteDB-1-1-1024-4 1033.0 1.0 8.0 446.07938 ### Tests TEST passed: Throughput (requests/second) contains no 0 or NaN diff --git a/logs_tests/doc_benchbase_yugabytedb_2_summary.txt b/logs_tests/doc_benchbase_yugabytedb_2_summary.txt index 87e66cdc..a9f73ab0 100644 --- a/logs_tests/doc_benchbase_yugabytedb_2_summary.txt +++ b/logs_tests/doc_benchbase_yugabytedb_2_summary.txt @@ -3,9 +3,9 @@ ### Workload Benchbase Workload SF=128 (warehouses for TPC-C) Type: benchbase - Duration: 16098s - Code: 1730226312 - This includes no queries. Benchbase runs the benchmark + Duration: 16283s + Code: 1734628788 + Benchbase runs the benchmark. This experiment compares run time and resource consumption of Benchbase queries in different DBMS. Benchbase data is generated and loaded using several threads. Benchmark is 'tpcc'. Scaling factor (e.g., number of warehouses) is 128. Benchmarking runs for 60 minutes. Target is based on multiples of '1024'. Factors for benchmarking are [16]. @@ -21,69 +21,153 @@ Benchbase Workload SF=128 (warehouses for TPC-C) ### Connections YugabyteDB-1-1-1024-1 uses docker image postgres:15.0 - RAM:541008605184 + RAM:541008576512 CPU:AMD Opteron(tm) Processor 6378 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254319580 + disk:249254048 datadisk:39428 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:458849332 + worker 1 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:603879336 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:143144816 YugabyteDB-1-1-1024-2 uses docker image postgres:15.0 - RAM:541008605184 + RAM:541008576512 CPU:AMD Opteron(tm) Processor 6378 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254319748 + disk:249254240 datadisk:39428 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:450176968 + worker 1 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:595302840 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:133934964 YugabyteDB-1-1-1024-3 uses docker image postgres:15.0 - RAM:541008605184 + RAM:541008576512 CPU:AMD Opteron(tm) Processor 6378 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254319920 + disk:249254432 datadisk:39428 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:450484892 + worker 1 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:595477572 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:134049872 YugabyteDB-1-1-1024-4 uses docker image postgres:15.0 - RAM:541008605184 + RAM:541008576512 CPU:AMD Opteron(tm) Processor 6378 Cores:64 - host:5.15.0-116-generic + host:5.15.0-126-generic node:cl-worker11 - disk:254320088 + disk:249254644 datadisk:39428 requests_cpu:4 requests_memory:16Gi + worker 0 + RAM:1081966526464 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker28 + disk:456563680 + worker 1 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:640944212 + worker 2 + RAM:540587499520 + CPU:AMD EPYC 7502 32-Core Processor + Cores:128 + host:5.15.0-126-generic + node:cl-worker22 + disk:134098940 ### Execution experiment_run terminals target pod_count time Throughput (requests/second) Latency Distribution.95th Percentile Latency (microseconds) Latency Distribution.Average Latency (microseconds) -YugabyteDB-1-1-1024-1 1 64 16384 1 3600.0 469.61 327056.0 136271.00 -YugabyteDB-1-1-1024-2 1 64 16384 2 3600.0 450.66 357886.0 141998.50 -YugabyteDB-1-1-1024-3 1 64 16384 4 3600.0 402.57 409184.0 159129.50 -YugabyteDB-1-1-1024-4 1 64 16384 8 3600.0 247.49 896527.0 258644.62 +YugabyteDB-1-1-1024-1 1 64 16384 1 3600.0 482.34 346967.0 132675.00 +YugabyteDB-1-1-1024-2 1 64 16384 2 3600.0 388.47 414796.0 164642.00 +YugabyteDB-1-1-1024-3 1 64 16384 4 3600.0 376.70 430011.0 169282.00 +YugabyteDB-1-1-1024-4 1 64 16384 8 3600.0 258.70 767035.0 244458.62 Warehouses: 128 ### Workflow #### Actual -DBMS YugabyteDB-1-1-1024 - Pods [[1, 4, 2, 8]] +DBMS YugabyteDB-1-1-1024 - Pods [[8, 4, 2, 1]] #### Planned DBMS YugabyteDB-1-1-1024 - Pods [[1, 2, 4, 8]] ### Loading time_load terminals pods Imported warehouses [1/h] -YugabyteDB-1-1-1024-1 1151.0 1.0 1.0 400.347524 -YugabyteDB-1-1-1024-2 1151.0 1.0 2.0 400.347524 -YugabyteDB-1-1-1024-3 1151.0 1.0 4.0 400.347524 -YugabyteDB-1-1-1024-4 1151.0 1.0 8.0 400.347524 +YugabyteDB-1-1-1024-1 1033.0 1.0 1.0 446.07938 +YugabyteDB-1-1-1024-2 1033.0 1.0 2.0 446.07938 +YugabyteDB-1-1-1024-3 1033.0 1.0 4.0 446.07938 +YugabyteDB-1-1-1024-4 1033.0 1.0 8.0 446.07938 ### Tests TEST passed: Throughput (requests/second) contains no 0 or NaN diff --git a/logs_tests/doc_tpch_monetdb_3.log b/logs_tests/doc_tpch_monetdb_3.log index 1c5199b0..12a5cc57 100644 --- a/logs_tests/doc_tpch_monetdb_3.log +++ b/logs_tests/doc_tpch_monetdb_3.log @@ -1,13 +1,15 @@ +nohup: Eingabe wird ignoriert Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1728344200 -Experiment : starts at 2024-10-08 01:36:40.264374 (3080140.388347578) +Experiment : has code 1734693433 +Experiment : starts at 2024-12-20 12:17:13.412070 (4378234.819201565) Experiment : This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=100) data is loaded and benchmark is executed. Query ordering is Q1 - Q22. All instances use the same query parameters. +Timeout per query is 1200. Import sets indexes and constraints after loading and recomputes statistics. System metrics are monitored by a cluster-wide installation. Benchmark is limited to DBMS ['MonetDB']. @@ -18,205 +20,583 @@ SUT is fixed to cl-worker11. Database is persisted to disk of type shared and size 300Gi. Loading is tested with [8] threads, split into [8] pods. Benchmarking is tested with [1] threads, split into [1] pods. -Benchmarking is run as [1, 1, 5, 5] times the number of benchmarking pods. +Benchmarking is run as [1, 1, 3] times the number of benchmarking pods. Experiment is run once. Cluster monitoring : is running +done +MonetDB-BHT-8 : will start now - waiting 30s - : done -MonetDB-BHT-8 : has to wait - 1 running and 0 pending pods: max is 1 pods per cluster +MonetDB-BHT-8 : is not loaded yet +MonetDB-BHT-8 : will start loading but not before 2024-12-20 11:19:14 (that is in 60 secs) - waiting 30s - : done -MonetDB-BHT-8 : has to wait - 1 running and 0 pending pods: max is 1 pods per cluster +MonetDB-BHT-8 : is not loaded yet +MonetDB-BHT-8 : will start loading but not before 2024-12-20 11:19:14 +done +MonetDB-BHT-8 : is not loaded yet +MonetDB-BHT-8 : start asynch loading scripts of type loaded +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has to wait - 1 running and 0 pending pods: max is 1 pods per cluster +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has to wait - 1 running and 0 pending pods: max is 1 pods per cluster +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has to wait - 1 running and 0 pending pods: max is 1 pods per cluster +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has to wait - 1 running and 0 pending pods: max is 1 pods per cluster -done -MonetDB-BHT-8 : will start now -MonetDB-BHT-8 : storage exists bexhoma-storage-monetdb-tpch-100 -MonetDB-BHT-8 : loading is set to finished +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : will start benchmarking but not before 2024-10-07 23:41:41 (that is in 60 secs) +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : will start benchmarking but not before 2024-10-07 23:41:41 -done -MonetDB-BHT-8 : benchmarks done 0 of 1. This will be client 1 -MonetDB-BHT-8-1 : start benchmarking -MonetDB-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1728344200 -- waiting 10s - : done -MonetDB-BHT-8 : benchmarking is waiting for job bexhoma-benchmarker-monetdb-bht-8-1728344200-1-1-nqxzb: found +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading done -MonetDB-BHT-8 : has running benchmarks -MonetDB-BHT-8-1 : showing benchmarker times -MonetDB-BHT-8-1 : benchmarker timespan (start to end single container [s]) = 1457 -MonetDB-BHT-8-1 : benchmarker times (start/end per pod and container) = [(1728344513, 1728345947)] -MonetDB-BHT-8-1 : found and updated times {'benchmarker': [(1728344513, 1728345947)]} -MonetDB-BHT-8-1 : collecting execution metrics of SUT -MonetDB-BHT-8-1 : collecting metrics of benchmarker +MonetDB-BHT-8 : is loading done -MonetDB-BHT-8 : benchmarks done 0 of 1. This will be client 2 -MonetDB-BHT-8-2 : start benchmarking -MonetDB-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1728344200 -- waiting 10s - : done -MonetDB-BHT-8 : benchmarking is waiting for job bexhoma-benchmarker-monetdb-bht-8-1728344200-1-2-rnkdm: found +MonetDB-BHT-8 : is loading +done +MonetDB-BHT-8 : is loading +done +MonetDB-BHT-8 : is loading +done +MonetDB-BHT-8 : is loading +done +MonetDB-BHT-8 : is loading +done +MonetDB-BHT-8 : showing loader times +MonetDB-BHT-8 : generator timespan (first to last [s]) = 955 +MonetDB-BHT-8 : loader timespan (first to last [s]) = 1602 +MonetDB-BHT-8 : total timespan (first to last [s]) = 2573 +MonetDB-BHT-8 : start asynch loading scripts of type indexed +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8 : is loading - waiting 30s - : done -MonetDB-BHT-8 : has running benchmarks -done -MonetDB-BHT-8 : has running benchmarks -MonetDB-BHT-8-2 : showing benchmarker times -MonetDB-BHT-8-2 : benchmarker timespan (start to end single container [s]) = 764 -MonetDB-BHT-8-2 : benchmarker times (start/end per pod and container) = [(1728346015, 1728346766)] -MonetDB-BHT-8-2 : found and updated times {'benchmarker': [(1728346015, 1728346766)]} -MonetDB-BHT-8-2 : collecting execution metrics of SUT -MonetDB-BHT-8-2 : collecting metrics of benchmarker +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading +- waiting 30s - : done +MonetDB-BHT-8 : is loading done -MonetDB-BHT-8 : benchmarks done 0 of 1. This will be client 3 -MonetDB-BHT-8-3 : start benchmarking -MonetDB-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1728344200 +MonetDB-BHT-8 : benchmarks done 0 of 1. This will be client 1 +MonetDB-BHT-8-1 : start benchmarking +Worker pods found: [] +MonetDB-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1734693433 - waiting 10s - : done -MonetDB-BHT-8 : benchmarking is waiting for job bexhoma-benchmarker-monetdb-bht-8-1728344200-1-3-44jlc: found +found +MonetDB-BHT-8-1 : collecting loading metrics of SUT +MonetDB-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="dbms"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +MonetDB-BHT-8-1 : collecting metrics of data generator +MonetDB-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +MonetDB-BHT-8-1 : collecting metrics of data injector +MonetDB-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} - waiting 30s - : done MonetDB-BHT-8 : has running benchmarks - waiting 30s - : done @@ -257,8 +637,29 @@ MonetDB-BHT-8 : has running benchmarks MonetDB-BHT-8 : has running benchmarks - waiting 30s - : done MonetDB-BHT-8 : has running benchmarks -- waiting 30s - : done +done MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8-1 : showing benchmarker times +MonetDB-BHT-8-1 : benchmarker timespan (start to end single container [s]) = 662 +MonetDB-BHT-8-1 : benchmarker times (start/end per pod and container) = [(1734701875, 1734702503)] +MonetDB-BHT-8-1 : found and updated times {'benchmarker': [(1734701875, 1734702503)]} +MonetDB-BHT-8-1 : collecting execution metrics of SUT +MonetDB-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="dbms"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +MonetDB-BHT-8-1 : collecting metrics of benchmarker +MonetDB-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +done +MonetDB-BHT-8 : benchmarks done 0 of 1. This will be client 2 +MonetDB-BHT-8-2 : start benchmarking +Worker pods found: [] +MonetDB-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1734693433 +- waiting 10s - : done +found +MonetDB-BHT-8-2 : collecting loading metrics of SUT +MonetDB-BHT-8-2 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="dbms"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +MonetDB-BHT-8-2 : collecting metrics of data generator +MonetDB-BHT-8-2 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +MonetDB-BHT-8-2 : collecting metrics of data injector +MonetDB-BHT-8-2 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} - waiting 30s - : done MonetDB-BHT-8 : has running benchmarks - waiting 30s - : done @@ -277,24 +678,6 @@ MonetDB-BHT-8 : has running benchmarks MonetDB-BHT-8 : has running benchmarks - waiting 30s - : done MonetDB-BHT-8 : has running benchmarks -done -MonetDB-BHT-8 : has running benchmarks -done -MonetDB-BHT-8 : has running benchmarks -done -MonetDB-BHT-8 : has running benchmarks -MonetDB-BHT-8-3 : showing benchmarker times -MonetDB-BHT-8-3 : benchmarker timespan (start to end single container [s]) = 1009 -MonetDB-BHT-8-3 : benchmarker times (start/end per pod and container) = [(1728346836, 1728347782), (1728346836, 1728347760), (1728346837, 1728347795), (1728346837, 1728347796), (1728346836, 1728347790)] -MonetDB-BHT-8-3 : found and updated times {'benchmarker': [(1728346836, 1728347782), (1728346836, 1728347760), (1728346837, 1728347795), (1728346837, 1728347796), (1728346836, 1728347790)]} -MonetDB-BHT-8-3 : collecting execution metrics of SUT -MonetDB-BHT-8-3 : collecting metrics of benchmarker -done -MonetDB-BHT-8 : benchmarks done 0 of 1. This will be client 4 -MonetDB-BHT-8-4 : start benchmarking -MonetDB-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1728344200 -- waiting 10s - : done -MonetDB-BHT-8 : benchmarking is waiting for job bexhoma-benchmarker-monetdb-bht-8-1728344200-1-4-27j5d: found - waiting 30s - : done MonetDB-BHT-8 : has running benchmarks - waiting 30s - : done @@ -313,8 +696,29 @@ MonetDB-BHT-8 : has running benchmarks MonetDB-BHT-8 : has running benchmarks - waiting 30s - : done MonetDB-BHT-8 : has running benchmarks -- waiting 30s - : done +done MonetDB-BHT-8 : has running benchmarks +MonetDB-BHT-8-2 : showing benchmarker times +MonetDB-BHT-8-2 : benchmarker timespan (start to end single container [s]) = 603 +MonetDB-BHT-8-2 : benchmarker times (start/end per pod and container) = [(1734702579, 1734703146)] +MonetDB-BHT-8-2 : found and updated times {'benchmarker': [(1734702579, 1734703146)]} +MonetDB-BHT-8-2 : collecting execution metrics of SUT +MonetDB-BHT-8-2 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="dbms"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +MonetDB-BHT-8-2 : collecting metrics of benchmarker +MonetDB-BHT-8-2 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +done +MonetDB-BHT-8 : benchmarks done 0 of 1. This will be client 3 +MonetDB-BHT-8-3 : start benchmarking +Worker pods found: [] +MonetDB-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1734693433 +- waiting 10s - : done +found +MonetDB-BHT-8-3 : collecting loading metrics of SUT +MonetDB-BHT-8-3 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="dbms"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +MonetDB-BHT-8-3 : collecting metrics of data generator +MonetDB-BHT-8-3 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +MonetDB-BHT-8-3 : collecting metrics of data injector +MonetDB-BHT-8-3 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} - waiting 30s - : done MonetDB-BHT-8 : has running benchmarks - waiting 30s - : done @@ -355,14 +759,226 @@ done MonetDB-BHT-8 : has running benchmarks done MonetDB-BHT-8 : has running benchmarks -MonetDB-BHT-8-4 : showing benchmarker times -MonetDB-BHT-8-4 : benchmarker timespan (start to end single container [s]) = 919 -MonetDB-BHT-8-4 : benchmarker times (start/end per pod and container) = [(1728347894, 1728348760), (1728347894, 1728348760), (1728347894, 1728348768), (1728347894, 1728348756), (1728347894, 1728348768)] -MonetDB-BHT-8-4 : found and updated times {'benchmarker': [(1728347894, 1728348760), (1728347894, 1728348760), (1728347894, 1728348768), (1728347894, 1728348756), (1728347894, 1728348768)]} -MonetDB-BHT-8-4 : collecting execution metrics of SUT -MonetDB-BHT-8-4 : collecting metrics of benchmarker +MonetDB-BHT-8-3 : showing benchmarker times +MonetDB-BHT-8-3 : benchmarker timespan (start to end single container [s]) = 646 +MonetDB-BHT-8-3 : benchmarker times (start/end per pod and container) = [(1734703227, 1734703832), (1734703227, 1734703827), (1734703227, 1734703819)] +MonetDB-BHT-8-3 : found and updated times {'benchmarker': [(1734703227, 1734703832), (1734703227, 1734703827), (1734703227, 1734703819)]} +MonetDB-BHT-8-3 : collecting execution metrics of SUT +MonetDB-BHT-8-3 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="dbms"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +MonetDB-BHT-8-3 : collecting metrics of benchmarker +MonetDB-BHT-8-3 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)monetdb-bht-8-1734693433(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} done MonetDB-BHT-8 : can be stopped -Experiment : ends at 2024-10-08 02:54:05.075584 (3084785.199564025) - 4644.81s total +Experiment : ends at 2024-12-20 15:11:45.007672 (4388706.414805365) - 10471.60s total Join results done! -Build evaluation cube \ No newline at end of file +done! +Experiment : downloading partial results +Experiment : uploading full results +Experiment : uploading workload file + +## Show Summary + +### Workload +TPC-H Queries SF=100 + Type: tpch + Duration: 10472s + Code: 1734693433 + This includes the reading queries of TPC-H. + This experiment compares run time and resource consumption of TPC-H queries in different DBMS. + TPC-H (SF=100) data is loaded and benchmark is executed. + Query ordering is Q1 - Q22. + All instances use the same query parameters. + Timeout per query is 1200. + Import sets indexes and constraints after loading and recomputes statistics. + System metrics are monitored by a cluster-wide installation. + Benchmark is limited to DBMS ['MonetDB']. + Import is handled by 8 processes (pods). + Loading is fixed to cl-worker19. + Benchmarking is fixed to cl-worker19. + SUT is fixed to cl-worker11. + Database is persisted to disk of type shared and size 300Gi. + Loading is tested with [8] threads, split into [8] pods. + Benchmarking is tested with [1] threads, split into [1] pods. + Benchmarking is run as [1, 1, 3] times the number of benchmarking pods. + Experiment is run once. + +### Connections +MonetDB-BHT-8-1-1 uses docker image monetdb/monetdb:Aug2024 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:250230356 + datadisk:218202266 + volume_size:300G + volume_used:209G + requests_cpu:4 + requests_memory:16Gi +MonetDB-BHT-8-2-1 uses docker image monetdb/monetdb:Aug2024 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:250230364 + datadisk:223241760 + volume_size:300G + volume_used:213G + requests_cpu:4 + requests_memory:16Gi +MonetDB-BHT-8-3-1 uses docker image monetdb/monetdb:Aug2024 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:250062784 + datadisk:223241831 + volume_size:300G + volume_used:213G + requests_cpu:4 + requests_memory:16Gi +MonetDB-BHT-8-3-2 uses docker image monetdb/monetdb:Aug2024 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:250062784 + datadisk:223241831 + volume_size:300G + volume_used:213G + requests_cpu:4 + requests_memory:16Gi +MonetDB-BHT-8-3-3 uses docker image monetdb/monetdb:Aug2024 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:250062784 + datadisk:223241831 + volume_size:300G + volume_used:213G + requests_cpu:4 + requests_memory:16Gi + +### Errors (failed queries) + MonetDB-BHT-8-1-1 MonetDB-BHT-8-2-1 MonetDB-BHT-8-3-1 MonetDB-BHT-8-3-2 MonetDB-BHT-8-3-3 +Pricing Summary Report (TPC-H Q1) False False True True True +Discounted Revenue (TPC-H Q19) True True True True True +Pricing Summary Report (TPC-H Q1) +MonetDB-BHT-8-3-2: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded +MonetDB-BHT-8-3-3: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded +MonetDB-BHT-8-3-1: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded +Discounted Revenue (TPC-H Q19) +MonetDB-BHT-8-3-2: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded +MonetDB-BHT-8-3-3: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded +MonetDB-BHT-8-1-1: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded +MonetDB-BHT-8-2-1: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded +MonetDB-BHT-8-3-1: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded + +### Warnings (result mismatch) + MonetDB-BHT-8-1-1 MonetDB-BHT-8-2-1 MonetDB-BHT-8-3-1 MonetDB-BHT-8-3-2 MonetDB-BHT-8-3-3 +Pricing Summary Report (TPC-H Q1) True True False False False + +### Latency of Timer Execution [ms] +DBMS MonetDB-BHT-8-1-1 MonetDB-BHT-8-2-1 MonetDB-BHT-8-3-1 MonetDB-BHT-8-3-2 MonetDB-BHT-8-3-3 +Minimum Cost Supplier Query (TPC-H Q2) 3194.11 1670.45 5828.99 366.88 5829.51 +Shipping Priority (TPC-H Q3) 13761.14 13605.94 47417.47 30838.94 47307.44 +Order Priority Checking Query (TPC-H Q4) 12545.57 12306.40 30806.00 11911.52 32084.56 +Local Supplier Volume (TPC-H Q5) 11432.09 10183.79 13922.48 9918.79 12919.25 +Forecasting Revenue Change (TPC-H Q6) 6597.76 4943.78 6681.21 4111.89 6729.92 +Forecasting Revenue Change (TPC-H Q7) 8123.89 3621.12 10699.60 12694.40 10829.30 +National Market Share (TPC-H Q8) 60812.96 35310.95 33701.69 32777.41 34062.49 +Product Type Profit Measure (TPC-H Q9) 17328.66 17680.32 18508.85 19264.58 17902.92 +Forecasting Revenue Change (TPC-H Q10) 27227.63 26825.03 26808.97 27131.83 26433.34 +Important Stock Identification (TPC-H Q11) 1193.79 1322.06 1379.97 670.82 1770.63 +Shipping Modes and Order Priority (TPC-H Q12) 4938.74 4920.75 8202.82 6538.83 8342.99 +Customer Distribution (TPC-H Q13) 106149.62 108491.70 127149.26 111621.90 124587.94 +Forecasting Revenue Change (TPC-H Q14) 8514.29 6729.16 274.97 7634.95 328.72 +Top Supplier Query (TPC-H Q15) 9665.45 6289.91 5467.42 7270.07 5792.61 +Parts/Supplier Relationship (TPC-H Q16) 12033.89 12450.62 11841.51 14016.50 11828.14 +Small-Quantity-Order Revenue (TPC-H Q17) 18140.36 15204.61 15480.68 19180.17 17408.95 +Large Volume Customer (TPC-H Q18) 37795.66 19125.43 24020.87 24815.32 25415.36 +Potential Part Promotion (TPC-H Q20) 4293.23 4181.49 7021.23 6428.42 6791.12 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 17644.97 16196.43 47957.20 42946.58 33884.11 +Global Sales Opportunity Query (TPC-H Q22) 6665.92 6907.86 6349.41 6282.68 7720.72 + +### Loading [s] + timeGenerate timeIngesting timeSchema timeIndex timeLoad +MonetDB-BHT-8-1-1 955.0 1602.0 6.0 5671.0 8250.0 +MonetDB-BHT-8-2-1 955.0 1602.0 6.0 5671.0 8250.0 +MonetDB-BHT-8-3-1 955.0 1602.0 6.0 5671.0 8250.0 +MonetDB-BHT-8-3-2 955.0 1602.0 6.0 5671.0 8250.0 +MonetDB-BHT-8-3-3 955.0 1602.0 6.0 5671.0 8250.0 + +### Geometric Mean of Medians of Timer Run [s] + Geo Times [s] +DBMS +MonetDB-BHT-8-1-1 12.49 +MonetDB-BHT-8-2-1 10.16 +MonetDB-BHT-8-3-1 12.53 +MonetDB-BHT-8-3-2 11.64 +MonetDB-BHT-8-3-3 12.69 + +### Power@Size + Power@Size [~Q/h] +DBMS +MonetDB-BHT-8-1-1 30931.10 +MonetDB-BHT-8-2-1 37542.45 +MonetDB-BHT-8-3-1 30428.55 +MonetDB-BHT-8-3-2 33693.59 +MonetDB-BHT-8-3-3 29802.28 + +### Throughput@Size + time [s] count SF Throughput@Size [~GB/h] +DBMS SF num_experiment num_client +MonetDB-BHT-8-1 100 1 1 628 1 100 12611.46 +MonetDB-BHT-8-2 100 1 2 567 1 100 13968.25 +MonetDB-BHT-8-3 100 1 3 605 3 100 39272.73 + +### Workflow + +#### Actual +DBMS MonetDB-BHT-8 - Pods [[1, 1, 3]] + +#### Planned +DBMS MonetDB-BHT-8 - Pods [[1, 1, 3]] + +### Ingestion - SUT + CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] +MonetDB-BHT-8-1 19913.85 12.72 107.69 205.07 +MonetDB-BHT-8-2 19913.85 12.72 107.69 205.07 +MonetDB-BHT-8-3 19913.85 12.72 107.69 205.07 + +### Ingestion - Loader + CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] +MonetDB-BHT-8-1 2348.27 1.26 0.02 0.02 +MonetDB-BHT-8-2 2348.27 1.26 0.02 0.02 +MonetDB-BHT-8-3 2348.27 1.26 0.02 0.02 + +### Execution - SUT + CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] +MonetDB-BHT-8-1 6101.85 23.08 132.79 260.12 +MonetDB-BHT-8-2 6120.54 11.71 151.97 276.09 +MonetDB-BHT-8-3 11331.73 21.06 183.26 313.77 + +### Execution - Benchmarker + CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] +MonetDB-BHT-8-1 26.05 0.01 0.29 0.30 +MonetDB-BHT-8-2 26.05 0.02 0.52 0.54 +MonetDB-BHT-8-3 59.23 0.08 0.98 0.99 + +### Tests +TEST passed: Geo Times [s] contains no 0 or NaN +TEST passed: Power@Size [~Q/h] contains no 0 or NaN +TEST passed: Throughput@Size [~GB/h] contains no 0 or NaN +TEST failed: SQL errors +TEST failed: SQL warnings (result mismatch) +TEST passed: Ingestion SUT contains no 0 or NaN in CPU [CPUs] +TEST passed: Ingestion Loader contains no 0 or NaN in CPU [CPUs] +TEST passed: Execution SUT contains no 0 or NaN in CPU [CPUs] +TEST passed: Execution Benchmarker contains no 0 or NaN in CPU [CPUs] +TEST passed: Workflow as planned diff --git a/logs_tests/doc_tpch_monetdb_3_summary.txt b/logs_tests/doc_tpch_monetdb_3_summary.txt index e69de29b..7fba2b98 100644 --- a/logs_tests/doc_tpch_monetdb_3_summary.txt +++ b/logs_tests/doc_tpch_monetdb_3_summary.txt @@ -0,0 +1,206 @@ +## Show Summary + +### Workload +TPC-H Queries SF=100 + Type: tpch + Duration: 10472s + Code: 1734693433 + This includes the reading queries of TPC-H. + This experiment compares run time and resource consumption of TPC-H queries in different DBMS. + TPC-H (SF=100) data is loaded and benchmark is executed. + Query ordering is Q1 - Q22. + All instances use the same query parameters. + Timeout per query is 1200. + Import sets indexes and constraints after loading and recomputes statistics. + System metrics are monitored by a cluster-wide installation. + Benchmark is limited to DBMS ['MonetDB']. + Import is handled by 8 processes (pods). + Loading is fixed to cl-worker19. + Benchmarking is fixed to cl-worker19. + SUT is fixed to cl-worker11. + Database is persisted to disk of type shared and size 300Gi. + Loading is tested with [8] threads, split into [8] pods. + Benchmarking is tested with [1] threads, split into [1] pods. + Benchmarking is run as [1, 1, 3] times the number of benchmarking pods. + Experiment is run once. + +### Connections +MonetDB-BHT-8-1-1 uses docker image monetdb/monetdb:Aug2024 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:250230356 + datadisk:218202266 + volume_size:300G + volume_used:209G + requests_cpu:4 + requests_memory:16Gi +MonetDB-BHT-8-2-1 uses docker image monetdb/monetdb:Aug2024 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:250230364 + datadisk:223241760 + volume_size:300G + volume_used:213G + requests_cpu:4 + requests_memory:16Gi +MonetDB-BHT-8-3-1 uses docker image monetdb/monetdb:Aug2024 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:250062784 + datadisk:223241831 + volume_size:300G + volume_used:213G + requests_cpu:4 + requests_memory:16Gi +MonetDB-BHT-8-3-2 uses docker image monetdb/monetdb:Aug2024 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:250062784 + datadisk:223241831 + volume_size:300G + volume_used:213G + requests_cpu:4 + requests_memory:16Gi +MonetDB-BHT-8-3-3 uses docker image monetdb/monetdb:Aug2024 + RAM:541008576512 + CPU:AMD Opteron(tm) Processor 6378 + Cores:64 + host:5.15.0-126-generic + node:cl-worker11 + disk:250062784 + datadisk:223241831 + volume_size:300G + volume_used:213G + requests_cpu:4 + requests_memory:16Gi + +### Errors (failed queries) + MonetDB-BHT-8-1-1 MonetDB-BHT-8-2-1 MonetDB-BHT-8-3-1 MonetDB-BHT-8-3-2 MonetDB-BHT-8-3-3 +Pricing Summary Report (TPC-H Q1) False False True True True +Discounted Revenue (TPC-H Q19) True True True True True +Pricing Summary Report (TPC-H Q1) +MonetDB-BHT-8-3-2: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded +MonetDB-BHT-8-3-3: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded +MonetDB-BHT-8-3-1: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded +Discounted Revenue (TPC-H Q19) +MonetDB-BHT-8-3-2: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded +MonetDB-BHT-8-3-3: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded +MonetDB-BHT-8-1-1: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded +MonetDB-BHT-8-2-1: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded +MonetDB-BHT-8-3-1: numRun 1: : java.sql.SQLException: GDK reported error: GDKextendf: could not extend file: Disk quota exceeded + +### Warnings (result mismatch) + MonetDB-BHT-8-1-1 MonetDB-BHT-8-2-1 MonetDB-BHT-8-3-1 MonetDB-BHT-8-3-2 MonetDB-BHT-8-3-3 +Pricing Summary Report (TPC-H Q1) True True False False False + +### Latency of Timer Execution [ms] +DBMS MonetDB-BHT-8-1-1 MonetDB-BHT-8-2-1 MonetDB-BHT-8-3-1 MonetDB-BHT-8-3-2 MonetDB-BHT-8-3-3 +Minimum Cost Supplier Query (TPC-H Q2) 3194.11 1670.45 5828.99 366.88 5829.51 +Shipping Priority (TPC-H Q3) 13761.14 13605.94 47417.47 30838.94 47307.44 +Order Priority Checking Query (TPC-H Q4) 12545.57 12306.40 30806.00 11911.52 32084.56 +Local Supplier Volume (TPC-H Q5) 11432.09 10183.79 13922.48 9918.79 12919.25 +Forecasting Revenue Change (TPC-H Q6) 6597.76 4943.78 6681.21 4111.89 6729.92 +Forecasting Revenue Change (TPC-H Q7) 8123.89 3621.12 10699.60 12694.40 10829.30 +National Market Share (TPC-H Q8) 60812.96 35310.95 33701.69 32777.41 34062.49 +Product Type Profit Measure (TPC-H Q9) 17328.66 17680.32 18508.85 19264.58 17902.92 +Forecasting Revenue Change (TPC-H Q10) 27227.63 26825.03 26808.97 27131.83 26433.34 +Important Stock Identification (TPC-H Q11) 1193.79 1322.06 1379.97 670.82 1770.63 +Shipping Modes and Order Priority (TPC-H Q12) 4938.74 4920.75 8202.82 6538.83 8342.99 +Customer Distribution (TPC-H Q13) 106149.62 108491.70 127149.26 111621.90 124587.94 +Forecasting Revenue Change (TPC-H Q14) 8514.29 6729.16 274.97 7634.95 328.72 +Top Supplier Query (TPC-H Q15) 9665.45 6289.91 5467.42 7270.07 5792.61 +Parts/Supplier Relationship (TPC-H Q16) 12033.89 12450.62 11841.51 14016.50 11828.14 +Small-Quantity-Order Revenue (TPC-H Q17) 18140.36 15204.61 15480.68 19180.17 17408.95 +Large Volume Customer (TPC-H Q18) 37795.66 19125.43 24020.87 24815.32 25415.36 +Potential Part Promotion (TPC-H Q20) 4293.23 4181.49 7021.23 6428.42 6791.12 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 17644.97 16196.43 47957.20 42946.58 33884.11 +Global Sales Opportunity Query (TPC-H Q22) 6665.92 6907.86 6349.41 6282.68 7720.72 + +### Loading [s] + timeGenerate timeIngesting timeSchema timeIndex timeLoad +MonetDB-BHT-8-1-1 955.0 1602.0 6.0 5671.0 8250.0 +MonetDB-BHT-8-2-1 955.0 1602.0 6.0 5671.0 8250.0 +MonetDB-BHT-8-3-1 955.0 1602.0 6.0 5671.0 8250.0 +MonetDB-BHT-8-3-2 955.0 1602.0 6.0 5671.0 8250.0 +MonetDB-BHT-8-3-3 955.0 1602.0 6.0 5671.0 8250.0 + +### Geometric Mean of Medians of Timer Run [s] + Geo Times [s] +DBMS +MonetDB-BHT-8-1-1 12.49 +MonetDB-BHT-8-2-1 10.16 +MonetDB-BHT-8-3-1 12.53 +MonetDB-BHT-8-3-2 11.64 +MonetDB-BHT-8-3-3 12.69 + +### Power@Size + Power@Size [~Q/h] +DBMS +MonetDB-BHT-8-1-1 30931.10 +MonetDB-BHT-8-2-1 37542.45 +MonetDB-BHT-8-3-1 30428.55 +MonetDB-BHT-8-3-2 33693.59 +MonetDB-BHT-8-3-3 29802.28 + +### Throughput@Size + time [s] count SF Throughput@Size [~GB/h] +DBMS SF num_experiment num_client +MonetDB-BHT-8-1 100 1 1 628 1 100 12611.46 +MonetDB-BHT-8-2 100 1 2 567 1 100 13968.25 +MonetDB-BHT-8-3 100 1 3 605 3 100 39272.73 + +### Workflow + +#### Actual +DBMS MonetDB-BHT-8 - Pods [[1, 1, 3]] + +#### Planned +DBMS MonetDB-BHT-8 - Pods [[1, 1, 3]] + +### Ingestion - SUT + CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] +MonetDB-BHT-8-1 19913.85 12.72 107.69 205.07 +MonetDB-BHT-8-2 19913.85 12.72 107.69 205.07 +MonetDB-BHT-8-3 19913.85 12.72 107.69 205.07 + +### Ingestion - Loader + CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] +MonetDB-BHT-8-1 2348.27 1.26 0.02 0.02 +MonetDB-BHT-8-2 2348.27 1.26 0.02 0.02 +MonetDB-BHT-8-3 2348.27 1.26 0.02 0.02 + +### Execution - SUT + CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] +MonetDB-BHT-8-1 6101.85 23.08 132.79 260.12 +MonetDB-BHT-8-2 6120.54 11.71 151.97 276.09 +MonetDB-BHT-8-3 11331.73 21.06 183.26 313.77 + +### Execution - Benchmarker + CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] +MonetDB-BHT-8-1 26.05 0.01 0.29 0.30 +MonetDB-BHT-8-2 26.05 0.02 0.52 0.54 +MonetDB-BHT-8-3 59.23 0.08 0.98 0.99 + +### Tests +TEST passed: Geo Times [s] contains no 0 or NaN +TEST passed: Power@Size [~Q/h] contains no 0 or NaN +TEST passed: Throughput@Size [~GB/h] contains no 0 or NaN +TEST failed: SQL errors +TEST failed: SQL warnings (result mismatch) +TEST passed: Ingestion SUT contains no 0 or NaN in CPU [CPUs] +TEST passed: Ingestion Loader contains no 0 or NaN in CPU [CPUs] +TEST passed: Execution SUT contains no 0 or NaN in CPU [CPUs] +TEST passed: Execution Benchmarker contains no 0 or NaN in CPU [CPUs] +TEST passed: Workflow as planned diff --git a/logs_tests/doc_tpch_testcase_databaseservice_1.log b/logs_tests/doc_tpch_testcase_databaseservice_1.log index 64bf8d39..b0ac3c9e 100644 --- a/logs_tests/doc_tpch_testcase_databaseservice_1.log +++ b/logs_tests/doc_tpch_testcase_databaseservice_1.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734427906 -Experiment : starts at 2024-12-17 10:31:46.639270 (4112708.046399269) +Experiment : has code 1734666830 +Experiment : starts at 2024-12-20 04:53:50.730248 (4351632.137378045) Experiment : This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. Query ordering is Q1 - Q22. @@ -31,10 +31,10 @@ DatabaseService-BHT-8 : waits for health check to succeed DatabaseService-BHT-8 : waits for health check to succeed - waiting 30s - : done DatabaseService-BHT-8 : is not loaded yet -DatabaseService-BHT-8 : will start loading but not before 2024-12-17 09:35:17 (that is in 60 secs) +DatabaseService-BHT-8 : will start loading but not before 2024-12-20 03:57:23 (that is in 60 secs) - waiting 30s - : done DatabaseService-BHT-8 : is not loaded yet -DatabaseService-BHT-8 : will start loading but not before 2024-12-17 09:35:17 +DatabaseService-BHT-8 : will start loading but not before 2024-12-20 03:57:23 done DatabaseService-BHT-8 : is not loaded yet DatabaseService-BHT-8 : start asynch loading scripts of type loaded @@ -50,8 +50,8 @@ DatabaseService-BHT-8 : is loading done DatabaseService-BHT-8 : showing loader times DatabaseService-BHT-8 : generator timespan (first to last [s]) = 1 -DatabaseService-BHT-8 : loader timespan (first to last [s]) = 101 -DatabaseService-BHT-8 : total timespan (first to last [s]) = 109 +DatabaseService-BHT-8 : loader timespan (first to last [s]) = 97 +DatabaseService-BHT-8 : total timespan (first to last [s]) = 105 DatabaseService-BHT-8 : start asynch loading scripts of type indexed DatabaseService-BHT-8 : is loading - waiting 30s - : done @@ -66,17 +66,19 @@ DatabaseService-BHT-8 : is loading DatabaseService-BHT-8 : is loading - waiting 30s - : done DatabaseService-BHT-8 : is loading +- waiting 30s - : done +DatabaseService-BHT-8 : is loading done DatabaseService-BHT-8 : benchmarks done 0 of 1. This will be client 1 DatabaseService-BHT-8-1 : start benchmarking Worker pods found: [] -DatabaseService-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1734427906 +DatabaseService-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1734666830 - waiting 10s - : done found DatabaseService-BHT-8-1 : collecting metrics of data generator -DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734427906(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734666830(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} DatabaseService-BHT-8-1 : collecting metrics of data injector -DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734427906(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734666830(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} - waiting 30s - : done DatabaseService-BHT-8 : has running benchmarks - waiting 30s - : done @@ -84,14 +86,14 @@ DatabaseService-BHT-8 : has running benchmarks done DatabaseService-BHT-8 : has running benchmarks DatabaseService-BHT-8-1 : showing benchmarker times -DatabaseService-BHT-8-1 : benchmarker timespan (start to end single container [s]) = 120 -DatabaseService-BHT-8-1 : benchmarker times (start/end per pod and container) = [(1734428503, 1734428579)] -DatabaseService-BHT-8-1 : found and updated times {'benchmarker': [(1734428503, 1734428579)]} +DatabaseService-BHT-8-1 : benchmarker timespan (start to end single container [s]) = 111 +DatabaseService-BHT-8-1 : benchmarker times (start/end per pod and container) = [(1734667451, 1734667528)] +DatabaseService-BHT-8-1 : found and updated times {'benchmarker': [(1734667451, 1734667528)]} DatabaseService-BHT-8-1 : collecting metrics of benchmarker -DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734427906(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734666830(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} done DatabaseService-BHT-8 : can be stopped -Experiment : ends at 2024-12-17 10:44:09.497332 (4113450.904466091) - 742.86s total +Experiment : ends at 2024-12-20 05:06:35.119825 (4352396.526959069) - 764.39s total Join results done! done! Experiment : downloading partial results @@ -103,8 +105,8 @@ Experiment : uploading workload file ### Workload TPC-H Queries SF=3 Type: tpch - Duration: 743s - Code: 1734427906 + Duration: 765s + Code: 1734666830 This includes the reading queries of TPC-H. This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. @@ -130,7 +132,7 @@ DatabaseService-BHT-8-1-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005324 + disk:249256232 datadisk:39348 requests_cpu:4 requests_memory:16Gi @@ -143,47 +145,47 @@ No warnings ### Latency of Timer Execution [ms] DBMS DatabaseService-BHT-8-1-1 -Pricing Summary Report (TPC-H Q1) 5931.50 -Minimum Cost Supplier Query (TPC-H Q2) 1926.02 -Shipping Priority (TPC-H Q3) 2296.96 -Order Priority Checking Query (TPC-H Q4) 2970.31 -Local Supplier Volume (TPC-H Q5) 2109.38 -Forecasting Revenue Change (TPC-H Q6) 1111.81 -Forecasting Revenue Change (TPC-H Q7) 2121.34 -National Market Share (TPC-H Q8) 1342.81 -Product Type Profit Measure (TPC-H Q9) 2880.66 -Forecasting Revenue Change (TPC-H Q10) 2864.12 -Important Stock Identification (TPC-H Q11) 572.12 -Shipping Modes and Order Priority (TPC-H Q12) 2198.12 -Customer Distribution (TPC-H Q13) 5642.77 -Forecasting Revenue Change (TPC-H Q14) 1207.98 -Top Supplier Query (TPC-H Q15) 1347.68 -Parts/Supplier Relationship (TPC-H Q16) 1227.46 -Small-Quantity-Order Revenue (TPC-H Q17) 4907.25 -Large Volume Customer (TPC-H Q18) 19409.20 -Discounted Revenue (TPC-H Q19) 1747.85 -Potential Part Promotion (TPC-H Q20) 1089.56 -Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2588.70 -Global Sales Opportunity Query (TPC-H Q22) 461.17 +Pricing Summary Report (TPC-H Q1) 6184.88 +Minimum Cost Supplier Query (TPC-H Q2) 2121.70 +Shipping Priority (TPC-H Q3) 2477.89 +Order Priority Checking Query (TPC-H Q4) 3120.41 +Local Supplier Volume (TPC-H Q5) 2263.59 +Forecasting Revenue Change (TPC-H Q6) 1158.38 +Forecasting Revenue Change (TPC-H Q7) 2326.20 +National Market Share (TPC-H Q8) 1410.37 +Product Type Profit Measure (TPC-H Q9) 3187.17 +Forecasting Revenue Change (TPC-H Q10) 3063.93 +Important Stock Identification (TPC-H Q11) 561.36 +Shipping Modes and Order Priority (TPC-H Q12) 2457.12 +Customer Distribution (TPC-H Q13) 6562.69 +Forecasting Revenue Change (TPC-H Q14) 1280.33 +Top Supplier Query (TPC-H Q15) 1396.18 +Parts/Supplier Relationship (TPC-H Q16) 1346.35 +Small-Quantity-Order Revenue (TPC-H Q17) 5626.89 +Large Volume Customer (TPC-H Q18) 19220.56 +Discounted Revenue (TPC-H Q19) 1909.47 +Potential Part Promotion (TPC-H Q20) 1216.54 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2784.81 +Global Sales Opportunity Query (TPC-H Q22) 465.76 ### Loading [s] timeGenerate timeIngesting timeSchema timeIndex timeLoad -DatabaseService-BHT-8-1-1 1.0 101.0 1.0 193.0 303.0 +DatabaseService-BHT-8-1-1 1.0 97.0 1.0 216.0 322.0 ### Geometric Mean of Medians of Timer Run [s] Geo Times [s] DBMS -DatabaseService-BHT-8-1-1 2.16 +DatabaseService-BHT-8-1-1 2.32 ### Power@Size Power@Size [~Q/h] DBMS -DatabaseService-BHT-8-1-1 5125.94 +DatabaseService-BHT-8-1-1 4783.05 ### Throughput@Size time [s] count SF Throughput@Size [~GB/h] DBMS SF num_experiment num_client -DatabaseService-BHT-8-1 3 1 1 76 1 3 3126.32 +DatabaseService-BHT-8-1 3 1 1 77 1 3 3085.71 ### Workflow @@ -195,11 +197,11 @@ DBMS DatabaseService-BHT-8 - Pods [[1]] ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 30.75 0.14 0.02 2.27 +DatabaseService-BHT-8-1 30.9 0.21 0.03 2.27 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 15.79 0 0.25 0.26 +DatabaseService-BHT-8-1 11.41 0 0.23 0.24 ### Tests TEST passed: Geo Times [s] contains no 0 or NaN diff --git a/logs_tests/doc_tpch_testcase_databaseservice_1_summary.txt b/logs_tests/doc_tpch_testcase_databaseservice_1_summary.txt index 04b9e205..085d01b3 100644 --- a/logs_tests/doc_tpch_testcase_databaseservice_1_summary.txt +++ b/logs_tests/doc_tpch_testcase_databaseservice_1_summary.txt @@ -3,8 +3,8 @@ ### Workload TPC-H Queries SF=3 Type: tpch - Duration: 743s - Code: 1734427906 + Duration: 765s + Code: 1734666830 This includes the reading queries of TPC-H. This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. @@ -30,7 +30,7 @@ DatabaseService-BHT-8-1-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005324 + disk:249256232 datadisk:39348 requests_cpu:4 requests_memory:16Gi @@ -43,47 +43,47 @@ No warnings ### Latency of Timer Execution [ms] DBMS DatabaseService-BHT-8-1-1 -Pricing Summary Report (TPC-H Q1) 5931.50 -Minimum Cost Supplier Query (TPC-H Q2) 1926.02 -Shipping Priority (TPC-H Q3) 2296.96 -Order Priority Checking Query (TPC-H Q4) 2970.31 -Local Supplier Volume (TPC-H Q5) 2109.38 -Forecasting Revenue Change (TPC-H Q6) 1111.81 -Forecasting Revenue Change (TPC-H Q7) 2121.34 -National Market Share (TPC-H Q8) 1342.81 -Product Type Profit Measure (TPC-H Q9) 2880.66 -Forecasting Revenue Change (TPC-H Q10) 2864.12 -Important Stock Identification (TPC-H Q11) 572.12 -Shipping Modes and Order Priority (TPC-H Q12) 2198.12 -Customer Distribution (TPC-H Q13) 5642.77 -Forecasting Revenue Change (TPC-H Q14) 1207.98 -Top Supplier Query (TPC-H Q15) 1347.68 -Parts/Supplier Relationship (TPC-H Q16) 1227.46 -Small-Quantity-Order Revenue (TPC-H Q17) 4907.25 -Large Volume Customer (TPC-H Q18) 19409.20 -Discounted Revenue (TPC-H Q19) 1747.85 -Potential Part Promotion (TPC-H Q20) 1089.56 -Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2588.70 -Global Sales Opportunity Query (TPC-H Q22) 461.17 +Pricing Summary Report (TPC-H Q1) 6184.88 +Minimum Cost Supplier Query (TPC-H Q2) 2121.70 +Shipping Priority (TPC-H Q3) 2477.89 +Order Priority Checking Query (TPC-H Q4) 3120.41 +Local Supplier Volume (TPC-H Q5) 2263.59 +Forecasting Revenue Change (TPC-H Q6) 1158.38 +Forecasting Revenue Change (TPC-H Q7) 2326.20 +National Market Share (TPC-H Q8) 1410.37 +Product Type Profit Measure (TPC-H Q9) 3187.17 +Forecasting Revenue Change (TPC-H Q10) 3063.93 +Important Stock Identification (TPC-H Q11) 561.36 +Shipping Modes and Order Priority (TPC-H Q12) 2457.12 +Customer Distribution (TPC-H Q13) 6562.69 +Forecasting Revenue Change (TPC-H Q14) 1280.33 +Top Supplier Query (TPC-H Q15) 1396.18 +Parts/Supplier Relationship (TPC-H Q16) 1346.35 +Small-Quantity-Order Revenue (TPC-H Q17) 5626.89 +Large Volume Customer (TPC-H Q18) 19220.56 +Discounted Revenue (TPC-H Q19) 1909.47 +Potential Part Promotion (TPC-H Q20) 1216.54 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2784.81 +Global Sales Opportunity Query (TPC-H Q22) 465.76 ### Loading [s] timeGenerate timeIngesting timeSchema timeIndex timeLoad -DatabaseService-BHT-8-1-1 1.0 101.0 1.0 193.0 303.0 +DatabaseService-BHT-8-1-1 1.0 97.0 1.0 216.0 322.0 ### Geometric Mean of Medians of Timer Run [s] Geo Times [s] DBMS -DatabaseService-BHT-8-1-1 2.16 +DatabaseService-BHT-8-1-1 2.32 ### Power@Size Power@Size [~Q/h] DBMS -DatabaseService-BHT-8-1-1 5125.94 +DatabaseService-BHT-8-1-1 4783.05 ### Throughput@Size time [s] count SF Throughput@Size [~GB/h] DBMS SF num_experiment num_client -DatabaseService-BHT-8-1 3 1 1 76 1 3 3126.32 +DatabaseService-BHT-8-1 3 1 1 77 1 3 3085.71 ### Workflow @@ -95,11 +95,11 @@ DBMS DatabaseService-BHT-8 - Pods [[1]] ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 30.75 0.14 0.02 2.27 +DatabaseService-BHT-8-1 30.9 0.21 0.03 2.27 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 15.79 0 0.25 0.26 +DatabaseService-BHT-8-1 11.41 0 0.23 0.24 ### Tests TEST passed: Geo Times [s] contains no 0 or NaN diff --git a/logs_tests/doc_tpch_testcase_databaseservice_2.log b/logs_tests/doc_tpch_testcase_databaseservice_2.log index 405fa922..937a12c5 100644 --- a/logs_tests/doc_tpch_testcase_databaseservice_2.log +++ b/logs_tests/doc_tpch_testcase_databaseservice_2.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734428687 -Experiment : starts at 2024-12-17 10:44:46.975852 (4113488.382981685) +Experiment : has code 1734667671 +Experiment : starts at 2024-12-20 05:07:50.927083 (4352472.334216012) Experiment : This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. Query ordering is Q1 - Q22. @@ -25,16 +25,16 @@ Cluster monitoring : is running done DatabaseService-BHT-8 : will start now - waiting 30s - : done -DatabaseService-BHT-8 : will start benchmarking but not before 2024-12-17 09:46:47 (that is in 60 secs) +DatabaseService-BHT-8 : will start benchmarking but not before 2024-12-20 04:09:51 (that is in 60 secs) - waiting 30s - : done -DatabaseService-BHT-8 : will start benchmarking but not before 2024-12-17 09:46:47 +DatabaseService-BHT-8 : will start benchmarking but not before 2024-12-20 04:09:51 done DatabaseService-BHT-8 : benchmarks done 0 of 1. This will be client 1 DatabaseService-BHT-8-1 : start benchmarking Worker pods found: [] -DatabaseService-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1734428687 +DatabaseService-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1734667671 - waiting 10s - : done -DatabaseService-BHT-8 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-bht-8-1734428687-1-1-47g8z: found +DatabaseService-BHT-8 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-bht-8-1734667671-1-1-q9wg9: found - waiting 30s - : done DatabaseService-BHT-8 : has running benchmarks - waiting 30s - : done @@ -43,13 +43,13 @@ done DatabaseService-BHT-8 : has running benchmarks DatabaseService-BHT-8-1 : showing benchmarker times DatabaseService-BHT-8-1 : benchmarker timespan (start to end single container [s]) = 101 -DatabaseService-BHT-8-1 : benchmarker times (start/end per pod and container) = [(1734428820, 1734428894)] -DatabaseService-BHT-8-1 : found and updated times {'benchmarker': [(1734428820, 1734428894)]} +DatabaseService-BHT-8-1 : benchmarker times (start/end per pod and container) = [(1734667800, 1734667876)] +DatabaseService-BHT-8-1 : found and updated times {'benchmarker': [(1734667800, 1734667876)]} DatabaseService-BHT-8-1 : collecting metrics of benchmarker -DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734428687(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734667671(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} done DatabaseService-BHT-8 : can be stopped -Experiment : ends at 2024-12-17 10:49:08.204995 (4113749.612128884) - 261.23s total +Experiment : ends at 2024-12-20 05:12:12.416402 (4352733.823536302) - 261.49s total Join results done! done! Experiment : downloading partial results @@ -62,7 +62,7 @@ Experiment : uploading workload file TPC-H Queries SF=3 Type: tpch Duration: 262s - Code: 1734428687 + Code: 1734667671 This includes the reading queries of TPC-H. This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. @@ -89,7 +89,7 @@ DatabaseService-BHT-8-1-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005304 + disk:249256216 datadisk:39348 requests_cpu:4 requests_memory:16Gi @@ -102,28 +102,28 @@ No warnings ### Latency of Timer Execution [ms] DBMS DatabaseService-BHT-8-1-1 -Pricing Summary Report (TPC-H Q1) 5799.63 -Minimum Cost Supplier Query (TPC-H Q2) 1945.49 -Shipping Priority (TPC-H Q3) 2259.57 -Order Priority Checking Query (TPC-H Q4) 2853.45 -Local Supplier Volume (TPC-H Q5) 2054.87 -Forecasting Revenue Change (TPC-H Q6) 1110.50 -Forecasting Revenue Change (TPC-H Q7) 2097.45 -National Market Share (TPC-H Q8) 1330.24 -Product Type Profit Measure (TPC-H Q9) 2838.46 -Forecasting Revenue Change (TPC-H Q10) 2848.10 -Important Stock Identification (TPC-H Q11) 571.67 -Shipping Modes and Order Priority (TPC-H Q12) 2222.90 -Customer Distribution (TPC-H Q13) 5663.70 -Forecasting Revenue Change (TPC-H Q14) 1178.77 -Top Supplier Query (TPC-H Q15) 1409.79 -Parts/Supplier Relationship (TPC-H Q16) 1229.45 -Small-Quantity-Order Revenue (TPC-H Q17) 5089.27 -Large Volume Customer (TPC-H Q18) 20425.52 -Discounted Revenue (TPC-H Q19) 1768.51 -Potential Part Promotion (TPC-H Q20) 1041.16 -Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2462.41 -Global Sales Opportunity Query (TPC-H Q22) 446.75 +Pricing Summary Report (TPC-H Q1) 6149.40 +Minimum Cost Supplier Query (TPC-H Q2) 2110.09 +Shipping Priority (TPC-H Q3) 2435.87 +Order Priority Checking Query (TPC-H Q4) 3075.44 +Local Supplier Volume (TPC-H Q5) 2234.95 +Forecasting Revenue Change (TPC-H Q6) 1171.11 +Forecasting Revenue Change (TPC-H Q7) 2288.60 +National Market Share (TPC-H Q8) 1388.84 +Product Type Profit Measure (TPC-H Q9) 3168.23 +Forecasting Revenue Change (TPC-H Q10) 3075.63 +Important Stock Identification (TPC-H Q11) 563.29 +Shipping Modes and Order Priority (TPC-H Q12) 2453.85 +Customer Distribution (TPC-H Q13) 6242.59 +Forecasting Revenue Change (TPC-H Q14) 1271.74 +Top Supplier Query (TPC-H Q15) 1382.80 +Parts/Supplier Relationship (TPC-H Q16) 1349.50 +Small-Quantity-Order Revenue (TPC-H Q17) 5621.15 +Large Volume Customer (TPC-H Q18) 18750.06 +Discounted Revenue (TPC-H Q19) 1919.85 +Potential Part Promotion (TPC-H Q20) 1131.92 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2704.33 +Global Sales Opportunity Query (TPC-H Q22) 444.20 ### Loading [s] timeGenerate timeIngesting timeSchema timeIndex timeLoad @@ -132,17 +132,17 @@ DatabaseService-BHT-8-1-1 0 0 0 0 ### Geometric Mean of Medians of Timer Run [s] Geo Times [s] DBMS -DatabaseService-BHT-8-1-1 2.16 +DatabaseService-BHT-8-1-1 2.29 ### Power@Size Power@Size [~Q/h] DBMS -DatabaseService-BHT-8-1-1 5156.12 +DatabaseService-BHT-8-1-1 4850.83 ### Throughput@Size time [s] count SF Throughput@Size [~GB/h] DBMS SF num_experiment num_client -DatabaseService-BHT-8-1 3 1 1 74 1 3 3210.81 +DatabaseService-BHT-8-1 3 1 1 76 1 3 3126.32 ### Workflow @@ -154,7 +154,7 @@ DBMS DatabaseService-BHT-8 - Pods [[1]] ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 15.5 0 0.26 0.28 +DatabaseService-BHT-8-1 11.36 0 0.22 0.24 ### Tests TEST passed: Geo Times [s] contains no 0 or NaN diff --git a/logs_tests/doc_tpch_testcase_databaseservice_2_summary.txt b/logs_tests/doc_tpch_testcase_databaseservice_2_summary.txt index 6a295b88..e42c05b3 100644 --- a/logs_tests/doc_tpch_testcase_databaseservice_2_summary.txt +++ b/logs_tests/doc_tpch_testcase_databaseservice_2_summary.txt @@ -4,7 +4,7 @@ TPC-H Queries SF=3 Type: tpch Duration: 262s - Code: 1734428687 + Code: 1734667671 This includes the reading queries of TPC-H. This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. @@ -31,7 +31,7 @@ DatabaseService-BHT-8-1-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236005304 + disk:249256216 datadisk:39348 requests_cpu:4 requests_memory:16Gi @@ -44,28 +44,28 @@ No warnings ### Latency of Timer Execution [ms] DBMS DatabaseService-BHT-8-1-1 -Pricing Summary Report (TPC-H Q1) 5799.63 -Minimum Cost Supplier Query (TPC-H Q2) 1945.49 -Shipping Priority (TPC-H Q3) 2259.57 -Order Priority Checking Query (TPC-H Q4) 2853.45 -Local Supplier Volume (TPC-H Q5) 2054.87 -Forecasting Revenue Change (TPC-H Q6) 1110.50 -Forecasting Revenue Change (TPC-H Q7) 2097.45 -National Market Share (TPC-H Q8) 1330.24 -Product Type Profit Measure (TPC-H Q9) 2838.46 -Forecasting Revenue Change (TPC-H Q10) 2848.10 -Important Stock Identification (TPC-H Q11) 571.67 -Shipping Modes and Order Priority (TPC-H Q12) 2222.90 -Customer Distribution (TPC-H Q13) 5663.70 -Forecasting Revenue Change (TPC-H Q14) 1178.77 -Top Supplier Query (TPC-H Q15) 1409.79 -Parts/Supplier Relationship (TPC-H Q16) 1229.45 -Small-Quantity-Order Revenue (TPC-H Q17) 5089.27 -Large Volume Customer (TPC-H Q18) 20425.52 -Discounted Revenue (TPC-H Q19) 1768.51 -Potential Part Promotion (TPC-H Q20) 1041.16 -Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2462.41 -Global Sales Opportunity Query (TPC-H Q22) 446.75 +Pricing Summary Report (TPC-H Q1) 6149.40 +Minimum Cost Supplier Query (TPC-H Q2) 2110.09 +Shipping Priority (TPC-H Q3) 2435.87 +Order Priority Checking Query (TPC-H Q4) 3075.44 +Local Supplier Volume (TPC-H Q5) 2234.95 +Forecasting Revenue Change (TPC-H Q6) 1171.11 +Forecasting Revenue Change (TPC-H Q7) 2288.60 +National Market Share (TPC-H Q8) 1388.84 +Product Type Profit Measure (TPC-H Q9) 3168.23 +Forecasting Revenue Change (TPC-H Q10) 3075.63 +Important Stock Identification (TPC-H Q11) 563.29 +Shipping Modes and Order Priority (TPC-H Q12) 2453.85 +Customer Distribution (TPC-H Q13) 6242.59 +Forecasting Revenue Change (TPC-H Q14) 1271.74 +Top Supplier Query (TPC-H Q15) 1382.80 +Parts/Supplier Relationship (TPC-H Q16) 1349.50 +Small-Quantity-Order Revenue (TPC-H Q17) 5621.15 +Large Volume Customer (TPC-H Q18) 18750.06 +Discounted Revenue (TPC-H Q19) 1919.85 +Potential Part Promotion (TPC-H Q20) 1131.92 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2704.33 +Global Sales Opportunity Query (TPC-H Q22) 444.20 ### Loading [s] timeGenerate timeIngesting timeSchema timeIndex timeLoad @@ -74,17 +74,17 @@ DatabaseService-BHT-8-1-1 0 0 0 0 ### Geometric Mean of Medians of Timer Run [s] Geo Times [s] DBMS -DatabaseService-BHT-8-1-1 2.16 +DatabaseService-BHT-8-1-1 2.29 ### Power@Size Power@Size [~Q/h] DBMS -DatabaseService-BHT-8-1-1 5156.12 +DatabaseService-BHT-8-1-1 4850.83 ### Throughput@Size time [s] count SF Throughput@Size [~GB/h] DBMS SF num_experiment num_client -DatabaseService-BHT-8-1 3 1 1 74 1 3 3210.81 +DatabaseService-BHT-8-1 3 1 1 76 1 3 3126.32 ### Workflow @@ -96,7 +96,7 @@ DBMS DatabaseService-BHT-8 - Pods [[1]] ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 15.5 0 0.26 0.28 +DatabaseService-BHT-8-1 11.36 0 0.22 0.24 ### Tests TEST passed: Geo Times [s] contains no 0 or NaN diff --git a/logs_tests/doc_tpch_testcase_databaseservice_3.log b/logs_tests/doc_tpch_testcase_databaseservice_3.log index 5ef4e9ca..b5d73171 100644 --- a/logs_tests/doc_tpch_testcase_databaseservice_3.log +++ b/logs_tests/doc_tpch_testcase_databaseservice_3.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734429037 -Experiment : starts at 2024-12-17 10:50:37.542866 (4113838.949996419) +Experiment : has code 1734668021 +Experiment : starts at 2024-12-20 05:13:41.371484 (4352822.778614899) Experiment : This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. Query ordering is Q1 - Q22. @@ -34,10 +34,10 @@ DatabaseService-BHT-8 : waits for health check to succeed DatabaseService-BHT-8 : waits for health check to succeed - waiting 30s - : done DatabaseService-BHT-8 : is not loaded yet -DatabaseService-BHT-8 : will start loading but not before 2024-12-17 09:54:39 (that is in 60 secs) +DatabaseService-BHT-8 : will start loading but not before 2024-12-20 04:17:43 (that is in 60 secs) - waiting 30s - : done DatabaseService-BHT-8 : is not loaded yet -DatabaseService-BHT-8 : will start loading but not before 2024-12-17 09:54:39 +DatabaseService-BHT-8 : will start loading but not before 2024-12-20 04:17:43 done DatabaseService-BHT-8 : is not loaded yet DatabaseService-BHT-8 : start asynch loading scripts of type loaded @@ -55,8 +55,8 @@ DatabaseService-BHT-8 : is loading done DatabaseService-BHT-8 : showing loader times DatabaseService-BHT-8 : generator timespan (first to last [s]) = 1 -DatabaseService-BHT-8 : loader timespan (first to last [s]) = 122 -DatabaseService-BHT-8 : total timespan (first to last [s]) = 131 +DatabaseService-BHT-8 : loader timespan (first to last [s]) = 123 +DatabaseService-BHT-8 : total timespan (first to last [s]) = 129 DatabaseService-BHT-8 : start asynch loading scripts of type indexed DatabaseService-BHT-8 : is loading - waiting 30s - : done @@ -75,13 +75,13 @@ done DatabaseService-BHT-8 : benchmarks done 0 of 1. This will be client 1 DatabaseService-BHT-8-1 : start benchmarking Worker pods found: [] -DatabaseService-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1734429037 +DatabaseService-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1734668021 - waiting 10s - : done found DatabaseService-BHT-8-1 : collecting metrics of data generator -DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734429037(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734668021(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} DatabaseService-BHT-8-1 : collecting metrics of data injector -DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734429037(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734668021(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} - waiting 30s - : done DatabaseService-BHT-8 : has running benchmarks - waiting 30s - : done @@ -89,14 +89,14 @@ DatabaseService-BHT-8 : has running benchmarks done DatabaseService-BHT-8 : has running benchmarks DatabaseService-BHT-8-1 : showing benchmarker times -DatabaseService-BHT-8-1 : benchmarker timespan (start to end single container [s]) = 110 -DatabaseService-BHT-8-1 : benchmarker times (start/end per pod and container) = [(1734429688, 1734429761)] -DatabaseService-BHT-8-1 : found and updated times {'benchmarker': [(1734429688, 1734429761)]} +DatabaseService-BHT-8-1 : benchmarker timespan (start to end single container [s]) = 112 +DatabaseService-BHT-8-1 : benchmarker times (start/end per pod and container) = [(1734668681, 1734668751)] +DatabaseService-BHT-8-1 : found and updated times {'benchmarker': [(1734668681, 1734668751)]} DatabaseService-BHT-8-1 : collecting metrics of benchmarker -DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734429037(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734668021(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} done DatabaseService-BHT-8 : can be stopped -Experiment : ends at 2024-12-17 11:03:52.900062 (4114634.307195308) - 795.36s total +Experiment : ends at 2024-12-20 05:26:59.025144 (4353620.432277678) - 797.65s total Join results done! done! Experiment : downloading partial results @@ -108,8 +108,8 @@ Experiment : uploading workload file ### Workload TPC-H Queries SF=3 Type: tpch - Duration: 796s - Code: 1734429037 + Duration: 798s + Code: 1734668021 This includes the reading queries of TPC-H. This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. @@ -136,7 +136,7 @@ DatabaseService-BHT-8-1-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:235965984 + disk:249216892 datadisk:39192 volume_size:1.0G volume_used:36M @@ -151,47 +151,47 @@ No warnings ### Latency of Timer Execution [ms] DBMS DatabaseService-BHT-8-1-1 -Pricing Summary Report (TPC-H Q1) 5744.14 -Minimum Cost Supplier Query (TPC-H Q2) 2022.89 -Shipping Priority (TPC-H Q3) 2325.90 -Order Priority Checking Query (TPC-H Q4) 2921.01 -Local Supplier Volume (TPC-H Q5) 2085.64 -Forecasting Revenue Change (TPC-H Q6) 1132.48 -Forecasting Revenue Change (TPC-H Q7) 2157.45 -National Market Share (TPC-H Q8) 1333.06 -Product Type Profit Measure (TPC-H Q9) 2966.91 -Forecasting Revenue Change (TPC-H Q10) 2810.67 -Important Stock Identification (TPC-H Q11) 589.44 -Shipping Modes and Order Priority (TPC-H Q12) 2227.10 -Customer Distribution (TPC-H Q13) 5779.52 -Forecasting Revenue Change (TPC-H Q14) 1224.41 -Top Supplier Query (TPC-H Q15) 1354.01 -Parts/Supplier Relationship (TPC-H Q16) 1196.48 -Small-Quantity-Order Revenue (TPC-H Q17) 5286.03 -Large Volume Customer (TPC-H Q18) 17941.22 -Discounted Revenue (TPC-H Q19) 1796.70 -Potential Part Promotion (TPC-H Q20) 1085.69 -Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2513.56 -Global Sales Opportunity Query (TPC-H Q22) 480.80 +Pricing Summary Report (TPC-H Q1) 5793.75 +Minimum Cost Supplier Query (TPC-H Q2) 1922.31 +Shipping Priority (TPC-H Q3) 2231.66 +Order Priority Checking Query (TPC-H Q4) 2835.42 +Local Supplier Volume (TPC-H Q5) 2072.36 +Forecasting Revenue Change (TPC-H Q6) 1066.68 +Forecasting Revenue Change (TPC-H Q7) 2155.03 +National Market Share (TPC-H Q8) 1301.26 +Product Type Profit Measure (TPC-H Q9) 2787.60 +Forecasting Revenue Change (TPC-H Q10) 2791.42 +Important Stock Identification (TPC-H Q11) 559.95 +Shipping Modes and Order Priority (TPC-H Q12) 2143.46 +Customer Distribution (TPC-H Q13) 5364.32 +Forecasting Revenue Change (TPC-H Q14) 1190.47 +Top Supplier Query (TPC-H Q15) 1291.81 +Parts/Supplier Relationship (TPC-H Q16) 1127.44 +Small-Quantity-Order Revenue (TPC-H Q17) 4912.21 +Large Volume Customer (TPC-H Q18) 16174.10 +Discounted Revenue (TPC-H Q19) 1735.84 +Potential Part Promotion (TPC-H Q20) 1088.33 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2474.25 +Global Sales Opportunity Query (TPC-H Q22) 488.84 ### Loading [s] timeGenerate timeIngesting timeSchema timeIndex timeLoad -DatabaseService-BHT-8-1-1 1.0 122.0 1.0 198.0 330.0 +DatabaseService-BHT-8-1-1 1.0 123.0 1.0 196.0 326.0 ### Geometric Mean of Medians of Timer Run [s] Geo Times [s] DBMS -DatabaseService-BHT-8-1-1 2.17 +DatabaseService-BHT-8-1-1 2.11 ### Power@Size Power@Size [~Q/h] DBMS -DatabaseService-BHT-8-1-1 5095.38 +DatabaseService-BHT-8-1-1 5279.56 ### Throughput@Size time [s] count SF Throughput@Size [~GB/h] DBMS SF num_experiment num_client -DatabaseService-BHT-8-1 3 1 1 73 1 3 3254.79 +DatabaseService-BHT-8-1 3 1 1 70 1 3 3394.29 ### Workflow @@ -203,11 +203,11 @@ DBMS DatabaseService-BHT-8 - Pods [[1]] ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 40.99 0.07 0.03 2.8 +DatabaseService-BHT-8-1 41.77 0.04 0.02 2.8 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 11.61 0 0.22 0.23 +DatabaseService-BHT-8-1 15.95 0 0.26 0.27 ### Tests TEST passed: Geo Times [s] contains no 0 or NaN diff --git a/logs_tests/doc_tpch_testcase_databaseservice_3_summary.txt b/logs_tests/doc_tpch_testcase_databaseservice_3_summary.txt index 36dbbc80..8b15671e 100644 --- a/logs_tests/doc_tpch_testcase_databaseservice_3_summary.txt +++ b/logs_tests/doc_tpch_testcase_databaseservice_3_summary.txt @@ -3,8 +3,8 @@ ### Workload TPC-H Queries SF=3 Type: tpch - Duration: 796s - Code: 1734429037 + Duration: 798s + Code: 1734668021 This includes the reading queries of TPC-H. This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. @@ -31,7 +31,7 @@ DatabaseService-BHT-8-1-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:235965984 + disk:249216892 datadisk:39192 volume_size:1.0G volume_used:36M @@ -46,47 +46,47 @@ No warnings ### Latency of Timer Execution [ms] DBMS DatabaseService-BHT-8-1-1 -Pricing Summary Report (TPC-H Q1) 5744.14 -Minimum Cost Supplier Query (TPC-H Q2) 2022.89 -Shipping Priority (TPC-H Q3) 2325.90 -Order Priority Checking Query (TPC-H Q4) 2921.01 -Local Supplier Volume (TPC-H Q5) 2085.64 -Forecasting Revenue Change (TPC-H Q6) 1132.48 -Forecasting Revenue Change (TPC-H Q7) 2157.45 -National Market Share (TPC-H Q8) 1333.06 -Product Type Profit Measure (TPC-H Q9) 2966.91 -Forecasting Revenue Change (TPC-H Q10) 2810.67 -Important Stock Identification (TPC-H Q11) 589.44 -Shipping Modes and Order Priority (TPC-H Q12) 2227.10 -Customer Distribution (TPC-H Q13) 5779.52 -Forecasting Revenue Change (TPC-H Q14) 1224.41 -Top Supplier Query (TPC-H Q15) 1354.01 -Parts/Supplier Relationship (TPC-H Q16) 1196.48 -Small-Quantity-Order Revenue (TPC-H Q17) 5286.03 -Large Volume Customer (TPC-H Q18) 17941.22 -Discounted Revenue (TPC-H Q19) 1796.70 -Potential Part Promotion (TPC-H Q20) 1085.69 -Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2513.56 -Global Sales Opportunity Query (TPC-H Q22) 480.80 +Pricing Summary Report (TPC-H Q1) 5793.75 +Minimum Cost Supplier Query (TPC-H Q2) 1922.31 +Shipping Priority (TPC-H Q3) 2231.66 +Order Priority Checking Query (TPC-H Q4) 2835.42 +Local Supplier Volume (TPC-H Q5) 2072.36 +Forecasting Revenue Change (TPC-H Q6) 1066.68 +Forecasting Revenue Change (TPC-H Q7) 2155.03 +National Market Share (TPC-H Q8) 1301.26 +Product Type Profit Measure (TPC-H Q9) 2787.60 +Forecasting Revenue Change (TPC-H Q10) 2791.42 +Important Stock Identification (TPC-H Q11) 559.95 +Shipping Modes and Order Priority (TPC-H Q12) 2143.46 +Customer Distribution (TPC-H Q13) 5364.32 +Forecasting Revenue Change (TPC-H Q14) 1190.47 +Top Supplier Query (TPC-H Q15) 1291.81 +Parts/Supplier Relationship (TPC-H Q16) 1127.44 +Small-Quantity-Order Revenue (TPC-H Q17) 4912.21 +Large Volume Customer (TPC-H Q18) 16174.10 +Discounted Revenue (TPC-H Q19) 1735.84 +Potential Part Promotion (TPC-H Q20) 1088.33 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2474.25 +Global Sales Opportunity Query (TPC-H Q22) 488.84 ### Loading [s] timeGenerate timeIngesting timeSchema timeIndex timeLoad -DatabaseService-BHT-8-1-1 1.0 122.0 1.0 198.0 330.0 +DatabaseService-BHT-8-1-1 1.0 123.0 1.0 196.0 326.0 ### Geometric Mean of Medians of Timer Run [s] Geo Times [s] DBMS -DatabaseService-BHT-8-1-1 2.17 +DatabaseService-BHT-8-1-1 2.11 ### Power@Size Power@Size [~Q/h] DBMS -DatabaseService-BHT-8-1-1 5095.38 +DatabaseService-BHT-8-1-1 5279.56 ### Throughput@Size time [s] count SF Throughput@Size [~GB/h] DBMS SF num_experiment num_client -DatabaseService-BHT-8-1 3 1 1 73 1 3 3254.79 +DatabaseService-BHT-8-1 3 1 1 70 1 3 3394.29 ### Workflow @@ -98,11 +98,11 @@ DBMS DatabaseService-BHT-8 - Pods [[1]] ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 40.99 0.07 0.03 2.8 +DatabaseService-BHT-8-1 41.77 0.04 0.02 2.8 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 11.61 0 0.22 0.23 +DatabaseService-BHT-8-1 15.95 0 0.26 0.27 ### Tests TEST passed: Geo Times [s] contains no 0 or NaN diff --git a/logs_tests/doc_tpch_testcase_databaseservice_4.log b/logs_tests/doc_tpch_testcase_databaseservice_4.log index f7896e19..df9c06c9 100644 --- a/logs_tests/doc_tpch_testcase_databaseservice_4.log +++ b/logs_tests/doc_tpch_testcase_databaseservice_4.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734429878 -Experiment : starts at 2024-12-17 11:04:37.797250 (4114679.204379662) +Experiment : has code 1734668861 +Experiment : starts at 2024-12-20 05:27:41.707337 (4353663.114466471) Experiment : This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. Query ordering is Q1 - Q22. @@ -27,16 +27,16 @@ DatabaseService-BHT-8 : will start now DatabaseService-BHT-8 : storage exists bexhoma-storage-databaseservice-tpch-3 DatabaseService-BHT-8 : loading is set to finished - waiting 30s - : done -DatabaseService-BHT-8 : will start benchmarking but not before 2024-12-17 10:06:38 (that is in 60 secs) +DatabaseService-BHT-8 : will start benchmarking but not before 2024-12-20 04:29:42 (that is in 60 secs) - waiting 30s - : done -DatabaseService-BHT-8 : will start benchmarking but not before 2024-12-17 10:06:38 +DatabaseService-BHT-8 : will start benchmarking but not before 2024-12-20 04:29:42 done DatabaseService-BHT-8 : benchmarks done 0 of 1. This will be client 1 DatabaseService-BHT-8-1 : start benchmarking Worker pods found: [] -DatabaseService-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1734429878 +DatabaseService-BHT-8 : benchmarking results in folder /home/perdelt/benchmarks/1734668861 - waiting 10s - : done -DatabaseService-BHT-8 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-bht-8-1734429878-1-1-qjbmg: found +DatabaseService-BHT-8 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-bht-8-1734668861-1-1-8l88l: .found - waiting 30s - : done DatabaseService-BHT-8 : has running benchmarks - waiting 30s - : done @@ -44,14 +44,14 @@ DatabaseService-BHT-8 : has running benchmarks done DatabaseService-BHT-8 : has running benchmarks DatabaseService-BHT-8-1 : showing benchmarker times -DatabaseService-BHT-8-1 : benchmarker timespan (start to end single container [s]) = 101 -DatabaseService-BHT-8-1 : benchmarker times (start/end per pod and container) = [(1734430005, 1734430075)] -DatabaseService-BHT-8-1 : found and updated times {'benchmarker': [(1734430005, 1734430075)]} +DatabaseService-BHT-8-1 : benchmarker timespan (start to end single container [s]) = 112 +DatabaseService-BHT-8-1 : benchmarker times (start/end per pod and container) = [(1734668997, 1734669068)] +DatabaseService-BHT-8-1 : found and updated times {'benchmarker': [(1734668997, 1734669068)]} DatabaseService-BHT-8-1 : collecting metrics of benchmarker -DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734429878(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +DatabaseService-BHT-8-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-bht-8-1734668861(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} done DatabaseService-BHT-8 : can be stopped -Experiment : ends at 2024-12-17 11:08:59.534137 (4114940.941270512) - 261.74s total +Experiment : ends at 2024-12-20 05:32:13.728204 (4353935.135337864) - 272.02s total Join results done! done! Experiment : downloading partial results @@ -63,8 +63,8 @@ Experiment : uploading workload file ### Workload TPC-H Queries SF=3 Type: tpch - Duration: 262s - Code: 1734429878 + Duration: 273s + Code: 1734668861 This includes the reading queries of TPC-H. This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. @@ -91,7 +91,7 @@ DatabaseService-BHT-8-1-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:235965964 + disk:249216876 datadisk:39192 volume_size:1.0G volume_used:36M @@ -106,47 +106,47 @@ No warnings ### Latency of Timer Execution [ms] DBMS DatabaseService-BHT-8-1-1 -Pricing Summary Report (TPC-H Q1) 5713.84 -Minimum Cost Supplier Query (TPC-H Q2) 1886.65 -Shipping Priority (TPC-H Q3) 2253.28 -Order Priority Checking Query (TPC-H Q4) 2854.45 -Local Supplier Volume (TPC-H Q5) 2075.71 -Forecasting Revenue Change (TPC-H Q6) 1101.50 -Forecasting Revenue Change (TPC-H Q7) 2145.88 -National Market Share (TPC-H Q8) 1332.73 -Product Type Profit Measure (TPC-H Q9) 2925.21 -Forecasting Revenue Change (TPC-H Q10) 2867.79 -Important Stock Identification (TPC-H Q11) 564.90 -Shipping Modes and Order Priority (TPC-H Q12) 2189.70 -Customer Distribution (TPC-H Q13) 5426.61 -Forecasting Revenue Change (TPC-H Q14) 1194.76 -Top Supplier Query (TPC-H Q15) 1364.10 -Parts/Supplier Relationship (TPC-H Q16) 1319.02 -Small-Quantity-Order Revenue (TPC-H Q17) 5020.23 -Large Volume Customer (TPC-H Q18) 16806.97 -Discounted Revenue (TPC-H Q19) 1775.99 -Potential Part Promotion (TPC-H Q20) 1037.63 -Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2513.75 -Global Sales Opportunity Query (TPC-H Q22) 475.99 +Pricing Summary Report (TPC-H Q1) 5785.36 +Minimum Cost Supplier Query (TPC-H Q2) 1938.19 +Shipping Priority (TPC-H Q3) 2255.59 +Order Priority Checking Query (TPC-H Q4) 2827.21 +Local Supplier Volume (TPC-H Q5) 2018.18 +Forecasting Revenue Change (TPC-H Q6) 1062.37 +Forecasting Revenue Change (TPC-H Q7) 2075.73 +National Market Share (TPC-H Q8) 1336.07 +Product Type Profit Measure (TPC-H Q9) 2808.84 +Forecasting Revenue Change (TPC-H Q10) 2804.71 +Important Stock Identification (TPC-H Q11) 583.68 +Shipping Modes and Order Priority (TPC-H Q12) 2137.04 +Customer Distribution (TPC-H Q13) 5569.93 +Forecasting Revenue Change (TPC-H Q14) 1130.83 +Top Supplier Query (TPC-H Q15) 1321.32 +Parts/Supplier Relationship (TPC-H Q16) 1239.32 +Small-Quantity-Order Revenue (TPC-H Q17) 5228.78 +Large Volume Customer (TPC-H Q18) 17602.90 +Discounted Revenue (TPC-H Q19) 1735.06 +Potential Part Promotion (TPC-H Q20) 1018.27 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2464.08 +Global Sales Opportunity Query (TPC-H Q22) 451.76 ### Loading [s] timeGenerate timeIngesting timeSchema timeIndex timeLoad -DatabaseService-BHT-8-1-1 1.0 122.0 1.0 198.0 330.0 +DatabaseService-BHT-8-1-1 1.0 123.0 1.0 196.0 326.0 ### Geometric Mean of Medians of Timer Run [s] Geo Times [s] DBMS -DatabaseService-BHT-8-1-1 2.14 +DatabaseService-BHT-8-1-1 2.11 ### Power@Size Power@Size [~Q/h] DBMS -DatabaseService-BHT-8-1-1 5185.98 +DatabaseService-BHT-8-1-1 5249.82 ### Throughput@Size time [s] count SF Throughput@Size [~GB/h] DBMS SF num_experiment num_client -DatabaseService-BHT-8-1 3 1 1 70 1 3 3394.29 +DatabaseService-BHT-8-1 3 1 1 71 1 3 3346.48 ### Workflow @@ -158,7 +158,7 @@ DBMS DatabaseService-BHT-8 - Pods [[1]] ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 16.29 0.32 0.25 0.27 +DatabaseService-BHT-8-1 16.0 0 0.25 0.27 ### Tests TEST passed: Geo Times [s] contains no 0 or NaN diff --git a/logs_tests/doc_tpch_testcase_databaseservice_4_summary.txt b/logs_tests/doc_tpch_testcase_databaseservice_4_summary.txt index 93f99945..322c742e 100644 --- a/logs_tests/doc_tpch_testcase_databaseservice_4_summary.txt +++ b/logs_tests/doc_tpch_testcase_databaseservice_4_summary.txt @@ -3,8 +3,8 @@ ### Workload TPC-H Queries SF=3 Type: tpch - Duration: 262s - Code: 1734429878 + Duration: 273s + Code: 1734668861 This includes the reading queries of TPC-H. This experiment compares run time and resource consumption of TPC-H queries in different DBMS. TPC-H (SF=3) data is loaded and benchmark is executed. @@ -31,7 +31,7 @@ DatabaseService-BHT-8-1-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:235965964 + disk:249216876 datadisk:39192 volume_size:1.0G volume_used:36M @@ -46,47 +46,47 @@ No warnings ### Latency of Timer Execution [ms] DBMS DatabaseService-BHT-8-1-1 -Pricing Summary Report (TPC-H Q1) 5713.84 -Minimum Cost Supplier Query (TPC-H Q2) 1886.65 -Shipping Priority (TPC-H Q3) 2253.28 -Order Priority Checking Query (TPC-H Q4) 2854.45 -Local Supplier Volume (TPC-H Q5) 2075.71 -Forecasting Revenue Change (TPC-H Q6) 1101.50 -Forecasting Revenue Change (TPC-H Q7) 2145.88 -National Market Share (TPC-H Q8) 1332.73 -Product Type Profit Measure (TPC-H Q9) 2925.21 -Forecasting Revenue Change (TPC-H Q10) 2867.79 -Important Stock Identification (TPC-H Q11) 564.90 -Shipping Modes and Order Priority (TPC-H Q12) 2189.70 -Customer Distribution (TPC-H Q13) 5426.61 -Forecasting Revenue Change (TPC-H Q14) 1194.76 -Top Supplier Query (TPC-H Q15) 1364.10 -Parts/Supplier Relationship (TPC-H Q16) 1319.02 -Small-Quantity-Order Revenue (TPC-H Q17) 5020.23 -Large Volume Customer (TPC-H Q18) 16806.97 -Discounted Revenue (TPC-H Q19) 1775.99 -Potential Part Promotion (TPC-H Q20) 1037.63 -Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2513.75 -Global Sales Opportunity Query (TPC-H Q22) 475.99 +Pricing Summary Report (TPC-H Q1) 5785.36 +Minimum Cost Supplier Query (TPC-H Q2) 1938.19 +Shipping Priority (TPC-H Q3) 2255.59 +Order Priority Checking Query (TPC-H Q4) 2827.21 +Local Supplier Volume (TPC-H Q5) 2018.18 +Forecasting Revenue Change (TPC-H Q6) 1062.37 +Forecasting Revenue Change (TPC-H Q7) 2075.73 +National Market Share (TPC-H Q8) 1336.07 +Product Type Profit Measure (TPC-H Q9) 2808.84 +Forecasting Revenue Change (TPC-H Q10) 2804.71 +Important Stock Identification (TPC-H Q11) 583.68 +Shipping Modes and Order Priority (TPC-H Q12) 2137.04 +Customer Distribution (TPC-H Q13) 5569.93 +Forecasting Revenue Change (TPC-H Q14) 1130.83 +Top Supplier Query (TPC-H Q15) 1321.32 +Parts/Supplier Relationship (TPC-H Q16) 1239.32 +Small-Quantity-Order Revenue (TPC-H Q17) 5228.78 +Large Volume Customer (TPC-H Q18) 17602.90 +Discounted Revenue (TPC-H Q19) 1735.06 +Potential Part Promotion (TPC-H Q20) 1018.27 +Suppliers Who Kept Orders Waiting Query (TPC-H Q21) 2464.08 +Global Sales Opportunity Query (TPC-H Q22) 451.76 ### Loading [s] timeGenerate timeIngesting timeSchema timeIndex timeLoad -DatabaseService-BHT-8-1-1 1.0 122.0 1.0 198.0 330.0 +DatabaseService-BHT-8-1-1 1.0 123.0 1.0 196.0 326.0 ### Geometric Mean of Medians of Timer Run [s] Geo Times [s] DBMS -DatabaseService-BHT-8-1-1 2.14 +DatabaseService-BHT-8-1-1 2.11 ### Power@Size Power@Size [~Q/h] DBMS -DatabaseService-BHT-8-1-1 5185.98 +DatabaseService-BHT-8-1-1 5249.82 ### Throughput@Size time [s] count SF Throughput@Size [~GB/h] DBMS SF num_experiment num_client -DatabaseService-BHT-8-1 3 1 1 70 1 3 3394.29 +DatabaseService-BHT-8-1 3 1 1 71 1 3 3346.48 ### Workflow @@ -98,7 +98,7 @@ DBMS DatabaseService-BHT-8 - Pods [[1]] ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-BHT-8-1 16.29 0.32 0.25 0.27 +DatabaseService-BHT-8-1 16.0 0 0.25 0.27 ### Tests TEST passed: Geo Times [s] contains no 0 or NaN diff --git a/logs_tests/doc_ycsb_cockroachdb_1.log b/logs_tests/doc_ycsb_cockroachdb_1.log index e32abe2c..704d6146 100644 --- a/logs_tests/doc_ycsb_cockroachdb_1.log +++ b/logs_tests/doc_ycsb_cockroachdb_1.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734604175 -Experiment : starts at 2024-12-19 11:29:35.263746 (4288976.670875062) +Experiment : has code 1734645173 +Experiment : starts at 2024-12-19 22:52:53.295133 (4329974.702264299) Experiment : This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. YCSB is performed using several threads and processes. Target is based on multiples of '16384'. Factors for loading are [4]. Factors for benchmarking are [4]. @@ -24,22 +24,22 @@ done CockroachDB-64-8-65536 : will start now - waiting 30s - : done CockroachDB-64-8-65536 : is not loaded yet -CockroachDB-64-8-65536 : will start loading but not before 2024-12-19 10:32:36 (that is in 120 secs) +CockroachDB-64-8-65536 : will start loading but not before 2024-12-19 21:55:54 (that is in 120 secs) - waiting 30s - : done CockroachDB-64-8-65536 : is not loaded yet -CockroachDB-64-8-65536 : will start loading but not before 2024-12-19 10:32:36 +CockroachDB-64-8-65536 : will start loading but not before 2024-12-19 21:55:54 - waiting 30s - : done CockroachDB-64-8-65536 : is not loaded yet -CockroachDB-64-8-65536 : will start loading but not before 2024-12-19 10:32:36 +CockroachDB-64-8-65536 : will start loading but not before 2024-12-19 21:55:54 - waiting 30s - : done CockroachDB-64-8-65536 : is not loaded yet -CockroachDB-64-8-65536 : will start loading but not before 2024-12-19 10:32:36 +CockroachDB-64-8-65536 : will start loading but not before 2024-12-19 21:55:54 - waiting 30s - : done CockroachDB-64-8-65536 : is not loaded yet done -Worker pods found: ['bexhoma-worker-cockroachdb-64-8-65536-1734604175-0', 'bexhoma-worker-cockroachdb-64-8-65536-1734604175-1', 'bexhoma-worker-cockroachdb-64-8-65536-1734604175-2'] +Worker pods found: ['bexhoma-worker-cockroachdb-64-8-65536-1734645173-0', 'bexhoma-worker-cockroachdb-64-8-65536-1734645173-1', 'bexhoma-worker-cockroachdb-64-8-65536-1734645173-2'] CockroachDB-64-8-65536 Workers 3 of 3 -Worker pods found: ['bexhoma-worker-cockroachdb-64-8-65536-1734604175-0', 'bexhoma-worker-cockroachdb-64-8-65536-1734604175-1', 'bexhoma-worker-cockroachdb-64-8-65536-1734604175-2'] +Worker pods found: ['bexhoma-worker-cockroachdb-64-8-65536-1734645173-0', 'bexhoma-worker-cockroachdb-64-8-65536-1734645173-1', 'bexhoma-worker-cockroachdb-64-8-65536-1734645173-2'] CockroachDB-64-8-65536 : start asynch loading scripts of type loaded CockroachDB-64-8-65536 : is loading - waiting 30s - : done @@ -48,23 +48,23 @@ CockroachDB-64-8-65536 : is loading CockroachDB-64-8-65536 : is loading done CockroachDB-64-8-65536 : showing loader times -CockroachDB-64-8-65536 : loader timespan (first to last [s]) = 60 +CockroachDB-64-8-65536 : loader timespan (first to last [s]) = 63 CockroachDB-64-8-65536 : benchmarks done 0 of 1. This will be client 1 CockroachDB-64-8-65536 : we will change parameters of benchmark as {'PARALLEL': '1', 'SF': '1', 'BEXHOMA_SYNCH_LOAD': 1, 'YCSB_THREADCOUNT': 64, 'YCSB_TARGET': 65536, 'YCSB_STATUS': 1, 'YCSB_WORKLOAD': 'a', 'YCSB_ROWS': 1000000, 'YCSB_OPERATIONS': 10000000, 'YCSB_BATCHSIZE': ''} CockroachDB-64-8-65536-1 : start benchmarking -Worker pods found: ['bexhoma-worker-cockroachdb-64-8-65536-1734604175-0', 'bexhoma-worker-cockroachdb-64-8-65536-1734604175-1', 'bexhoma-worker-cockroachdb-64-8-65536-1734604175-2'] -CockroachDB-64-8-65536 : distributed system - get host info for worker bexhoma-worker-cockroachdb-64-8-65536-1734604175-0 -CockroachDB-64-8-65536 : distributed system - get host info for worker bexhoma-worker-cockroachdb-64-8-65536-1734604175-1 -CockroachDB-64-8-65536 : distributed system - get host info for worker bexhoma-worker-cockroachdb-64-8-65536-1734604175-2 -CockroachDB-64-8-65536 : benchmarking results in folder /home/perdelt/benchmarks/1734604175 +Worker pods found: ['bexhoma-worker-cockroachdb-64-8-65536-1734645173-0', 'bexhoma-worker-cockroachdb-64-8-65536-1734645173-1', 'bexhoma-worker-cockroachdb-64-8-65536-1734645173-2'] +CockroachDB-64-8-65536 : distributed system - get host info for worker bexhoma-worker-cockroachdb-64-8-65536-1734645173-0 +CockroachDB-64-8-65536 : distributed system - get host info for worker bexhoma-worker-cockroachdb-64-8-65536-1734645173-1 +CockroachDB-64-8-65536 : distributed system - get host info for worker bexhoma-worker-cockroachdb-64-8-65536-1734645173-2 +CockroachDB-64-8-65536 : benchmarking results in folder /home/perdelt/benchmarks/1734645173 - waiting 10s - : done found CockroachDB-64-8-65536-1 : collecting loading metrics of SUT -CockroachDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)cockroachdb-64-8-65536-1734604175(.*)", container="dbms"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +CockroachDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)cockroachdb-64-8-65536-1734645173(.*)", container="dbms"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} CockroachDB-64-8-65536-1 : collecting metrics of data generator -CockroachDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)cockroachdb-64-8-65536-1734604175(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +CockroachDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)cockroachdb-64-8-65536-1734645173(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} CockroachDB-64-8-65536-1 : collecting metrics of data injector -CockroachDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)cockroachdb-64-8-65536-1734604175(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +CockroachDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)cockroachdb-64-8-65536-1734645173(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} - waiting 30s - : done CockroachDB-64-8-65536 : has running benchmarks - waiting 30s - : done @@ -113,15 +113,15 @@ done CockroachDB-64-8-65536 : has running benchmarks CockroachDB-64-8-65536-1 : showing benchmarker times CockroachDB-64-8-65536-1 : benchmarker timespan (start to end single container [s]) = 720 -CockroachDB-64-8-65536-1 : benchmarker times (start/end per pod and container) = [(1734604468, 1734605163)] -CockroachDB-64-8-65536-1 : found and updated times {'benchmarker': [(1734604468, 1734605163)]} +CockroachDB-64-8-65536-1 : benchmarker times (start/end per pod and container) = [(1734645467, 1734646176)] +CockroachDB-64-8-65536-1 : found and updated times {'benchmarker': [(1734645467, 1734646176)]} CockroachDB-64-8-65536-1 : collecting execution metrics of SUT -CockroachDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)cockroachdb-64-8-65536-1734604175(.*)", container="dbms"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +CockroachDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)cockroachdb-64-8-65536-1734645173(.*)", container="dbms"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} CockroachDB-64-8-65536-1 : collecting metrics of benchmarker -CockroachDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)cockroachdb-64-8-65536-1734604175(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +CockroachDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)cockroachdb-64-8-65536-1734645173(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} done CockroachDB-64-8-65536 : can be stopped -Experiment : ends at 2024-12-19 11:47:09.777104 (4290031.184237729) - 1054.51s total +Experiment : ends at 2024-12-19 23:10:29.231723 (4331030.638857118) - 1055.94s total Experiment : downloading partial results Experiment : uploading full results Experiment : uploading workload file @@ -132,8 +132,8 @@ Result workflow complete ### Workload YCSB SF=1 Type: ycsb - Duration: 1055s - Code: 1734604175 + Duration: 1056s + Code: 1734645173 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. @@ -155,25 +155,25 @@ CockroachDB-64-8-65536-1 uses docker image cockroachdb/cockroach:v24.2.4 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:249207160 + disk:249215592 requests_cpu:4 requests_memory:16Gi worker 0 - RAM:1081965555712 + RAM:1081966526464 Cores:256 host:5.15.0-1067-nvidia - node:cl-worker27 - disk:560813152 - datadisk:116276744 + node:cl-worker28 + disk:446476124 + datadisk:116276640 volume_size:1000G volume_used:109G worker 1 - RAM:1081966526464 + RAM:1081965555712 Cores:256 host:5.15.0-1067-nvidia - node:cl-worker28 - disk:442719496 - datadisk:116066508 + node:cl-worker27 + disk:634371096 + datadisk:116064760 volume_size:1000G volume_used:109G worker 2 @@ -181,18 +181,18 @@ CockroachDB-64-8-65536-1 uses docker image cockroachdb/cockroach:v24.2.4 Cores:128 host:5.15.0-126-generic node:cl-worker29 - disk:153230928 - datadisk:116065692 + disk:153231576 + datadisk:116065120 volume_size:1000G volume_used:109G ### Loading experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [INSERT].Return=OK [INSERT].99thPercentileLatency(us) -CockroachDB-64-8-65536 1 64 65536 8 17071.978081 58791.0 1000000 7190.0 +CockroachDB-64-8-65536 1 64 65536 8 16211.343884 61937.0 1000000 7579.5 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -CockroachDB-64-8-65536-1 1 64 65536 1 14391.04 694877.0 5000111 5691.0 4999889 126207.0 +CockroachDB-64-8-65536-1 1 64 65536 1 14106.68 708884.0 5000094 5851.0 4999906 130879.0 ### Workflow @@ -204,19 +204,19 @@ DBMS CockroachDB-64-8-65536 - Pods [[1]] ### Ingestion - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -CockroachDB-64-8-65536-1 1295.39 6.78 3.68 6.88 +CockroachDB-64-8-65536-1 888.04 0.03 3.05 5.76 ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -CockroachDB-64-8-65536-1 22.52 0.41 0.58 0.59 +CockroachDB-64-8-65536-1 103.7 0 4.34 4.37 ### Execution - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -CockroachDB-64-8-65536-1 21640.62 21.35 12.76 28.74 +CockroachDB-64-8-65536-1 20657.12 20.74 12.51 26.94 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -CockroachDB-64-8-65536-1 1181.64 1.78 0.6 0.61 +CockroachDB-64-8-65536-1 1024.62 1.56 0.6 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN diff --git a/logs_tests/doc_ycsb_cockroachdb_1_summary.txt b/logs_tests/doc_ycsb_cockroachdb_1_summary.txt index 4f2840e2..c9f5499c 100644 --- a/logs_tests/doc_ycsb_cockroachdb_1_summary.txt +++ b/logs_tests/doc_ycsb_cockroachdb_1_summary.txt @@ -3,8 +3,8 @@ ### Workload YCSB SF=1 Type: ycsb - Duration: 1055s - Code: 1734604175 + Duration: 1056s + Code: 1734645173 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. @@ -26,25 +26,25 @@ CockroachDB-64-8-65536-1 uses docker image cockroachdb/cockroach:v24.2.4 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:249207160 + disk:249215592 requests_cpu:4 requests_memory:16Gi worker 0 - RAM:1081965555712 + RAM:1081966526464 Cores:256 host:5.15.0-1067-nvidia - node:cl-worker27 - disk:560813152 - datadisk:116276744 + node:cl-worker28 + disk:446476124 + datadisk:116276640 volume_size:1000G volume_used:109G worker 1 - RAM:1081966526464 + RAM:1081965555712 Cores:256 host:5.15.0-1067-nvidia - node:cl-worker28 - disk:442719496 - datadisk:116066508 + node:cl-worker27 + disk:634371096 + datadisk:116064760 volume_size:1000G volume_used:109G worker 2 @@ -52,18 +52,18 @@ CockroachDB-64-8-65536-1 uses docker image cockroachdb/cockroach:v24.2.4 Cores:128 host:5.15.0-126-generic node:cl-worker29 - disk:153230928 - datadisk:116065692 + disk:153231576 + datadisk:116065120 volume_size:1000G volume_used:109G ### Loading experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [INSERT].Return=OK [INSERT].99thPercentileLatency(us) -CockroachDB-64-8-65536 1 64 65536 8 17071.978081 58791.0 1000000 7190.0 +CockroachDB-64-8-65536 1 64 65536 8 16211.343884 61937.0 1000000 7579.5 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -CockroachDB-64-8-65536-1 1 64 65536 1 14391.04 694877.0 5000111 5691.0 4999889 126207.0 +CockroachDB-64-8-65536-1 1 64 65536 1 14106.68 708884.0 5000094 5851.0 4999906 130879.0 ### Workflow @@ -75,19 +75,19 @@ DBMS CockroachDB-64-8-65536 - Pods [[1]] ### Ingestion - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -CockroachDB-64-8-65536-1 1295.39 6.78 3.68 6.88 +CockroachDB-64-8-65536-1 888.04 0.03 3.05 5.76 ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -CockroachDB-64-8-65536-1 22.52 0.41 0.58 0.59 +CockroachDB-64-8-65536-1 103.7 0 4.34 4.37 ### Execution - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -CockroachDB-64-8-65536-1 21640.62 21.35 12.76 28.74 +CockroachDB-64-8-65536-1 20657.12 20.74 12.51 26.94 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -CockroachDB-64-8-65536-1 1181.64 1.78 0.6 0.61 +CockroachDB-64-8-65536-1 1024.62 1.56 0.6 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN diff --git a/logs_tests/doc_ycsb_databaseservice_1.log b/logs_tests/doc_ycsb_databaseservice_1.log index ee9023c9..dfe50e1b 100644 --- a/logs_tests/doc_ycsb_databaseservice_1.log +++ b/logs_tests/doc_ycsb_databaseservice_1.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734449423 -Experiment : starts at 2024-12-17 16:30:22.808199 (4134224.215333348) +Experiment : has code 1734663459 +Experiment : starts at 2024-12-20 03:57:38.591399 (4348259.998533409) Experiment : This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 1000000. Batch size is ''. YCSB is performed using several threads and processes. Target is based on multiples of '16384'. Factors for loading are [4]. Factors for benchmarking are [4]. @@ -27,26 +27,26 @@ DatabaseService-64-8-65536 : waits for health check to succeed DatabaseService-64-8-65536 : waits for health check to succeed - waiting 30s - : done DatabaseService-64-8-65536 : is not loaded yet -DatabaseService-64-8-65536 : will start loading but not before 2024-12-17 15:33:54 (that is in 60 secs) +DatabaseService-64-8-65536 : will start loading but not before 2024-12-20 03:01:10 (that is in 60 secs) - waiting 30s - : done DatabaseService-64-8-65536 : is not loaded yet -DatabaseService-64-8-65536 : will start loading but not before 2024-12-17 15:33:54 +DatabaseService-64-8-65536 : will start loading but not before 2024-12-20 03:01:10 done DatabaseService-64-8-65536 : is not loaded yet DatabaseService-64-8-65536 : start asynch loading scripts of type loaded DatabaseService-64-8-65536 : is loading -done +- waiting 30s - : done DatabaseService-64-8-65536 : is loading done DatabaseService-64-8-65536 : showing loader times -DatabaseService-64-8-65536 : loader timespan (first to last [s]) = 20 +DatabaseService-64-8-65536 : loader timespan (first to last [s]) = 21 DatabaseService-64-8-65536 : benchmarks done 0 of 1. This will be client 1 DatabaseService-64-8-65536 : we will change parameters of benchmark as {'PARALLEL': '1', 'SF': '1', 'BEXHOMA_SYNCH_LOAD': 1, 'YCSB_THREADCOUNT': 64, 'YCSB_TARGET': 65536, 'YCSB_STATUS': 1, 'YCSB_WORKLOAD': 'a', 'YCSB_ROWS': 1000000, 'YCSB_OPERATIONS': 1000000, 'YCSB_BATCHSIZE': ''} DatabaseService-64-8-65536-1 : start benchmarking Worker pods found: [] -DatabaseService-64-8-65536 : benchmarking results in folder /home/perdelt/benchmarks/1734449423 +DatabaseService-64-8-65536 : benchmarking results in folder /home/perdelt/benchmarks/1734663459 - waiting 10s - : done -DatabaseService-64-8-65536 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-64-8-65536-1734449423-xkbcb: found +DatabaseService-64-8-65536 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-64-8-65536-1734663459-dhjg4: found done DatabaseService-64-8-65536 : has running benchmarks DatabaseService-64-8-65536-1 : showing benchmarker times @@ -55,7 +55,7 @@ DatabaseService-64-8-65536-1 : benchmarker times (start/end per pod and contain DatabaseService-64-8-65536-1 : found and updated times {'benchmarker': []} done DatabaseService-64-8-65536 : can be stopped -Experiment : ends at 2024-12-17 16:36:15.869726 (4134577.276859674) - 353.06s total +Experiment : ends at 2024-12-20 04:03:32.296496 (4348613.703629796) - 353.71s total Experiment : downloading partial results Experiment : uploading full results Experiment : uploading workload file @@ -67,7 +67,7 @@ Result workflow complete YCSB SF=1 Type: ycsb Duration: 354s - Code: 1734449423 + Code: 1734663459 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 1000000. Batch size is ''. @@ -89,18 +89,18 @@ DatabaseService-64-8-65536-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236006380 + disk:249256012 datadisk:39348 requests_cpu:4 requests_memory:16Gi ### Loading experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [INSERT].Return=OK [INSERT].99thPercentileLatency(us) -DatabaseService-64-8-65536 1 64 65536 8 50279.137219 20168.0 1000000 25355.0 +DatabaseService-64-8-65536 1 64 65536 8 49973.150502 20251.0 1000000 25397.0 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -DatabaseService-64-8-65536-1 1 64 65536 1 55263.89 18095.0 499843 2201.0 500157 39103.0 +DatabaseService-64-8-65536-1 1 64 65536 1 55202.87 18115.0 499487 2095.0 500513 42239.0 ### Workflow diff --git a/logs_tests/doc_ycsb_databaseservice_1_summary.txt b/logs_tests/doc_ycsb_databaseservice_1_summary.txt index 76887636..4f55a81b 100644 --- a/logs_tests/doc_ycsb_databaseservice_1_summary.txt +++ b/logs_tests/doc_ycsb_databaseservice_1_summary.txt @@ -4,7 +4,7 @@ YCSB SF=1 Type: ycsb Duration: 354s - Code: 1734449423 + Code: 1734663459 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 1000000. Batch size is ''. @@ -26,18 +26,18 @@ DatabaseService-64-8-65536-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236006380 + disk:249256012 datadisk:39348 requests_cpu:4 requests_memory:16Gi ### Loading experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [INSERT].Return=OK [INSERT].99thPercentileLatency(us) -DatabaseService-64-8-65536 1 64 65536 8 50279.137219 20168.0 1000000 25355.0 +DatabaseService-64-8-65536 1 64 65536 8 49973.150502 20251.0 1000000 25397.0 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -DatabaseService-64-8-65536-1 1 64 65536 1 55263.89 18095.0 499843 2201.0 500157 39103.0 +DatabaseService-64-8-65536-1 1 64 65536 1 55202.87 18115.0 499487 2095.0 500513 42239.0 ### Workflow diff --git a/logs_tests/doc_ycsb_databaseservice_2.log b/logs_tests/doc_ycsb_databaseservice_2.log index 915a8838..64ce1707 100644 --- a/logs_tests/doc_ycsb_databaseservice_2.log +++ b/logs_tests/doc_ycsb_databaseservice_2.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734449783 -Experiment : starts at 2024-12-17 16:36:22.990276 (4134584.397405118) +Experiment : has code 1734663819 +Experiment : starts at 2024-12-20 04:03:38.820550 (4348620.227677921) Experiment : This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. YCSB is performed using several threads and processes. Target is based on multiples of '16384'. Factors for loading are [4]. Factors for benchmarking are [4]. @@ -22,19 +22,17 @@ Cluster monitoring : is running done DatabaseService-64-8-65536 : will start now - waiting 30s - : done -DatabaseService-64-8-65536 : will start benchmarking but not before 2024-12-17 15:38:23 (that is in 60 secs) +DatabaseService-64-8-65536 : will start benchmarking but not before 2024-12-20 03:05:39 (that is in 60 secs) - waiting 30s - : done -DatabaseService-64-8-65536 : will start benchmarking but not before 2024-12-17 15:38:23 +DatabaseService-64-8-65536 : will start benchmarking but not before 2024-12-20 03:05:39 done DatabaseService-64-8-65536 : benchmarks done 0 of 1. This will be client 1 DatabaseService-64-8-65536 : we will change parameters of benchmark as {'PARALLEL': '1', 'SF': '1', 'BEXHOMA_SYNCH_LOAD': 1, 'YCSB_THREADCOUNT': 64, 'YCSB_TARGET': 65536, 'YCSB_STATUS': 1, 'YCSB_WORKLOAD': 'a', 'YCSB_ROWS': 1000000, 'YCSB_OPERATIONS': 10000000, 'YCSB_BATCHSIZE': ''} DatabaseService-64-8-65536-1 : start benchmarking Worker pods found: [] -DatabaseService-64-8-65536 : benchmarking results in folder /home/perdelt/benchmarks/1734449783 +DatabaseService-64-8-65536 : benchmarking results in folder /home/perdelt/benchmarks/1734663819 - waiting 10s - : done -DatabaseService-64-8-65536 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-64-8-65536-1734449783-7hp2c: found -- waiting 30s - : done -DatabaseService-64-8-65536 : has running benchmarks +DatabaseService-64-8-65536 : benchmarking is waiting for job bexhoma-benchmarker-databaseservice-64-8-65536-1734663819-bxk2f: found - waiting 30s - : done DatabaseService-64-8-65536 : has running benchmarks - waiting 30s - : done @@ -46,14 +44,14 @@ DatabaseService-64-8-65536 : has running benchmarks done DatabaseService-64-8-65536 : has running benchmarks DatabaseService-64-8-65536-1 : showing benchmarker times -DatabaseService-64-8-65536-1 : benchmarker timespan (start to end single container [s]) = 192 +DatabaseService-64-8-65536-1 : benchmarker timespan (start to end single container [s]) = 162 DatabaseService-64-8-65536-1 : benchmarker times (start/end per pod and container) = [] DatabaseService-64-8-65536-1 : found and updated times {'benchmarker': []} DatabaseService-64-8-65536-1 : collecting metrics of benchmarker -DatabaseService-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-64-8-65536-1734449783(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +DatabaseService-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-64-8-65536-1734663819(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} done DatabaseService-64-8-65536 : can be stopped -Experiment : ends at 2024-12-17 16:42:14.194525 (4134935.601658705) - 351.20s total +Experiment : ends at 2024-12-20 04:08:59.884356 (4348941.291489666) - 321.06s total Experiment : downloading partial results Experiment : uploading full results Experiment : uploading workload file @@ -64,8 +62,8 @@ Result workflow complete ### Workload YCSB SF=1 Type: ycsb - Duration: 352s - Code: 1734449783 + Duration: 322s + Code: 1734663819 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. @@ -89,14 +87,14 @@ DatabaseService-64-8-65536-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236006372 + disk:249256004 datadisk:39348 requests_cpu:4 requests_memory:16Gi ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -DatabaseService-64-8-65536-1 1 64 65536 1 62695.14 159502.0 5000594 2147.0 4999406 31087.0 +DatabaseService-64-8-65536-1 1 64 65536 1 64975.99 153903.0 4999780 2423.0 5000220 30719.0 ### Workflow @@ -108,7 +106,7 @@ DBMS DatabaseService-64-8-65536 - Pods [[1]] ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-64-8-65536-1 992.25 7.14 0.6 0.61 +DatabaseService-64-8-65536-1 745.84 6.07 0.6 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN diff --git a/logs_tests/doc_ycsb_databaseservice_2_summary.txt b/logs_tests/doc_ycsb_databaseservice_2_summary.txt index 558bd7f1..ef47b300 100644 --- a/logs_tests/doc_ycsb_databaseservice_2_summary.txt +++ b/logs_tests/doc_ycsb_databaseservice_2_summary.txt @@ -3,8 +3,8 @@ ### Workload YCSB SF=1 Type: ycsb - Duration: 352s - Code: 1734449783 + Duration: 322s + Code: 1734663819 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. @@ -28,14 +28,14 @@ DatabaseService-64-8-65536-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:236006372 + disk:249256004 datadisk:39348 requests_cpu:4 requests_memory:16Gi ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -DatabaseService-64-8-65536-1 1 64 65536 1 62695.14 159502.0 5000594 2147.0 4999406 31087.0 +DatabaseService-64-8-65536-1 1 64 65536 1 64975.99 153903.0 4999780 2423.0 5000220 30719.0 ### Workflow @@ -47,7 +47,7 @@ DBMS DatabaseService-64-8-65536 - Pods [[1]] ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-64-8-65536-1 992.25 7.14 0.6 0.61 +DatabaseService-64-8-65536-1 745.84 6.07 0.6 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN diff --git a/logs_tests/doc_ycsb_databaseservice_3.log b/logs_tests/doc_ycsb_databaseservice_3.log index 584d1572..81284bd0 100644 --- a/logs_tests/doc_ycsb_databaseservice_3.log +++ b/logs_tests/doc_ycsb_databaseservice_3.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734458418 -Experiment : starts at 2024-12-17 19:00:18.158577 (4143219.565707659) +Experiment : has code 1734700853 +Experiment : starts at 2024-12-20 14:20:53.019251 (4385654.426380379) Experiment : This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 5000000. Number of operations is 10000000. Batch size is ''. YCSB is performed using several threads and processes. Target is based on multiples of '16384'. Factors for loading are [4]. Factors for benchmarking are [4]. @@ -19,6 +19,210 @@ Benchmarking is tested with [64] threads, split into [1] pods. Benchmarking is run as [1] times the number of benchmarking pods. Experiment is run once. Cluster monitoring : is running +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster +- waiting 30s - : done +DatabaseService-64-8-65536 : has to wait - 2 running and 0 pending pods: max is 2 pods per cluster done DatabaseService-64-8-65536 : will start now - waiting 30s - : done @@ -31,10 +235,10 @@ DatabaseService-64-8-65536 : waits for health check to succeed DatabaseService-64-8-65536 : waits for health check to succeed - waiting 30s - : done DatabaseService-64-8-65536 : is not loaded yet -DatabaseService-64-8-65536 : will start loading but not before 2024-12-17 18:04:19 (that is in 60 secs) +DatabaseService-64-8-65536 : will start loading but not before 2024-12-20 14:16:07 (that is in 60 secs) - waiting 30s - : done DatabaseService-64-8-65536 : is not loaded yet -DatabaseService-64-8-65536 : will start loading but not before 2024-12-17 18:04:19 +DatabaseService-64-8-65536 : will start loading but not before 2024-12-20 14:16:07 done DatabaseService-64-8-65536 : is not loaded yet DatabaseService-64-8-65536 : start asynch loading scripts of type loaded @@ -51,18 +255,18 @@ DatabaseService-64-8-65536 : is loading DatabaseService-64-8-65536 : is loading done DatabaseService-64-8-65536 : showing loader times -DatabaseService-64-8-65536 : loader timespan (first to last [s]) = 168 +DatabaseService-64-8-65536 : loader timespan (first to last [s]) = 145 DatabaseService-64-8-65536 : benchmarks done 0 of 1. This will be client 1 DatabaseService-64-8-65536 : we will change parameters of benchmark as {'PARALLEL': '1', 'SF': '5', 'BEXHOMA_SYNCH_LOAD': 1, 'YCSB_THREADCOUNT': 64, 'YCSB_TARGET': 65536, 'YCSB_STATUS': 1, 'YCSB_WORKLOAD': 'a', 'YCSB_ROWS': 5000000, 'YCSB_OPERATIONS': 10000000, 'YCSB_BATCHSIZE': ''} DatabaseService-64-8-65536-1 : start benchmarking Worker pods found: [] -DatabaseService-64-8-65536 : benchmarking results in folder /home/perdelt/benchmarks/1734458418 +DatabaseService-64-8-65536 : benchmarking results in folder /home/perdelt/benchmarks/1734700853 - waiting 10s - : done found DatabaseService-64-8-65536-1 : collecting metrics of data generator -DatabaseService-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-64-8-65536-1734458418(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +DatabaseService-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-64-8-65536-1734700853(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} DatabaseService-64-8-65536-1 : collecting metrics of data injector -DatabaseService-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-64-8-65536-1734458418(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +DatabaseService-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-64-8-65536-1734700853(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} - waiting 30s - : done DatabaseService-64-8-65536 : has running benchmarks - waiting 30s - : done @@ -74,14 +278,14 @@ DatabaseService-64-8-65536 : has running benchmarks done DatabaseService-64-8-65536 : has running benchmarks DatabaseService-64-8-65536-1 : showing benchmarker times -DatabaseService-64-8-65536-1 : benchmarker timespan (start to end single container [s]) = 176 +DatabaseService-64-8-65536-1 : benchmarker timespan (start to end single container [s]) = 171 DatabaseService-64-8-65536-1 : benchmarker times (start/end per pod and container) = [] DatabaseService-64-8-65536-1 : found and updated times {'benchmarker': []} DatabaseService-64-8-65536-1 : collecting metrics of benchmarker -DatabaseService-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-64-8-65536-1734458418(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +DatabaseService-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)databaseservice-64-8-65536-1734700853(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} done DatabaseService-64-8-65536 : can be stopped -Experiment : ends at 2024-12-17 19:11:02.970161 (4143864.377294936) - 644.81s total +Experiment : ends at 2024-12-20 15:22:50.227514 (4389371.634647944) - 3717.21s total Experiment : downloading partial results Experiment : uploading full results Experiment : uploading workload file @@ -92,8 +296,8 @@ Result workflow complete ### Workload YCSB SF=5 Type: ycsb - Duration: 645s - Code: 1734458418 + Duration: 3718s + Code: 1734700853 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 5000000. Number of operations is 10000000. Batch size is ''. @@ -117,7 +321,7 @@ DatabaseService-64-8-65536-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:235967632 + disk:250060572 datadisk:39192 volume_size:1.0G volume_used:36M @@ -126,11 +330,11 @@ DatabaseService-64-8-65536-1 uses docker image postgres:16.1 ### Loading experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [INSERT].Return=OK [INSERT].99thPercentileLatency(us) -DatabaseService-64-8-65536 1 64 65536 8 30211.248367 167036.0 5000000 26383.0 +DatabaseService-64-8-65536 1 64 65536 8 34656.857878 145263.0 5000000 6815.5 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -DatabaseService-64-8-65536-1 1 64 65536 1 60402.52 165556.0 5001510 2889.0 4998490 32895.0 +DatabaseService-64-8-65536-1 1 64 65536 1 65353.5 153014.0 4997965 653.0 5002035 1296.0 ### Workflow @@ -142,11 +346,11 @@ DBMS DatabaseService-64-8-65536 - Pods [[1]] ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-64-8-65536-1 590.05 1.69 4.6 4.62 +DatabaseService-64-8-65536-1 397.12 0.88 4.6 4.62 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-64-8-65536-1 837.81 7.58 0.61 0.61 +DatabaseService-64-8-65536-1 743.75 5.11 0.6 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN diff --git a/logs_tests/doc_ycsb_databaseservice_3_summary.txt b/logs_tests/doc_ycsb_databaseservice_3_summary.txt index cb894fca..62da3987 100644 --- a/logs_tests/doc_ycsb_databaseservice_3_summary.txt +++ b/logs_tests/doc_ycsb_databaseservice_3_summary.txt @@ -3,8 +3,8 @@ ### Workload YCSB SF=5 Type: ycsb - Duration: 645s - Code: 1734458418 + Duration: 3718s + Code: 1734700853 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 5000000. Number of operations is 10000000. Batch size is ''. @@ -28,7 +28,7 @@ DatabaseService-64-8-65536-1 uses docker image postgres:16.1 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:235967632 + disk:250060572 datadisk:39192 volume_size:1.0G volume_used:36M @@ -37,11 +37,11 @@ DatabaseService-64-8-65536-1 uses docker image postgres:16.1 ### Loading experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [INSERT].Return=OK [INSERT].99thPercentileLatency(us) -DatabaseService-64-8-65536 1 64 65536 8 30211.248367 167036.0 5000000 26383.0 +DatabaseService-64-8-65536 1 64 65536 8 34656.857878 145263.0 5000000 6815.5 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -DatabaseService-64-8-65536-1 1 64 65536 1 60402.52 165556.0 5001510 2889.0 4998490 32895.0 +DatabaseService-64-8-65536-1 1 64 65536 1 65353.5 153014.0 4997965 653.0 5002035 1296.0 ### Workflow @@ -53,11 +53,11 @@ DBMS DatabaseService-64-8-65536 - Pods [[1]] ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-64-8-65536-1 590.05 1.69 4.6 4.62 +DatabaseService-64-8-65536-1 397.12 0.88 4.6 4.62 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -DatabaseService-64-8-65536-1 837.81 7.58 0.61 0.61 +DatabaseService-64-8-65536-1 743.75 5.11 0.6 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN diff --git a/logs_tests/doc_ycsb_yugabytedb_1.log b/logs_tests/doc_ycsb_yugabytedb_1.log index 236a4f48..ed7835bd 100644 --- a/logs_tests/doc_ycsb_yugabytedb_1.log +++ b/logs_tests/doc_ycsb_yugabytedb_1.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734597746 -Experiment : starts at 2024-12-19 09:42:26.510639 (4282547.917767901) +Experiment : has code 1734625544 +Experiment : starts at 2024-12-19 17:25:44.048070 (4310345.455200298) Experiment : This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. YCSB is performed using several threads and processes. Target is based on multiples of '16384'. Factors for loading are [4]. Factors for benchmarking are [4]. @@ -22,10 +22,10 @@ done YugabyteDB-64-8-65536 : will start now - waiting 30s - : done YugabyteDB-64-8-65536 : is not loaded yet -YugabyteDB-64-8-65536 : will start loading but not before 2024-12-19 08:44:27 (that is in 60 secs) +YugabyteDB-64-8-65536 : will start loading but not before 2024-12-19 16:27:44 (that is in 60 secs) - waiting 30s - : done YugabyteDB-64-8-65536 : is not loaded yet -YugabyteDB-64-8-65536 : will start loading but not before 2024-12-19 08:44:27 +YugabyteDB-64-8-65536 : will start loading but not before 2024-12-19 16:27:44 done YugabyteDB-64-8-65536 : is not loaded yet YugabyteDB-64-8-65536 : start asynch loading scripts of type loaded @@ -34,22 +34,22 @@ YugabyteDB-64-8-65536 : is loading YugabyteDB-64-8-65536 : is loading done YugabyteDB-64-8-65536 : showing loader times -YugabyteDB-64-8-65536 : loader timespan (first to last [s]) = 37 +YugabyteDB-64-8-65536 : loader timespan (first to last [s]) = 36 YugabyteDB-64-8-65536 : benchmarks done 0 of 1. This will be client 1 YugabyteDB-64-8-65536 : we will change parameters of benchmark as {'PARALLEL': '1', 'SF': '1', 'BEXHOMA_SYNCH_LOAD': 1, 'YCSB_THREADCOUNT': 64, 'YCSB_TARGET': 65536, 'YCSB_STATUS': 1, 'YCSB_WORKLOAD': 'a', 'YCSB_ROWS': 1000000, 'YCSB_OPERATIONS': 10000000, 'YCSB_BATCHSIZE': ''} YugabyteDB-64-8-65536-1 : start benchmarking YugabyteDB-64-8-65536 : distributed system - get host info for worker yb-tserver-0 YugabyteDB-64-8-65536 : distributed system - get host info for worker yb-tserver-1 YugabyteDB-64-8-65536 : distributed system - get host info for worker yb-tserver-2 -YugabyteDB-64-8-65536 : benchmarking results in folder /home/perdelt/benchmarks/1734597746 +YugabyteDB-64-8-65536 : benchmarking results in folder /home/perdelt/benchmarks/1734625544 - waiting 10s - : done found YugabyteDB-64-8-65536-1 : collecting loading metrics of SUT YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yb-tserver-(.*)"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} YugabyteDB-64-8-65536-1 : collecting metrics of data generator -YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yugabytedb-64-8-65536-1734597746(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yugabytedb-64-8-65536-1734625544(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} YugabyteDB-64-8-65536-1 : collecting metrics of data injector -YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yugabytedb-64-8-65536-1734597746(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yugabytedb-64-8-65536-1734625544(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} - waiting 30s - : done YugabyteDB-64-8-65536 : has running benchmarks - waiting 30s - : done @@ -71,16 +71,16 @@ YugabyteDB-64-8-65536 : has running benchmarks done YugabyteDB-64-8-65536 : has running benchmarks YugabyteDB-64-8-65536-1 : showing benchmarker times -YugabyteDB-64-8-65536-1 : benchmarker timespan (start to end single container [s]) = 326 -YugabyteDB-64-8-65536-1 : benchmarker times (start/end per pod and container) = [(1734597943, 1734598255)] -YugabyteDB-64-8-65536-1 : found and updated times {'benchmarker': [(1734597943, 1734598255)]} +YugabyteDB-64-8-65536-1 : benchmarker timespan (start to end single container [s]) = 327 +YugabyteDB-64-8-65536-1 : benchmarker times (start/end per pod and container) = [(1734625740, 1734626054)] +YugabyteDB-64-8-65536-1 : found and updated times {'benchmarker': [(1734625740, 1734626054)]} YugabyteDB-64-8-65536-1 : collecting execution metrics of SUT YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yb-tserver-(.*)"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} YugabyteDB-64-8-65536-1 : collecting metrics of benchmarker -YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yugabytedb-64-8-65536-1734597746(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yugabytedb-64-8-65536-1734625544(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} done YugabyteDB-64-8-65536 : can be stopped -Experiment : ends at 2024-12-19 09:51:48.369425 (4283109.776557674) - 561.86s total +Experiment : ends at 2024-12-19 17:35:05.156119 (4310906.563252344) - 561.11s total Experiment : downloading partial results Experiment : uploading full results Experiment : uploading workload file @@ -92,7 +92,7 @@ Result workflow complete YCSB SF=1 Type: ycsb Duration: 562s - Code: 1734597746 + Code: 1734625544 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. @@ -115,7 +115,7 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:248621272 + disk:249253840 datadisk:39428 requests_cpu:4 requests_memory:16Gi @@ -125,29 +125,29 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:441857356 + disk:439206828 worker 1 RAM:540587499520 CPU:AMD EPYC 7502 32-Core Processor Cores:128 host:5.15.0-126-generic node:cl-worker22 - disk:122885856 + disk:122936080 worker 2 RAM:1081965555712 CPU:AMD EPYC 7742 64-Core Processor Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:559736836 + disk:584264864 ### Loading experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [INSERT].Return=OK [INSERT].99thPercentileLatency(us) -YugabyteDB-64-8-65536 1 64 65536 8 28245.1905 35540.0 1000000 18471.0 +YugabyteDB-64-8-65536 1 64 65536 8 28456.559524 35509.0 1000000 15762.0 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -YugabyteDB-64-8-65536-1 1 64 65536 1 32042.24 312088.0 4996704 22463.0 5003296 35711.0 +YugabyteDB-64-8-65536-1 1 64 65536 1 31861.75 313856.0 4998066 37663.0 5001934 43039.0 ### Workflow @@ -159,19 +159,19 @@ DBMS YugabyteDB-64-8-65536 - Pods [[1]] ### Ingestion - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 910.63 9.64 2.18 5.5 +YugabyteDB-64-8-65536-1 925.63 4.64 1.75 5.32 ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 63.84 0 3.39 3.42 +YugabyteDB-64-8-65536-1 0.09 0 0.01 0.01 ### Execution - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 14689.53 15.97 4.92 16.73 +YugabyteDB-64-8-65536-1 13499.7 15.99 4.92 16.26 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 944.15 3.14 0.61 0.61 +YugabyteDB-64-8-65536-1 932.99 3.2 0.61 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN diff --git a/logs_tests/doc_ycsb_yugabytedb_1_summary.txt b/logs_tests/doc_ycsb_yugabytedb_1_summary.txt index f61923f7..d9b9a21f 100644 --- a/logs_tests/doc_ycsb_yugabytedb_1_summary.txt +++ b/logs_tests/doc_ycsb_yugabytedb_1_summary.txt @@ -4,7 +4,7 @@ YCSB SF=1 Type: ycsb Duration: 562s - Code: 1734597746 + Code: 1734625544 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. @@ -27,7 +27,7 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:248621272 + disk:249253840 datadisk:39428 requests_cpu:4 requests_memory:16Gi @@ -37,29 +37,29 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:441857356 + disk:439206828 worker 1 RAM:540587499520 CPU:AMD EPYC 7502 32-Core Processor Cores:128 host:5.15.0-126-generic node:cl-worker22 - disk:122885856 + disk:122936080 worker 2 RAM:1081965555712 CPU:AMD EPYC 7742 64-Core Processor Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:559736836 + disk:584264864 ### Loading experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [INSERT].Return=OK [INSERT].99thPercentileLatency(us) -YugabyteDB-64-8-65536 1 64 65536 8 28245.1905 35540.0 1000000 18471.0 +YugabyteDB-64-8-65536 1 64 65536 8 28456.559524 35509.0 1000000 15762.0 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -YugabyteDB-64-8-65536-1 1 64 65536 1 32042.24 312088.0 4996704 22463.0 5003296 35711.0 +YugabyteDB-64-8-65536-1 1 64 65536 1 31861.75 313856.0 4998066 37663.0 5001934 43039.0 ### Workflow @@ -71,19 +71,19 @@ DBMS YugabyteDB-64-8-65536 - Pods [[1]] ### Ingestion - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 910.63 9.64 2.18 5.5 +YugabyteDB-64-8-65536-1 925.63 4.64 1.75 5.32 ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 63.84 0 3.39 3.42 +YugabyteDB-64-8-65536-1 0.09 0 0.01 0.01 ### Execution - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 14689.53 15.97 4.92 16.73 +YugabyteDB-64-8-65536-1 13499.7 15.99 4.92 16.26 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 944.15 3.14 0.61 0.61 +YugabyteDB-64-8-65536-1 932.99 3.2 0.61 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN diff --git a/logs_tests/doc_ycsb_yugabytedb_2.log b/logs_tests/doc_ycsb_yugabytedb_2.log index ab456b4e..541adb0d 100644 --- a/logs_tests/doc_ycsb_yugabytedb_2.log +++ b/logs_tests/doc_ycsb_yugabytedb_2.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734598346 -Experiment : starts at 2024-12-19 09:52:26.658961 (4283148.066091008) +Experiment : has code 1734626144 +Experiment : starts at 2024-12-19 17:35:44.158663 (4310945.565792602) Experiment : This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. YCSB is performed using several threads and processes. Target is based on multiples of '16384'. Factors for loading are [4]. Factors for benchmarking are [4]. @@ -22,9 +22,9 @@ Cluster monitoring : is running done YugabyteDB-64-8-65536 : will start now - waiting 30s - : done -YugabyteDB-64-8-65536 : will start benchmarking but not before 2024-12-19 08:54:27 (that is in 60 secs) +YugabyteDB-64-8-65536 : will start benchmarking but not before 2024-12-19 16:37:44 (that is in 60 secs) - waiting 30s - : done -YugabyteDB-64-8-65536 : will start benchmarking but not before 2024-12-19 08:54:27 +YugabyteDB-64-8-65536 : will start benchmarking but not before 2024-12-19 16:37:44 done YugabyteDB-64-8-65536 : benchmarks done 0 of 1. This will be client 1 YugabyteDB-64-8-65536 : we will change parameters of benchmark as {'PARALLEL': '1', 'SF': '1', 'BEXHOMA_SYNCH_LOAD': 1, 'YCSB_THREADCOUNT': 64, 'YCSB_TARGET': 65536, 'YCSB_STATUS': 1, 'YCSB_WORKLOAD': 'a', 'YCSB_ROWS': 1000000, 'YCSB_OPERATIONS': 10000000, 'YCSB_BATCHSIZE': ''} @@ -32,9 +32,9 @@ YugabyteDB-64-8-65536-1 : start benchmarking YugabyteDB-64-8-65536 : distributed system - get host info for worker yb-tserver-0 YugabyteDB-64-8-65536 : distributed system - get host info for worker yb-tserver-1 YugabyteDB-64-8-65536 : distributed system - get host info for worker yb-tserver-2 -YugabyteDB-64-8-65536 : benchmarking results in folder /home/perdelt/benchmarks/1734598346 +YugabyteDB-64-8-65536 : benchmarking results in folder /home/perdelt/benchmarks/1734626144 - waiting 10s - : done -YugabyteDB-64-8-65536 : benchmarking is waiting for job bexhoma-benchmarker-yugabytedb-64-8-65536-1734598346-1-1-p76nj: found +YugabyteDB-64-8-65536 : benchmarking is waiting for job bexhoma-benchmarker-yugabytedb-64-8-65536-1734626144-1-1-5r565: found - waiting 30s - : done YugabyteDB-64-8-65536 : has running benchmarks - waiting 30s - : done @@ -58,16 +58,16 @@ YugabyteDB-64-8-65536 : has running benchmarks done YugabyteDB-64-8-65536 : has running benchmarks YugabyteDB-64-8-65536-1 : showing benchmarker times -YugabyteDB-64-8-65536-1 : benchmarker timespan (start to end single container [s]) = 343 -YugabyteDB-64-8-65536-1 : benchmarker times (start/end per pod and container) = [(1734598476, 1734598799)] -YugabyteDB-64-8-65536-1 : found and updated times {'benchmarker': [(1734598476, 1734598799)]} +YugabyteDB-64-8-65536-1 : benchmarker timespan (start to end single container [s]) = 342 +YugabyteDB-64-8-65536-1 : benchmarker times (start/end per pod and container) = [(1734626273, 1734626593)] +YugabyteDB-64-8-65536-1 : found and updated times {'benchmarker': [(1734626273, 1734626593)]} YugabyteDB-64-8-65536-1 : collecting execution metrics of SUT YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yb-tserver-(.*)"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} YugabyteDB-64-8-65536-1 : collecting metrics of benchmarker -YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yugabytedb-64-8-65536-1734598346(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yugabytedb-64-8-65536-1734626144(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} done YugabyteDB-64-8-65536 : can be stopped -Experiment : ends at 2024-12-19 10:00:57.154412 (4283658.561544618) - 510.50s total +Experiment : ends at 2024-12-19 17:44:14.966144 (4311456.373277081) - 510.81s total Experiment : downloading partial results Experiment : uploading full results Experiment : uploading workload file @@ -79,7 +79,7 @@ Result workflow complete YCSB SF=1 Type: ycsb Duration: 511s - Code: 1734598346 + Code: 1734626144 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. @@ -103,7 +103,7 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:248621276 + disk:249253676 datadisk:39268 requests_cpu:4 requests_memory:16Gi @@ -113,25 +113,25 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:444140892 + disk:441407636 worker 1 RAM:540587499520 CPU:AMD EPYC 7502 32-Core Processor Cores:128 host:5.15.0-126-generic node:cl-worker22 - disk:125097616 + disk:125219072 worker 2 RAM:1081965555712 CPU:AMD EPYC 7742 64-Core Processor Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:561947884 + disk:586475888 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -YugabyteDB-64-8-65536-1 1 64 65536 1 30985.17 322735.0 4997322 39711.0 5002678 44959.0 +YugabyteDB-64-8-65536-1 1 64 65536 1 31287.55 319616.0 4999554 39551.0 5000446 43999.0 ### Workflow @@ -143,11 +143,11 @@ DBMS YugabyteDB-64-8-65536 - Pods [[1]] ### Execution - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 14114.83 15.67 7.24 23.99 +YugabyteDB-64-8-65536-1 13772.52 16.0 7.23 23.99 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 962.1 3.24 0.61 0.61 +YugabyteDB-64-8-65536-1 982.63 3.4 0.61 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN diff --git a/logs_tests/doc_ycsb_yugabytedb_2_summary.txt b/logs_tests/doc_ycsb_yugabytedb_2_summary.txt index 6b4eeb06..35bfad77 100644 --- a/logs_tests/doc_ycsb_yugabytedb_2_summary.txt +++ b/logs_tests/doc_ycsb_yugabytedb_2_summary.txt @@ -4,7 +4,7 @@ YCSB SF=1 Type: ycsb Duration: 511s - Code: 1734598346 + Code: 1734626144 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. @@ -28,7 +28,7 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:248621276 + disk:249253676 datadisk:39268 requests_cpu:4 requests_memory:16Gi @@ -38,25 +38,25 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:444140892 + disk:441407636 worker 1 RAM:540587499520 CPU:AMD EPYC 7502 32-Core Processor Cores:128 host:5.15.0-126-generic node:cl-worker22 - disk:125097616 + disk:125219072 worker 2 RAM:1081965555712 CPU:AMD EPYC 7742 64-Core Processor Cores:256 host:5.15.0-1067-nvidia node:cl-worker27 - disk:561947884 + disk:586475888 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -YugabyteDB-64-8-65536-1 1 64 65536 1 30985.17 322735.0 4997322 39711.0 5002678 44959.0 +YugabyteDB-64-8-65536-1 1 64 65536 1 31287.55 319616.0 4999554 39551.0 5000446 43999.0 ### Workflow @@ -68,11 +68,11 @@ DBMS YugabyteDB-64-8-65536 - Pods [[1]] ### Execution - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 14114.83 15.67 7.24 23.99 +YugabyteDB-64-8-65536-1 13772.52 16.0 7.23 23.99 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 962.1 3.24 0.61 0.61 +YugabyteDB-64-8-65536-1 982.63 3.4 0.61 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN diff --git a/logs_tests/doc_ycsb_yugabytedb_3.log b/logs_tests/doc_ycsb_yugabytedb_3.log index e6bc693e..2f56c59e 100644 --- a/logs_tests/doc_ycsb_yugabytedb_3.log +++ b/logs_tests/doc_ycsb_yugabytedb_3.log @@ -2,8 +2,8 @@ Data Directory : is running Result Directory : is running Dashboard : is running Message Queue : is running -Experiment : has code 1734599008 -Experiment : starts at 2024-12-19 10:03:27.922856 (4283809.329984883) +Experiment : has code 1734626805 +Experiment : starts at 2024-12-19 17:46:45.657557 (4311607.064689156) Experiment : This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. YCSB is performed using several threads and processes. Target is based on multiples of '16384'. Factors for loading are [4]. Factors for benchmarking are [4]. @@ -23,10 +23,10 @@ done YugabyteDB-64-8-65536 : will start now - waiting 30s - : done YugabyteDB-64-8-65536 : is not loaded yet -YugabyteDB-64-8-65536 : will start loading but not before 2024-12-19 09:05:28 (that is in 60 secs) +YugabyteDB-64-8-65536 : will start loading but not before 2024-12-19 16:48:46 (that is in 60 secs) - waiting 30s - : done YugabyteDB-64-8-65536 : is not loaded yet -YugabyteDB-64-8-65536 : will start loading but not before 2024-12-19 09:05:28 +YugabyteDB-64-8-65536 : will start loading but not before 2024-12-19 16:48:46 done YugabyteDB-64-8-65536 : is not loaded yet YugabyteDB-64-8-65536 : start asynch loading scripts of type loaded @@ -35,22 +35,24 @@ YugabyteDB-64-8-65536 : is loading YugabyteDB-64-8-65536 : is loading done YugabyteDB-64-8-65536 : showing loader times -YugabyteDB-64-8-65536 : loader timespan (first to last [s]) = 34 +YugabyteDB-64-8-65536 : loader timespan (first to last [s]) = 37 YugabyteDB-64-8-65536 : benchmarks done 0 of 1. This will be client 1 YugabyteDB-64-8-65536 : we will change parameters of benchmark as {'PARALLEL': '1', 'SF': '1', 'BEXHOMA_SYNCH_LOAD': 1, 'YCSB_THREADCOUNT': 64, 'YCSB_TARGET': 65536, 'YCSB_STATUS': 1, 'YCSB_WORKLOAD': 'a', 'YCSB_ROWS': 1000000, 'YCSB_OPERATIONS': 10000000, 'YCSB_BATCHSIZE': ''} YugabyteDB-64-8-65536-1 : start benchmarking YugabyteDB-64-8-65536 : distributed system - get host info for worker yb-tserver-0 YugabyteDB-64-8-65536 : distributed system - get host info for worker yb-tserver-1 YugabyteDB-64-8-65536 : distributed system - get host info for worker yb-tserver-2 -YugabyteDB-64-8-65536 : benchmarking results in folder /home/perdelt/benchmarks/1734599008 +YugabyteDB-64-8-65536 : benchmarking results in folder /home/perdelt/benchmarks/1734626805 - waiting 10s - : done found YugabyteDB-64-8-65536-1 : collecting loading metrics of SUT YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yb-tserver-(.*)"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} YugabyteDB-64-8-65536-1 : collecting metrics of data generator -YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yugabytedb-64-8-65536-1734599008(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yugabytedb-64-8-65536-1734626805(.*)", container="datagenerator"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} YugabyteDB-64-8-65536-1 : collecting metrics of data injector -YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yugabytedb-64-8-65536-1734599008(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yugabytedb-64-8-65536-1734626805(.*)", container="sensor"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +- waiting 30s - : done +YugabyteDB-64-8-65536 : has running benchmarks - waiting 30s - : done YugabyteDB-64-8-65536 : has running benchmarks - waiting 30s - : done @@ -72,16 +74,16 @@ YugabyteDB-64-8-65536 : has running benchmarks done YugabyteDB-64-8-65536 : has running benchmarks YugabyteDB-64-8-65536-1 : showing benchmarker times -YugabyteDB-64-8-65536-1 : benchmarker timespan (start to end single container [s]) = 325 -YugabyteDB-64-8-65536-1 : benchmarker times (start/end per pod and container) = [(1734599204, 1734599524)] -YugabyteDB-64-8-65536-1 : found and updated times {'benchmarker': [(1734599204, 1734599524)]} +YugabyteDB-64-8-65536-1 : benchmarker timespan (start to end single container [s]) = 355 +YugabyteDB-64-8-65536-1 : benchmarker times (start/end per pod and container) = [(1734627001, 1734627326)] +YugabyteDB-64-8-65536-1 : found and updated times {'benchmarker': [(1734627001, 1734627326)]} YugabyteDB-64-8-65536-1 : collecting execution metrics of SUT YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yb-tserver-(.*)"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} YugabyteDB-64-8-65536-1 : collecting metrics of benchmarker -YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yugabytedb-64-8-65536-1734599008(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} +YugabyteDB-64-8-65536-1 : example metric {'query': '(sum(max(container_memory_working_set_bytes{pod=~"(.*)yugabytedb-64-8-65536-1734626805(.*)", container="dbmsbenchmarker"}) by (pod)))/1024/1024', 'title': 'CPU Memory [MiB]'} done YugabyteDB-64-8-65536 : can be stopped -Experiment : ends at 2024-12-19 10:12:49.248938 (4284370.656070662) - 561.33s total +Experiment : ends at 2024-12-19 17:56:35.924937 (4312197.33207037) - 590.27s total Experiment : downloading partial results Experiment : uploading full results Experiment : uploading workload file @@ -92,8 +94,8 @@ Result workflow complete ### Workload YCSB SF=1 Type: ycsb - Duration: 562s - Code: 1734599008 + Duration: 591s + Code: 1734626805 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. @@ -117,8 +119,8 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:248582020 - datadisk:39262 + disk:249214420 + datadisk:39106 volume_size:1.0G volume_used:36M requests_cpu:4 @@ -129,29 +131,29 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:441849660 + disk:439198928 worker 1 - RAM:1081965555712 - CPU:AMD EPYC 7742 64-Core Processor - Cores:256 - host:5.15.0-1067-nvidia - node:cl-worker27 - disk:559737852 - worker 2 RAM:540587499520 CPU:AMD EPYC 7502 32-Core Processor Cores:128 host:5.15.0-126-generic node:cl-worker22 - disk:122889020 + disk:122937852 + worker 2 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:584265648 ### Loading experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [INSERT].Return=OK [INSERT].99thPercentileLatency(us) -YugabyteDB-64-8-65536 1 64 65536 8 29685.991562 34115.0 1000000 11968.0 +YugabyteDB-64-8-65536 1 64 65536 8 27920.198198 36082.0 1000000 15099.0 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -YugabyteDB-64-8-65536-1 1 64 65536 1 31292.83 319562.0 4998841 26335.0 5001159 40159.0 +YugabyteDB-64-8-65536-1 1 64 65536 1 30852.87 324119.0 4999076 26703.0 5000924 44287.0 ### Workflow @@ -163,19 +165,19 @@ DBMS YugabyteDB-64-8-65536 - Pods [[1]] ### Ingestion - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 25686.07 1.15 6.32 20.86 +YugabyteDB-64-8-65536-1 14677.38 4.86 4.23 13.99 ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 36.85 0 2.05 2.06 +YugabyteDB-64-8-65536-1 0.08 0 0.01 0.01 ### Execution - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 14651.28 15.71 5.01 16.47 +YugabyteDB-64-8-65536-1 14586.01 17.96 4.88 16.69 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 1009.99 3.37 0.61 0.61 +YugabyteDB-64-8-65536-1 771.47 2.69 0.61 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN diff --git a/logs_tests/doc_ycsb_yugabytedb_3_summary.txt b/logs_tests/doc_ycsb_yugabytedb_3_summary.txt index de1524e4..aeb7f953 100644 --- a/logs_tests/doc_ycsb_yugabytedb_3_summary.txt +++ b/logs_tests/doc_ycsb_yugabytedb_3_summary.txt @@ -3,8 +3,8 @@ ### Workload YCSB SF=1 Type: ycsb - Duration: 562s - Code: 1734599008 + Duration: 591s + Code: 1734626805 YCSB tool runs the benchmark. This experiment compares run time and resource consumption of YCSB queries. Workload is 'A'. Number of rows to insert is 1000000. Number of operations is 10000000. Batch size is ''. @@ -28,8 +28,8 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:64 host:5.15.0-126-generic node:cl-worker11 - disk:248582020 - datadisk:39262 + disk:249214420 + datadisk:39106 volume_size:1.0G volume_used:36M requests_cpu:4 @@ -40,29 +40,29 @@ YugabyteDB-64-8-65536-1 uses docker image postgres:15.0 Cores:256 host:5.15.0-1067-nvidia node:cl-worker28 - disk:441849660 + disk:439198928 worker 1 - RAM:1081965555712 - CPU:AMD EPYC 7742 64-Core Processor - Cores:256 - host:5.15.0-1067-nvidia - node:cl-worker27 - disk:559737852 - worker 2 RAM:540587499520 CPU:AMD EPYC 7502 32-Core Processor Cores:128 host:5.15.0-126-generic node:cl-worker22 - disk:122889020 + disk:122937852 + worker 2 + RAM:1081965555712 + CPU:AMD EPYC 7742 64-Core Processor + Cores:256 + host:5.15.0-1067-nvidia + node:cl-worker27 + disk:584265648 ### Loading experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [INSERT].Return=OK [INSERT].99thPercentileLatency(us) -YugabyteDB-64-8-65536 1 64 65536 8 29685.991562 34115.0 1000000 11968.0 +YugabyteDB-64-8-65536 1 64 65536 8 27920.198198 36082.0 1000000 15099.0 ### Execution experiment_run threads target pod_count [OVERALL].Throughput(ops/sec) [OVERALL].RunTime(ms) [READ].Return=OK [READ].99thPercentileLatency(us) [UPDATE].Return=OK [UPDATE].99thPercentileLatency(us) -YugabyteDB-64-8-65536-1 1 64 65536 1 31292.83 319562.0 4998841 26335.0 5001159 40159.0 +YugabyteDB-64-8-65536-1 1 64 65536 1 30852.87 324119.0 4999076 26703.0 5000924 44287.0 ### Workflow @@ -74,19 +74,19 @@ DBMS YugabyteDB-64-8-65536 - Pods [[1]] ### Ingestion - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 25686.07 1.15 6.32 20.86 +YugabyteDB-64-8-65536-1 14677.38 4.86 4.23 13.99 ### Ingestion - Loader CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 36.85 0 2.05 2.06 +YugabyteDB-64-8-65536-1 0.08 0 0.01 0.01 ### Execution - SUT CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 14651.28 15.71 5.01 16.47 +YugabyteDB-64-8-65536-1 14586.01 17.96 4.88 16.69 ### Execution - Benchmarker CPU [CPUs] Max CPU Max RAM [Gb] Max RAM Cached [Gb] -YugabyteDB-64-8-65536-1 1009.99 3.37 0.61 0.61 +YugabyteDB-64-8-65536-1 771.47 2.69 0.61 0.61 ### Tests TEST passed: [OVERALL].Throughput(ops/sec) contains no 0 or NaN diff --git a/test-cloud.sh b/test-cloud.sh index 2e70e9ce..e81f03c4 100755 --- a/test-cloud.sh +++ b/test-cloud.sh @@ -383,10 +383,11 @@ sleep 30 # start database service placeholder kubectl create -f k8s/deploymenttemplate-PostgreSQLService.yml -sleep 10 +# we have to be sure the "cloud service" is ready - bexhoma does not check this in case dbms is not managed by bexhoma +sleep 300 # delete pvc of placeholder -kubectl delete pvc bexhoma-storage-databaseservice-ycsb-1 +kubectl delete pvc bexhoma-storage-databaseservice-ycsb-5 sleep 10 @@ -598,6 +599,9 @@ nohup python tpch.py -ms 2 -dt -tr \ wait_process "tpch" +# delete database service placeholder +kubectl delete deployment bexhoma-deployment-postgres +kubectl delete svc bexhoma-service