From 1b734658b0bff672b4a34dc46c2b6144fd306d9d Mon Sep 17 00:00:00 2001 From: Jordan Last Date: Sat, 6 Aug 2022 13:57:59 -0500 Subject: [PATCH 1/2] adding the TLDR section --- README.md | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/README.md b/README.md index ef4d403675..935f6e0c78 100644 --- a/README.md +++ b/README.md @@ -2438,6 +2438,34 @@ The following is a comparison of all of the major features of the [Rust CDK](htt ### Benchmarks +#### TLDR + +Last major update to this benchmarking information: August 2022 + +Here's a quick rough estimate summary of the benchmarks: + +Average/Median Azle Wasm instructions per function call: 121_919_698 / 2_387_322 +Average/Median Motoko Wasm instructions per function call: 272_877 / 5_169 +Average/Median Rust Wasm instructions per function call: 26_378_678 / 44_074 + +Average/Median Azle/Motoko Wasm instructions difference: 1_273x / 396x +Average/Median Azle/Rust Wasm instructions difference: 204x / 51x +Average/Median Motoko/Rust Wasm instructions difference: -22x / -8x + +Average/Median Azle/Motoko USD cost estimate difference using average Wasm instructions: 5x / 3x +Maximum Azle/Motoko USD cost estimate difference using average Wasm instructions: 13x + +Average/Median Azle/Rust USD cost estimate difference using average Wasm instructions: 2x / 2x +Maximum Azle/Rust USD cost estimate difference using average Wasm instructions: 4x + +Average/Median Azle/Motoko USD cost difference using median Wasm instructions: 1x / 1x +Maximum Azle/Motoko USD cost difference using median Wasm instructions: 1x + +Average/Median Azle/Rust USD cost difference using median Wasm instructions: 1x / 1x +Maximum Azle/Rust USD cost difference using median Wasm instructions: 1x + +#### Introduction to the Benchmarks + Azle's automated benchmarking framework is currently based on the `ic0.performance_counter` [System API](https://internetcomputer.org/docs/current/references/ic-interface-spec/#system-api-imports). `ic0.performance_counter` seems to have a number of limitations. The following may not be measured accurately: - Candid serialization/deserialization of function parameters and return types From ca6b847cfb48e547cc38bd3aefd84957592ed6ee Mon Sep 17 00:00:00 2001 From: Jordan Last Date: Sat, 6 Aug 2022 14:00:57 -0500 Subject: [PATCH 2/2] adding the TLDR section --- README.md | 28 ++++++++++++++-------------- 1 file changed, 14 insertions(+), 14 deletions(-) diff --git a/README.md b/README.md index 935f6e0c78..155df1e59f 100644 --- a/README.md +++ b/README.md @@ -2444,25 +2444,25 @@ Last major update to this benchmarking information: August 2022 Here's a quick rough estimate summary of the benchmarks: -Average/Median Azle Wasm instructions per function call: 121_919_698 / 2_387_322 -Average/Median Motoko Wasm instructions per function call: 272_877 / 5_169 -Average/Median Rust Wasm instructions per function call: 26_378_678 / 44_074 +- Average/Median Azle Wasm instructions per function call: 121_919_698 / 2_387_322 +- Average/Median Motoko Wasm instructions per function call: 272_877 / 5_169 +- Average/Median Rust Wasm instructions per function call: 26_378_678 / 44_074 -Average/Median Azle/Motoko Wasm instructions difference: 1_273x / 396x -Average/Median Azle/Rust Wasm instructions difference: 204x / 51x -Average/Median Motoko/Rust Wasm instructions difference: -22x / -8x +- Average/Median Azle/Motoko Wasm instructions difference: 1_273x / 396x +- Average/Median Azle/Rust Wasm instructions difference: 204x / 51x +- Average/Median Motoko/Rust Wasm instructions difference: -22x / -8x -Average/Median Azle/Motoko USD cost estimate difference using average Wasm instructions: 5x / 3x -Maximum Azle/Motoko USD cost estimate difference using average Wasm instructions: 13x +- Average/Median Azle/Motoko USD cost estimate difference using average Wasm instructions: 5x / 3x +- Maximum Azle/Motoko USD cost estimate difference using average Wasm instructions: 13x -Average/Median Azle/Rust USD cost estimate difference using average Wasm instructions: 2x / 2x -Maximum Azle/Rust USD cost estimate difference using average Wasm instructions: 4x +- Average/Median Azle/Rust USD cost estimate difference using average Wasm instructions: 2x / 2x +- Maximum Azle/Rust USD cost estimate difference using average Wasm instructions: 4x -Average/Median Azle/Motoko USD cost difference using median Wasm instructions: 1x / 1x -Maximum Azle/Motoko USD cost difference using median Wasm instructions: 1x +- Average/Median Azle/Motoko USD cost difference using median Wasm instructions: 1x / 1x +- Maximum Azle/Motoko USD cost difference using median Wasm instructions: 1x -Average/Median Azle/Rust USD cost difference using median Wasm instructions: 1x / 1x -Maximum Azle/Rust USD cost difference using median Wasm instructions: 1x +- Average/Median Azle/Rust USD cost difference using median Wasm instructions: 1x / 1x +- Maximum Azle/Rust USD cost difference using median Wasm instructions: 1x #### Introduction to the Benchmarks