Reduce thread-count in benchmarks

GitHub's linux runners only have two cores. Therefore, running 4 threads for the benchmarks doesn't make sense. Could run 2 threads, but then other OS/GC operations can skew benchmark results. Reducing to 1 thread should give the most stable results.
creek-service · Nov 16, 2023 · 8e60d89 · 8e60d89
1 parent 3bcbc3b
commit 8e60d89
Show file tree

Hide file tree

Showing 6 changed files with 13 additions and 12 deletions.
diff --git a/README.md b/README.md
@@ -55,11 +55,11 @@ Running the functional tests will create result files in the `docs/_include` dir
 
 Generated files:
 
-| filename                  | description                                                                                                                                                                           | use                                                                     |
-|---------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-------------------------------------------------------------------------|
-| `functional-summary.json` | JSON document containing a summary of pass/fail rates of required/optional test cases for each implementation, per supported JSON schema version.                                     | Used to build functional tables and charts in [micro-site][micro-site]. |
-| `functional-summary.md`   | Markdown document containing a summary of pass/fail rates of required/optional test cases for each implementation, per supported JSON schema version.                                 | Appended to the GitHub workflow job                                     |
-| `per-draft.md`            | Markdown document containing one table for each implementation and supported schema specification combination, showing the number of test cases that pass and fail in each test file. | Appended to the GitHub workflow job                                     |
+| filename                  | description                                                                                                                                                                           | use                                                                                                             |
+|---------------------------|---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|-----------------------------------------------------------------------------------------------------------------|
+| `functional-summary.json` | JSON document containing a summary of pass/fail rates of required/optional test cases for each implementation, per supported JSON schema version.                                     | Used to build functional tables and charts in [micro-site][micro-site].                                         |
+| `functional-summary.md`   | Markdown document containing a summary of pass/fail rates of required/optional test cases for each implementation, per supported JSON schema version.                                 | Appended to the GitHub workflow job                                                                             |
+| `per-draft.md`            | Markdown document containing one table for each implementation and supported schema specification combination, showing the number of test cases that pass and fail in each test file. | Used to build functional tables and charts in [micro-site][micro-site], and appended to the GitHub workflow job |
 
 #### Performance comparison
 

diff --git a/buildSrc/src/main/kotlin/creek-plugin-publishing-convention.gradle.kts b/buildSrc/src/main/kotlin/creek-plugin-publishing-convention.gradle.kts
@@ -95,9 +95,9 @@ publishing {
             name.set("${project.group}:${artifactId}")
 
             if (prependRootName) {
-                description.set("${rootProject.name.capitalize()} ${project.name} library".replace("-", " "))
+                description.set("${rootProject.name} ${project.name} library".replace("-", " "))
             } else {
-                description.set("${project.name.capitalize()} library".replace("-", " "))
+                description.set("${project.name} library".replace("-", " "))
             }
 
             url.set("https://www.creekservice.org")

diff --git a/buildSrc/src/main/kotlin/creek-publishing-convention.gradle.kts b/buildSrc/src/main/kotlin/creek-publishing-convention.gradle.kts
@@ -77,9 +77,9 @@ publishing {
                 name.set("${project.group}:${artifactId}")
 
                 if (prependRootName) {
-                    description.set("${rootProject.name.capitalize()} ${project.name} library".replace("-", " "))
+                    description.set("${rootProject.name} ${project.name} library".replace("-", " "))
                 } else {
-                    description.set("${project.name.capitalize()} library".replace("-", " "))
+                    description.set("${project.name} library".replace("-", " "))
                 }
 
                 url.set("https://www.creekservice.org")

diff --git a/docs/_docs/3. performance.md b/docs/_docs/3. performance.md
@@ -22,7 +22,8 @@ real-world example, using a small common subset of functionality, in the context
 as a serialization format.  Combined, these should give a good comparison of performance.
 
 **Note:**
-The benchmarks are run on GitHub's own infrastructure. These may not be dedicated machines, which can influence performance results.   
+The benchmarks are run on [GitHub's own infrastructure](https://docs.github.com/en/actions/using-github-hosted-runners/about-github-hosted-runners/about-github-hosted-runners). 
+These may not be dedicated machines, which can influence performance results.   
 {: .notice--warning}
 
 ### JSON schema test suite benchmark

diff --git a/src/main/java/org/creekservice/kafka/test/perf/performance/JsonSerdeBenchmark.java b/src/main/java/org/creekservice/kafka/test/perf/performance/JsonSerdeBenchmark.java
@@ -61,7 +61,7 @@
  */
 @BenchmarkMode(Mode.AverageTime)
 @OutputTimeUnit(MICROSECONDS)
-@Threads(4)
+@Threads(1) // GitHub linux runners have two cores, so running more threads is pointless.
 @Fork(4) // Note: to debug, set fork to 0.
 // @Warmup(iterations = 0, time = 10)
 // @Measurement(iterations = 1, time = 10)

diff --git a/src/main/java/org/creekservice/kafka/test/perf/performance/JsonValidateBenchmark.java b/src/main/java/org/creekservice/kafka/test/perf/performance/JsonValidateBenchmark.java
@@ -54,7 +54,7 @@
  */
 @BenchmarkMode(Mode.AverageTime)
 @OutputTimeUnit(MILLISECONDS)
-@Threads(4)
+@Threads(1) // GitHub linux runners have two cores, so running more threads is pointless.
 @Fork(4) // Note: to debug, set fork to 0.
 // @Warmup(iterations = 0, time = 10)
 // @Measurement(iterations = 1, time = 10)