Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/arm64: 268 Regressions on 7/29/2023 7:04:01 PM #89940

Closed
performanceautofiler bot opened this issue Aug 3, 2023 · 31 comments
Closed

[Perf] Linux/arm64: 268 Regressions on 7/29/2023 7:04:01 PM #89940

performanceautofiler bot opened this issue Aug 3, 2023 · 31 comments
Assignees
@EgorBo
Labels
arch-arm64 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI os-linux Linux OS (any supported distro) runtime-coreclr specific to the CoreCLR runtime tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Milestone
8.0.0

Comments

@performanceautofiler
Copy link

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
23.85 ns 26.14 ns 1.10 0.05 False
33.49 ns 37.63 ns 1.12 0.02 False
33.23 ns 37.92 ns 1.14 0.02 False
11.17 ns 14.99 ns 1.34 0.04 False
33.40 ns 36.51 ns 1.09 0.03 False
6.22 ns 7.81 ns 1.25 0.40 False
11.29 ns 14.34 ns 1.27 0.05 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.137705630589398 > 24.946168217632582.
IsChangePoint: Marked as a change because one of 5/14/2023 7:30:23 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.96988176081783 (T) = (0 -26.177414265844696) / Math.Sqrt((0.1795955695337053 / (17)) + (0.0053610676933496864 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.09321733472089504 = (23.945297457735563 - 26.177414265844696) / 23.945297457735563 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParse(value: "-9223372036854775808")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.63239881623019 > 35.19169745471809.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.655672490042395 (T) = (0 -37.27632669811096) / Math.Sqrt((0.05575448746761843 / (19)) + (0.5155246679550679 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.11281388266078136 = (33.49735951260943 - 37.27632669811096) / 33.49735951260943 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParseSpan(value: "-9223372036854775808")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.91541186721144 > 35.10850531701371.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.05495389754118 (T) = (0 -37.17139801700723) / Math.Sqrt((0.07819416609189558 / (17)) + (0.5994983827453598 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.10793876735167553 = (33.55004727008392 - 37.17139801700723) / 33.55004727008392 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.98854804202657 > 11.723113429994191.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -26.061266785816077 (T) = (0 -14.895645118514304) / Math.Sqrt((0.037213479000615945 / (19)) + (0.13681616706416241 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.3451033461516067 = (11.073978189951967 - 14.895645118514304) / 11.073978189951967 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParse(value: "9223372036854775807")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.50534067023052 > 35.07997456237158.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.417045291246943 (T) = (0 -36.63475704991735) / Math.Sqrt((0.2969835311564679 / (19)) + (0.18037131662308017 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.10032586393426689 = (33.29446144156609 - 36.63475704991735) / 33.29446144156609 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryFormat(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.809253575213003 > 6.5500550657364585.
IsChangePoint: Marked as a change because one of 4/26/2023 8:38:54 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.896320532104834 (T) = (0 -7.676588644599028) / Math.Sqrt((0.06498843870042768 / (17)) + (0.09454463207941372 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.25628422057555245 = (6.1105508760446625 - 7.676588644599028) / 6.1105508760446625 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParseSpan(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.33778883668515 > 11.681744153510063.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.206760882260394 (T) = (0 -14.71737208357051) / Math.Sqrt((0.025067926714099274 / (17)) + (0.21036756619539698 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.3223766431694294 = (11.129485808442888 - 14.71737208357051) / 11.129485808442888 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.85 ns 4.45 ns 2.41 0.55 False
25.30 ns 28.12 ns 1.11 0.17 False
1.82 ns 4.36 ns 2.40 0.52 False
1.54 ns 4.46 ns 2.89 0.53 False
1.87 ns 4.34 ns 2.33 0.51 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'

Payloads

Baseline
Compare

System.Tests.Perf_Char.Char_ToUpper(c: 'A', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.454784779761455 > 1.940471593226209.
IsChangePoint: Marked as a change because one of 4/27/2023 8:46:44 AM, 5/18/2023 7:10:13 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.404654951633336 (T) = (0 -4.29344463753373) / Math.Sqrt((0.03263198650029867 / (17)) + (0.12907222072202787 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.3719410274745631 = (1.810097547874121 - 4.29344463753373) / 1.810097547874121 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Char.Char_ToLower(c: 'İ', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.124586897956824 > 26.59612595016098.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/10/2023 2:36:50 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -6.765847488000034 (T) = (0 -27.603716528515747) / Math.Sqrt((0.4517760996729567 / (17)) + (0.3938400422609239 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.07589988531982853 = (25.65639880174362 - 27.603716528515747) / 25.65639880174362 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Char.Char_ToLower(c: 'A', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.355267051395538 > 2.0041186461607876.
IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -52.16133250015398 (T) = (0 -4.44309144718427) / Math.Sqrt((0.03545241092210663 / (17)) + (0.002638495898547744 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.3955452567372602 = (1.8547307485377145 - 4.44309144718427) / 1.8547307485377145 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Char.Char_ToUpper(c: 'a', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.460913033845659 > 1.8259304854261336.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.024422375425454 (T) = (0 -4.144715129014192) / Math.Sqrt((0.0336722002161439 / (17)) + (0.2511553105099265 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.2961849969320636 = (1.8050440772637886 - 4.144715129014192) / 1.8050440772637886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Char.Char_ToLower(c: 'a', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.341934270862805 > 1.9843361565410813.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.620088663019327 (T) = (0 -4.288372340468756) / Math.Sqrt((0.014158487864279378 / (17)) + (0.13863464484267096 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.256143690366358 = (1.9007532005961907 - 4.288372340468756) / 1.9007532005961907 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler performanceautofiler bot added arch-arm64 os-linux Linux OS (any supported distro) runtime-coreclr specific to the CoreCLR runtime untriaged New issue has not been triaged by the area owner labels Aug 3, 2023
@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.89 ns 6.06 ns 2.10 0.60 True
2.87 ns 5.21 ns 1.81 0.55 True
4.43 ns 6.21 ns 1.40 0.41 False
8.70 ns 11.98 ns 1.38 0.13 False
3.76 ns 6.95 ns 1.85 0.37 True
1.31 ns 4.60 ns 3.50 0.65 False
5.61 ns 8.26 ns 1.47 0.35 False
2.11 ns 3.54 ns 1.68 0.61 False
3.95 ns 6.94 ns 1.76 0.42 False
14.36 ns 16.58 ns 1.15 0.06 False
10.01 ns 13.03 ns 1.30 0.08 False
2.06 ns 4.16 ns 2.02 0.59 False
10.36 ns 13.31 ns 1.28 0.07 False
2.33 ns 3.75 ns 1.61 0.57 False
4.45 ns 7.18 ns 1.61 0.41 True
3.43 ns 5.49 ns 1.60 0.51 False
4.07 ns 6.98 ns 1.71 0.40 False
28.55 ns 31.13 ns 1.09 0.00 False
4.27 ns 7.48 ns 1.75 0.42 True
2.66 ns 3.75 ns 1.41 0.53 False
5.92 ns 8.56 ns 1.45 0.37 False
6.60 ns 9.92 ns 1.50 0.33 True
3.74 ns 5.82 ns 1.56 0.40 False
1.47 ns 4.70 ns 3.19 0.56 False
3.58 ns 7.36 ns 2.06 0.37 True
2.76 ns 5.94 ns 2.15 0.53 False
4.41 ns 7.55 ns 1.71 0.38 False
2.02 ns 3.74 ns 1.85 0.59 False
30.63 ns 34.44 ns 1.12 0.03 False
29.07 ns 32.32 ns 1.11 0.01 False
7.38 ns 10.38 ns 1.41 0.26 False
3.20 ns 4.25 ns 1.33 0.47 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: true)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.060150125688046 > 2.8393180362117323.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.652869359842937 (T) = (0 -6.233082169075598) / Math.Sqrt((0.061306727315642164 / (17)) + (0.5437645108237029 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.3739286096746757 = (2.625640107151235 - 6.233082169075598) / 2.625640107151235 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: TRUE)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.2104444962967 > 3.012744414957247.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.123903641702578 (T) = (0 -6.127817825623278) / Math.Sqrt((0.03382566698570514 / (16)) + (0.6334472898527835 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -1.2342557942990378 = (2.7426661894574087 - 6.127817825623278) / 2.7426661894574087 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.2149464284657805 > 4.636880526475989.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.493097165882386 (T) = (0 -7.070866771841205) / Math.Sqrt((0.05425931958190923 / (17)) + (0.14297782184855426 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.6134133877191712 = (4.382551195907116 - 7.070866771841205) / 4.382551195907116 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32(value: 4294967295)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.982443953157242 > 8.951715687665065.
IsChangePoint: Marked as a change because one of 4/26/2023 5:34:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.499752082466916 (T) = (0 -11.831884972764868) / Math.Sqrt((0.04516764815558914 / (19)) + (0.14986935736687715 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.3646625619276335 = (8.670190934271632 - 11.831884972764868) / 8.670190934271632 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: 3039)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.954263701428884 > 3.7260771988896675.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -49.83310350659807 (T) = (0 -6.931206249006684) / Math.Sqrt((0.07472960590014388 / (16)) + (0.0006772354056236843 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.9857740634908029 = (3.4904304454567603 - 6.931206249006684) / 3.4904304454567603 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.595433833463872 > 1.4359223764198803.
IsChangePoint: Marked as a change because one of 5/19/2023 9:46:56 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.0078897782295 (T) = (0 -4.412953645602964) / Math.Sqrt((0.1533897987436705 / (19)) + (0.1337874754282348 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -2.450532618147583 = (1.2789195564747498 - 4.412953645602964) / 1.2789195564747498 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: 32767)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.263298775515965 > 5.674711240786213.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/9/2023 12:26:27 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.210456616829628 (T) = (0 -8.203635766884366) / Math.Sqrt((0.07685194940644527 / (18)) + (0.11380334301062994 / (7))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (7) - 2, .025) and -0.545516662733933 = (5.3080215598404425 - 8.203635766884366) / 5.3080215598404425 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32(value: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.540479339994556 > 2.31920509846433.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 3:52:42 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.064414333476678 (T) = (0 -3.3834932366732664) / Math.Sqrt((0.08826288697376317 / (19)) + (0.15396697769221307 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.6367349125101222 = (2.067221277442105 - 3.3834932366732664) / 2.067221277442105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.942298462735907 > 4.096354876964611.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.247699196485247 (T) = (0 -6.832237832107334) / Math.Sqrt((0.0793807854264958 / (19)) + (0.14303727429290264 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.7920021087830447 = (3.8126282321995326 - 6.832237832107334) / 3.8126282321995326 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: FFFFFFFFFFFFFFFF)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.583243765413112 > 14.954570600293723.
IsChangePoint: Marked as a change because one of 4/29/2023 8:32:21 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.76723710218092 (T) = (0 -16.480056310933286) / Math.Sqrt((0.11157128917623173 / (16)) + (0.10772367029770423 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.16695941169592596 = (14.122218944173087 - 16.480056310933286) / 14.122218944173087 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: 2147483647)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.033939229223856 > 10.587921911569946.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -31.69459200620588 (T) = (0 -13.003123104257076) / Math.Sqrt((0.16286766935144567 / (19)) + (0.004076149840172549 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.3041404492936974 = (9.970646268428657 - 13.003123104257076) / 9.970646268428657 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.163038910766477 > 2.1319820822666484.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 6/21/2023 4:19:00 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.668843554616505 (T) = (0 -4.128471269894606) / Math.Sqrt((0.3751242494649259 / (16)) + (0.0033396251377079563 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.9033986131389835 = (2.1689998308268974 - 4.128471269894606) / 2.1689998308268974 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: -2147483648)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.311555789460005 > 10.851962924613646.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -61.30113479802869 (T) = (0 -13.305194384715715) / Math.Sqrt((0.03744115264051727 / (16)) + (0.000765943116036145 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.2953812752078598 = (10.271257304210094 - 13.305194384715715) / 10.271257304210094 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: false)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.7524874702405224 > 2.523759505076376.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -32.73525901670061 (T) = (0 -3.7479123949298585) / Math.Sqrt((0.028035646792395636 / (16)) + (0.00015688190240579605 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.5821376817882935 = (2.3688914296596395 - 3.7479123949298585) / 2.3688914296596395 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 65535)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.181968726973394 > 4.679027251781499.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -43.45075079932444 (T) = (0 -7.213165604470514) / Math.Sqrt((0.07449350914287153 / (17)) + (0.0009621458917078363 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.6805609538008383 = (4.292117812303605 - 7.213165604470514) / 4.292117812303605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseByte(value: 255)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.492376654769195 > 3.779806842529444.
IsChangePoint: Marked as a change because one of 4/26/2023 5:34:01 AM, 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.76505254401687 (T) = (0 -5.478347134434996) / Math.Sqrt((0.12092370645066228 / (17)) + (0.004325292552656163 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.5757331798803546 = (3.476697199998649 - 5.478347134434996) / 3.476697199998649 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseSByte(value: -128)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.980769543306396 > 4.361487885056127.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.083223785158847 (T) = (0 -6.680542658663618) / Math.Sqrt((0.06438578208708108 / (17)) + (0.20129412215227108 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.6871982031155096 = (3.9595482299160865 - 6.680542658663618) / 3.9595482299160865 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 9223372036854775807)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 31.128062510700452 > 30.060702235406747.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.861211647885392 (T) = (0 -31.471003243217865) / Math.Sqrt((0.0770240564135543 / (16)) + (0.23134768591522747 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.10118478257762045 = (28.579220981924106 - 31.471003243217865) / 28.579220981924106 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt32(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.478078336633942 > 4.567102183434747.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -31.1481278553222 (T) = (0 -7.448871126717838) / Math.Sqrt((0.16044728806385802 / (16)) + (0.0038600238098759035 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.7547008418034473 = (4.245094633374674 - 7.448871126717838) / 4.245094633374674 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: True )

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.750682074534774 > 2.6675772275406064.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.92893153053267 (T) = (0 -3.7986132542070226) / Math.Sqrt((0.07297043577582636 / (16)) + (0.006519635351460401 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.49314198035056517 = (2.544040221356023 - 3.7986132542070226) / 2.544040221356023 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: -32768)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.560751957406698 > 5.782953655662641.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -44.29745435248982 (T) = (0 -8.56283799460387) / Math.Sqrt((0.08305340581949877 / (19)) + (0.0004318385775016923 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5253996580605754 = (5.613504598192213 - 8.56283799460387) / 5.613504598192213 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: FFFFFFFFFFFFFFFF)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.92286305302456 > 6.842825981700517.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -65.29354521764932 (T) = (0 -9.902878168887325) / Math.Sqrt((0.044181360252925725 / (19)) + (0.0030473320321082937 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5300587409298764 = (6.472220904975818 - 9.902878168887325) / 6.472220904975818 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseSByte(value: 127)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.819830843052835 > 3.8962793334251042.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -24.677936747206918 (T) = (0 -6.428576941912474) / Math.Sqrt((0.043747347416222854 / (17)) + (0.07292647207225728 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.7779198036144389 = (3.6157856663970094 - 6.428576941912474) / 3.6157856663970094 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64Hex(value: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.697863691807611 > 1.5015470171392074.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.52260285081556 (T) = (0 -4.531051342547648) / Math.Sqrt((0.07434337948391584 / (17)) + (0.14734935936157742 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -2.5994919469757254 = (1.2588030225639522 - 4.531051342547648) / 1.2588030225639522 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.364726714523267 > 4.215648812723667.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -48.26153961104548 (T) = (0 -7.34896883736837) / Math.Sqrt((0.07382515623948822 / (15)) + (0.0008830521839955511 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.8746987448466802 = (3.920079883538513 - 7.34896883736837) / 3.920079883538513 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt32Hex(value: 3039)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.942068547278083 > 3.3873784057587977.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.110661837679242 (T) = (0 -6.33867160981337) / Math.Sqrt((0.05588524154351604 / (19)) + (0.203843342905814 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.0474996987490517 = (3.095810765533235 - 6.33867160981337) / 3.095810765533235 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.5507380210777 > 4.768307887955589.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/7/2023 7:43:49 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -51.76171954137517 (T) = (0 -7.644464630773882) / Math.Sqrt((0.053694755107667295 / (16)) + (0.0025890216893391517 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.7044623237237789 = (4.484971315806407 - 7.644464630773882) / 4.484971315806407 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseBool(value: False)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.740158393650813 > 2.4827472819862435.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -23.364171078265727 (T) = (0 -3.803398140464817) / Math.Sqrt((0.03882327940956414 / (16)) + (0.011176840290329594 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.6383724699395503 = (2.321449005185706 - 3.803398140464817) / 2.321449005185706 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt64(value: 18446744073709551615)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 34.442287732144635 > 32.41623327404391.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.850054052625389 (T) = (0 -34.14900169433548) / Math.Sqrt((0.33431077688348587 / (16)) + (0.5288769990718987 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.10947603513592281 = (30.779395510018258 - 34.14900169433548) / 30.779395510018258 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt64(value: -9223372036854775808)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.31722103000354 > 30.849162050962054.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.258590285026624 (T) = (0 -32.17597989860552) / Math.Sqrt((0.10978754412946518 / (16)) + (0.12263555701087059 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.09140868759682676 = (29.481146947303326 - 32.17597989860552) / 29.481146947303326 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDateTimeOffset(value: 12/30/2017 3:45:22 AM -08:00)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.383993453290415 > 7.881357430707848.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.445383204799622 (T) = (0 -10.682492440596471) / Math.Sqrt((0.042820315443092394 / (16)) + (0.156021665702069 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.433496751083664 = (7.4520520765191485 - 10.682492440596471) / 7.4520520765191485 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseInt16(value: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.251423198842611 > 3.750793597726134.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.33891131206077 (T) = (0 -4.388570726119437) / Math.Sqrt((0.03532630472901376 / (19)) + (0.031080791194122628 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.229985870661938 = (3.5679846661634027 - 4.388570726119437) / 3.5679846661634027 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.36 ns 10.27 ns 1.92 0.23 False
4.11 ns 6.28 ns 1.53 0.67 False
10.09 ns 14.77 ns 1.46 0.06 False
17.39 ns 20.40 ns 1.17 0.04 False
18.27 ns 22.81 ns 1.25 0.03 False
5.44 ns 10.33 ns 1.90 0.22 False
7.56 ns 10.23 ns 1.35 0.28 False
10.79 ns 14.81 ns 1.37 0.04 False
17.63 ns 21.76 ns 1.23 0.07 False
17.35 ns 21.25 ns 1.22 0.04 False
18.97 ns 21.77 ns 1.15 0.02 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int32*'

Payloads

Baseline
Compare

System.Tests.Perf_Int32.TryParse(value: "4")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.26723137469337 > 5.306408473907406.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -44.66599376048455 (T) = (0 -10.129045409721495) / Math.Sqrt((0.05643952720161316 / (17)) + (0.06318335458269081 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.9606908555510822 = (5.1660593923026035 - 10.129045409721495) / 5.1660593923026035 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryFormat(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.279966460227437 > 4.626287125001114.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.64625734549464 (T) = (0 -6.376160400034913) / Math.Sqrt((0.5592156566326388 / (17)) + (0.08371990796987215 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.6308988340049375 = (3.909598969040412 - 6.376160400034913) / 3.909598969040412 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.770435533699347 > 10.857240138614378.
IsChangePoint: Marked as a change because one of 4/25/2023 6:51:20 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -25.625241143813845 (T) = (0 -14.587377814095657) / Math.Sqrt((0.08765604517174783 / (17)) + (0.14968094975100124 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.4009077256702167 = (10.412804174605306 - 14.587377814095657) / 10.412804174605306 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "2147483647")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.400025737869814 > 18.19313010551981.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -31.522735995040023 (T) = (0 -20.987198830289383) / Math.Sqrt((0.054835263199147576 / (16)) + (0.07746582982326636 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.22074121852380768 = (17.19217677901328 - 20.987198830289383) / 17.19217677901328 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "-2147483648")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.809230475702368 > 19.450465085589432.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -57.26394540471593 (T) = (0 -22.873161777364732) / Math.Sqrt((0.06036155577472621 / (17)) + (0.012419191270756117 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.22352216359051835 = (18.694521814171072 - 22.873161777364732) / 18.694521814171072 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "4")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.325757278816269 > 5.626045020689939.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -36.9751975545917 (T) = (0 -10.075861798678275) / Math.Sqrt((0.047279260179264954 / (16)) + (0.09685036921117075 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.9066272439460731 = (5.284652168204967 - 10.075861798678275) / 5.284652168204967 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryFormat(value: 2147483647)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.227705974513873 > 7.831107469074557.
IsChangePoint: Marked as a change because one of 4/17/2023 5:51:51 AM, 5/15/2023 9:18:21 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.581059829005737 (T) = (0 -10.000953227251845) / Math.Sqrt((0.08622513828333536 / (16)) + (0.16188484604093958 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.3570084292296187 = (7.369853430409008 - 10.000953227251845) / 7.369853430409008 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.81432128086062 > 11.363579924945343.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -51.69633704360612 (T) = (0 -14.644635086866813) / Math.Sqrt((0.06473855206697286 / (16)) + (0.011584153787706749 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.3633960935216906 = (10.741291658713285 - 14.644635086866813) / 10.741291658713285 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryFormat(value: -2147483648)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.76381104329591 > 17.662200287651867.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -34.993902485541405 (T) = (0 -21.884992962767686) / Math.Sqrt((0.15462052482697136 / (15)) + (0.07576735895646834 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.30283412382298613 = (16.797988755889513 - 21.884992962767686) / 16.797988755889513 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParse(value: "2147483647")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.246654710285863 > 18.174991345582736.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -26.856184540432853 (T) = (0 -21.096135198709877) / Math.Sqrt((0.05815855824539443 / (17)) + (0.12507300519219203 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.2281155202225829 = (17.177647258204527 - 21.096135198709877) / 17.177647258204527 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int32.TryParseSpan(value: "-2147483648")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.77337799452872 > 19.9251875266209.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.803483355876637 (T) = (0 -22.423492228197965) / Math.Sqrt((0.05009146683406581 / (16)) + (0.20201219026919412 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.18758979353173538 = (18.881513086697602 - 22.423492228197965) / 18.881513086697602 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.08 ns 8.33 ns 1.64 0.33 False
30.78 ns 33.14 ns 1.08 0.02 False
11.47 ns 15.05 ns 1.31 0.04 False
5.85 ns 9.97 ns 1.70 0.33 False
11.08 ns 13.57 ns 1.22 0.10 False
23.94 ns 26.73 ns 1.12 0.04 False
5.20 ns 7.33 ns 1.41 0.40 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt64*'

Payloads

Baseline
Compare

System.Tests.Perf_UInt64.TryParseHex(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.33247808848713 > 5.410589282037846.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -69.56215941120203 (T) = (0 -8.373439354057883) / Math.Sqrt((0.03192582009073492 / (16)) + (0.0010847720434131998 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.6266287046024422 = (5.147726294492265 - 8.373439354057883) / 5.147726294492265 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParseHex(value: "FFFFFFFFFFFFFFFF")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 33.13782123157244 > 32.13656947665084.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.122928069780823 (T) = (0 -33.02610171712204) / Math.Sqrt((0.10073741626266393 / (16)) + (0.12586785055425628 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.08232744986679953 = (30.513964809066344 - 33.02610171712204) / 30.513964809066344 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.053299384653377 > 11.679391636990186.
IsChangePoint: Marked as a change because one of 5/2/2023 12:51:17 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -66.31464098845599 (T) = (0 -15.033398985567642) / Math.Sqrt((0.061358693329334954 / (17)) + (0.0005768161425592317 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.36615803541430214 = (11.004143441581121 - 15.033398985567642) / 11.004143441581121 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.97273902113208 > 5.57055885950666.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -33.422181726640765 (T) = (0 -10.034051247168458) / Math.Sqrt((0.06289851476298791 / (16)) + (0.11718191343550478 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.919013038672951 = (5.228756160045308 - 10.034051247168458) / 5.228756160045308 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryParseHex(value: "3039")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.57278615712766 > 12.125642603738799.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -21.964506624129086 (T) = (0 -14.232269645476995) / Math.Sqrt((0.04387607496074064 / (16)) + (0.08850981815420766 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.23675630018353971 = (11.507739757108872 - 14.232269645476995) / 11.507739757108872 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryFormat(value: 18446744073709551615)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.728718399732816 > 25.12921659587541.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 5/15/2023 9:18:21 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.652555458645992 (T) = (0 -26.777235184097343) / Math.Sqrt((0.23563835208951064 / (17)) + (0.00774755541073834 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.1154463489875449 = (24.005847711458454 - 26.777235184097343) / 24.005847711458454 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt64.TryFormat(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.3277312504990055 > 5.636566484278424.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 5/15/2023 9:18:21 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.48753676854301 (T) = (0 -7.2041708029441) / Math.Sqrt((0.08083814035631834 / (17)) + (0.1539589183242347 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.312495396827356 = (5.488911290933638 - 7.2041708029441) / 5.488911290933638 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.Primitives.StringSegmentBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.09 ns 7.15 ns 1.75 0.41 False
0.34 ns 1.72 ns 5.13 0.73 False
3.49 ns 4.87 ns 1.39 0.29 False
5.29 ns 8.80 ns 1.66 0.42 False
0.00 ns 1.78 ns ∞ 0.75 False
13.21 ns 15.42 ns 1.17 0.03 False
0.49 ns 2.27 ns 4.61 0.70 False
11.05 ns 13.82 ns 1.25 0.13 False
3.84 ns 5.99 ns 1.56 0.37 False
5.52 ns 7.47 ns 1.35 0.05 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.StringSegmentBenchmark*'

Payloads

Baseline
Compare

Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Valid

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.146105616857297 > 4.2136496932475245.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -41.139661583239636 (T) = (0 -7.133657827690746) / Math.Sqrt((0.0852491745203756 / (16)) + (0.003835524205228003 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.7923346082798604 = (3.980092665028134 - 7.133657827690746) / 3.980092665028134 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.StartsWith

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7210550059718486 > 0.31671682169168763.
IsChangePoint: Marked as a change because one of 3/24/2023 7:13:05 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.890136260987143 (T) = (0 -1.702749710645355) / Math.Sqrt((0.152436369312884 / (16)) + (0.0022305981800565373 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -4.247660223899119 = (0.3244778888104492 - 1.702749710645355) / 0.3244778888104492 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.LastIndexOf

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.866085978197592 > 3.6232700163029836.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 5/11/2023 10:01:37 AM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -39.15920071999772 (T) = (0 -4.849771806294892) / Math.Sqrt((0.015373486227819875 / (16)) + (0.0030935857543908213 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.43352356309864853 = (3.3831127238758563 - 4.849771806294892) / 3.3831127238758563 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_Object_Valid

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.801784531337153 > 5.450445161651418.
IsChangePoint: Marked as a change because one of 7/17/2023 4:07:27 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.196964610250546 (T) = (0 -8.363619030503289) / Math.Sqrt((0.02552949614736657 / (16)) + (0.35378374378203414 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.6328884338731395 = (5.12197824236231 - 8.363619030503289) / 5.12197824236231 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.Equals_String

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.7770335209068568 > 0.3510643763003663.
IsChangePoint: Marked as a change because one of 5/10/2023 2:36:50 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -25.25244878893778 (T) = (0 -1.8104969457624869) / Math.Sqrt((0.04919451480233212 / (16)) + (0.0030035534902663878 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -4.734052532344544 = (0.3157447434514887 - 1.8104969457624869) / 0.3157447434514887 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.GetSegmentHashCode

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.423240851408805 > 13.8444192878233.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 7/5/2023 5:05:14 PM, 7/17/2023 4:07:27 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -96.72376114015546 (T) = (0 -15.423610218731069) / Math.Sqrt((0.008254245804607143 / (15)) + (3.1162531369298583E-06 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.17256500742622352 = (13.15373571703785 - 15.423610218731069) / 13.15373571703785 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.EndsWith

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.271684517865494 > 0.
IsChangePoint: Marked as a change because one of 3/29/2023 12:02:44 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -31.30899280092221 (T) = (0 -2.2513027138832706) / Math.Sqrt((0.06554480798118431 / (17)) + (0.0005125368979987073 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -6.793978995670098 = (0.2888515243797769 - 2.2513027138832706) / 0.2888515243797769 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.Indexer

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.822458503602078 > 11.533834227818794.
IsChangePoint: Marked as a change because one of 5/10/2023 6:05:58 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.715710003105578 (T) = (0 -13.686925849468835) / Math.Sqrt((0.05670043183176693 / (17)) + (0.1063278883940577 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.2594503624590613 = (10.867380134573372 - 13.686925849468835) / 10.867380134573372 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOf

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.990075104843287 > 3.9022129394866236.
IsChangePoint: Marked as a change because one of 5/18/2023 7:10:13 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -26.70327555616013 (T) = (0 -5.87772224144826) / Math.Sqrt((0.02281888985045834 / (16)) + (0.03296340107890588 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.552441757660484 = (3.7861144950815646 - 5.87772224144826) / 3.7861144950815646 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.StringSegmentBenchmark.IndexOfAny

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.466952453396478 > 5.785440488929066.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.44304945172272 (T) = (0 -7.299824129874795) / Math.Sqrt((0.0021154760966346064 / (16)) + (0.10742345030109662 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.32653385463351375 = (5.502930893453558 - 7.299824129874795) / 5.502930893453558 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Char>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.06 ns 11.84 ns 1.95 0.09 False
4.24 ns 6.50 ns 1.53 0.29 False
4.26 ns 6.72 ns 1.58 0.19 False
3.78 ns 5.92 ns 1.57 0.32 False
6.83 ns 10.35 ns 1.52 0.08 False
1.92 ns 3.64 ns 1.90 0.44 False
4.50 ns 6.71 ns 1.49 0.19 False
0.81 ns 2.44 ns 3.02 0.59 False
3.08 ns 4.19 ns 1.36 0.41 False
12.50 ns 14.72 ns 1.18 0.05 False
4.82 ns 7.07 ns 1.47 0.28 False
25.40 ns 27.62 ns 1.09 0.02 False
1.59 ns 3.84 ns 2.42 0.69 False
3.56 ns 5.71 ns 1.60 0.43 False
2.70 ns 5.56 ns 2.06 0.64 False
4.01 ns 6.15 ns 1.53 0.39 False
1.53 ns 4.60 ns 3.00 0.52 False
0.94 ns 3.96 ns 4.19 0.69 False
4.05 ns 8.66 ns 2.14 0.60 False
6.65 ns 7.97 ns 1.20 0.08 False
1.18 ns 2.89 ns 2.44 0.52 False
4.71 ns 6.94 ns 1.47 0.20 False
4.28 ns 7.62 ns 1.78 0.48 False
2.88 ns 4.82 ns 1.67 0.34 False
2.69 ns 4.92 ns 1.83 0.41 False
28.22 ns 32.46 ns 1.15 0.06 False
8.45 ns 10.91 ns 1.29 0.07 False
0.93 ns 2.34 ns 2.52 0.58 False
5.72 ns 7.91 ns 1.38 0.09 False
1.19 ns 3.24 ns 2.72 0.57 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Char&gt;*'

Payloads

Baseline
Compare

System.Memory.Span<Char>.IndexOfAnyFiveValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.842130701015188 > 9.973816250452517.
IsChangePoint: Marked as a change because one of 4/28/2023 1:33:49 PM, 5/3/2023 4:46:09 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.595649279037447 (T) = (0 -11.795461668846821) / Math.Sqrt((0.7445652114627844 / (16)) + (0.0031510722208450952 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.27079247443367155 = (9.281973182996277 - 11.795461668846821) / 9.281973182996277 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceCompareToDifferent(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.500432853535271 > 4.458770869229907.
IsChangePoint: Marked as a change because one of 7/28/2023 12:13:51 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.474477211815012 (T) = (0 -6.488293507298378) / Math.Sqrt((0.6215539207006971 / (19)) + (0.0015926475878714843 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.6049300953070923 = (4.042726550066274 - 6.488293507298378) / 4.042726550066274 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyFiveValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.720899534483501 > 4.666349718828953.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/28/2023 1:33:49 PM, 5/3/2023 7:05:50 AM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.969032224764703 (T) = (0 -6.682934883960715) / Math.Sqrt((0.009108249644983087 / (16)) + (0.07695773855821884 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.5092161176503996 = (4.428083430731538 - 6.682934883960715) / 4.428083430731538 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyFourValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.921565650695271 > 3.9661979459776355.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/28/2023 1:33:49 PM, 5/3/2023 7:05:50 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.401426430908817 (T) = (0 -5.85579451357157) / Math.Sqrt((0.014790613490389216 / (16)) + (0.1190673961450553 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.5437343471939963 = (3.7932656769705795 - 5.85579451357157) / 3.7932656769705795 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceCompareTo(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.352364811161733 > 7.2007304544480455.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 7/5/2023 5:05:14 PM, 7/17/2023 4:07:27 PM, 7/29/2023 6:24:06 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.925051053935187 (T) = (0 -9.253163419135237) / Math.Sqrt((0.010835563900740034 / (18)) + (0.2741198670745841 / (8))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (8) - 2, .025) and -0.3528594480739217 = (6.839707873799492 - 9.253163419135237) / 6.839707873799492 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.644656071988278 > 1.9396832453748367.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -51.215339812938744 (T) = (0 -3.619727978838409) / Math.Sqrt((0.0159356165292093 / (16)) + (0.002251398257747802 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -1.055879163864699 = (1.7606715620552054 - 3.619727978838409) / 1.7606715620552054 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.LastIndexOfValue(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.705212253563173 > 4.801562335256202.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -34.91136613769393 (T) = (0 -6.7475681053225545) / Math.Sqrt((0.06460575714524061 / (19)) + (0.003032029349859305 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.47132916764565136 = (4.586035710907357 - 6.7475681053225545) / 4.586035710907357 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfValue(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.4391552087103454 > 0.8280260027281358.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -48.57029024320473 (T) = (0 -2.4951082783572756) / Math.Sqrt((0.014512446620947455 / (17)) + (0.002622094661150301 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -2.146857666363049 = (0.7928888252645291 - 2.4951082783572756) / 0.7928888252645291 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.BinarySearch(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.188750951950313 > 3.2081132863016846.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.144945003778577 (T) = (0 -4.230368502259961) / Math.Sqrt((0.49572068839683125 / (19)) + (0.005543822064529279 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.46130157258809174 = (2.8949318755386075 - 4.230368502259961) / 2.8949318755386075 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.BinarySearch(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.720191454535426 > 13.20302923421091.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 5/3/2023 7:05:50 AM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.364773252653045 (T) = (0 -14.577505470592653) / Math.Sqrt((0.009053686295399453 / (19)) + (0.08307646689919633 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.15253848854935095 = (12.648172373783988 - 14.577505470592653) / 12.648172373783988 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceCompareToDifferent(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.074570723705417 > 5.208360773092186.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 7/17/2023 4:07:27 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -46.09995473172993 (T) = (0 -7.054095274418403) / Math.Sqrt((0.018268763032552048 / (19)) + (0.00722626068492117 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.4120555440072837 = (4.9956216696685525 - 7.054095274418403) / 4.9956216696685525 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfValue(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 27.61633536327819 > 26.669402397254608.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 5/3/2023 7:05:50 AM, 7/17/2023 4:07:27 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.063750824931638 (T) = (0 -27.3527481430666) / Math.Sqrt((0.048388725561644245 / (17)) + (0.10353605678732108 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.07891017662923548 = (25.352201448801697 - 27.3527481430666) / 25.352201448801697 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.StartsWith(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.84002279292419 > 1.548159672847552.
IsChangePoint: Marked as a change because one of 4/26/2023 2:24:59 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.122609257975295 (T) = (0 -3.7889829134705897) / Math.Sqrt((0.07778031467904577 / (19)) + (0.18068342533040907 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.8134771111747283 = (1.3467260488529627 - 3.7889829134705897) / 1.3467260488529627 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyTwoValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.713127130253822 > 3.7439117796068255.
IsChangePoint: Marked as a change because one of 4/28/2023 1:33:49 PM, 5/3/2023 7:05:50 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -54.70109583880756 (T) = (0 -5.690954006821407) / Math.Sqrt((0.0208418936121787 / (16)) + (0.0018187198551941881 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.6127402768197542 = (3.528747987892814 - 5.690954006821407) / 3.528747987892814 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.StartsWith(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.556162400165844 > 2.95040458608276.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -23.363667410969576 (T) = (0 -5.418728055573419) / Math.Sqrt((0.08695161287057672 / (18)) + (0.056199619355845574 / (7))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (7) - 2, .025) and -0.9566933950209627 = (2.769329149555067 - 5.418728055573419) / 2.769329149555067 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceCompareToDifferent(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.154132046685862 > 4.414010940999795.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -52.22301892131416 (T) = (0 -6.192894422263935) / Math.Sqrt((0.017868867918168413 / (19)) + (0.003466212238296376 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.46953980215115526 = (4.214172636357719 - 6.192894422263935) / 4.214172636357719 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.EndsWith(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.602074991253647 > 1.8441133953071311.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.262092410433144 (T) = (0 -4.476430010097918) / Math.Sqrt((0.039279306031926337 / (17)) + (0.1098487920902496 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.3660814432908193 = (1.8919171285464969 - 4.476430010097918) / 1.8919171285464969 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceEqual(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.9560128547406395 > 1.407977730412624.
IsChangePoint: Marked as a change because one of 5/3/2023 4:46:09 PM, 7/6/2023 3:11:59 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.65451558519821 (T) = (0 -3.894837149480061) / Math.Sqrt((0.15840596419450975 / (16)) + (0.058242195774851235 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -1.8289556972240397 = (1.376775590123923 - 3.894837149480061) / 1.376775590123923 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceEqual(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.65650565089822 > 4.730588131223482.
IsChangePoint: Marked as a change because one of 5/1/2023 10:26:58 PM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -24.648297878100458 (T) = (0 -8.202903996506132) / Math.Sqrt((0.11955396580021842 / (19)) + (0.10109023580628758 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.7626478937264471 = (4.6537394256116675 - 8.202903996506132) / 4.6537394256116675 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.BinarySearch(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.966478730873523 > 7.031111861770632.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 5/3/2023 7:05:50 AM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.719244829108353 (T) = (0 -8.654287575978815) / Math.Sqrt((0.028880668446397417 / (19)) + (0.09295056971023437 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.28362971190910363 = (6.742043671696844 - 8.654287575978815) / 6.742043671696844 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.LastIndexOfAnyValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.893417117781211 > 1.2512660447965525.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.597198588870913 (T) = (0 -2.777984589245869) / Math.Sqrt((0.007017728368545039 / (19)) + (0.048875034808677 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.347372997537733 = (1.1834440424081831 - 2.777984589245869) / 1.1834440424081831 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyThreeValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.938250569928789 > 4.937268103292019.
IsChangePoint: Marked as a change because one of 4/8/2023 1:46:22 AM, 4/8/2023 9:16:19 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.640685813154812 (T) = (0 -6.925455350276744) / Math.Sqrt((0.9139953343191632 / (16)) + (0.0013584584165036923 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.4259416103129244 = (4.856759421416243 - 6.925455350276744) / 4.856759421416243 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.EndsWith(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.62216773606253 > 4.991881167026544.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -7.686417603090983 (T) = (0 -7.546687079270453) / Math.Sqrt((1.4524092179477468 / (17)) + (0.06861682252969803 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.45840825058275764 = (5.174605311136242 - 7.546687079270453) / 5.174605311136242 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.SequenceCompareTo(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.823930398343468 > 3.0228890042220598.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 7/5/2023 5:05:14 PM, 7/17/2023 4:07:27 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.059936270217234 (T) = (0 -4.633225905722431) / Math.Sqrt((0.13702420130696297 / (19)) + (0.16241458405694634 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.6094208665133153 = (2.8788156051188483 - 4.633225905722431) / 2.8788156051188483 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfValue(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.916644894554504 > 2.8280612367213345.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 7/5/2023 5:05:14 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -93.10273864121613 (T) = (0 -4.865607176622967) / Math.Sqrt((0.003660389329408269 / (17)) + (0.002362866598862748 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.8179221416221921 = (2.6764662056875683 - 4.865607176622967) / 2.6764662056875683 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.EndsWith(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 32.463706829842614 > 29.91726723863616.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/25/2023 6:46:56 PM, 6/22/2023 12:06:43 AM, 6/26/2023 6:52:13 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.262741429215493 (T) = (0 -31.98471486457573) / Math.Sqrt((0.05576453417720152 / (17)) + (0.4046373216997709 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.12386355154476582 = (28.459606880757278 - 31.98471486457573) / 28.459606880757278 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyFourValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.914629142823715 > 8.995460150981977.
IsChangePoint: Marked as a change because one of 4/28/2023 1:33:49 PM, 5/3/2023 7:05:50 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.72543974293046 (T) = (0 -10.672423937587437) / Math.Sqrt((0.026218047181898883 / (17)) + (0.15840138233208603 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.227562512294075 = (8.69399629811337 - 10.672423937587437) / 8.69399629811337 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.LastIndexOfValue(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.3435717096276667 > 0.9213803248509225.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/27/2023 5:16:33 PM, 4/28/2023 1:33:49 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.499113555160227 (T) = (0 -2.2925051308111892) / Math.Sqrt((0.018767008074102018 / (16)) + (0.05398952453601333 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -1.7570896467494495 = (0.8314945919564148 - 2.2925051308111892) / 0.8314945919564148 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.LastIndexOfAnyValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.910490866392485 > 6.002560023987723.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -91.92708754213528 (T) = (0 -7.895883531907648) / Math.Sqrt((0.006648583136399174 / (19)) + (0.0016142354168486054 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.38988203597358734 = (5.68097387227307 - 7.895883531907648) / 5.68097387227307 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Char>.IndexOfAnyTwoValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.2377733786915703 > 1.483908312411072.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -23.276526370174285 (T) = (0 -3.2389979280541428) / Math.Sqrt((0.12049966226558172 / (17)) + (3.118873385573243E-05 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.533048648161502 = (1.2786955080412774 - 3.2389979280541428) / 1.2786955080412774 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.90 ns 13.18 ns 1.21 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int64&gt;*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_VectorOf<Int64>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.178618281242887 > 11.367267932131526.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.228155705044163 (T) = (0 -13.191128578744957) / Math.Sqrt((0.010720703073682664 / (16)) + (0.07763151011898499 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.22359848897733023 = (10.780602213533259 - 13.191128578744957) / 10.780602213533259 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.58 ns 11.97 ns 1.25 0.08 False
1.03 ns 2.20 ns 2.13 0.61 False
0.57 ns 2.00 ns 3.48 0.59 False
1.68 ns 4.18 ns 2.48 0.65 False
2.11 ns 3.57 ns 1.69 0.55 False
5.93 ns 9.89 ns 1.67 0.46 False
1.12 ns 3.35 ns 2.99 0.64 False
2.21 ns 4.24 ns 1.92 0.61 False
8.46 ns 12.04 ns 1.42 0.45 False
5.67 ns 8.29 ns 1.46 0.12 False
4.13 ns 5.74 ns 1.39 0.32 False
7.10 ns 8.20 ns 1.15 0.36 False
7.97 ns 9.96 ns 1.25 0.25 False
5.59 ns 7.69 ns 1.37 0.13 False
1.05 ns 2.15 ns 2.04 0.60 False
2.21 ns 3.50 ns 1.59 0.54 False
1.12 ns 2.58 ns 2.30 0.54 False
11.25 ns 13.66 ns 1.21 0.13 False
2.85 ns 6.73 ns 2.36 0.52 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Memory.Span<Int32>.IndexOfAnyFiveValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.966868373966863 > 10.014498260650656.
IsChangePoint: Marked as a change because one of 4/28/2023 1:33:49 PM, 5/3/2023 7:05:50 AM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -56.2593756391572 (T) = (0 -11.90116056640871) / Math.Sqrt((0.019481780959675607 / (16)) + (0.003568261972323372 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.2445094166667562 = (9.562933318965394 - 11.90116056640871) / 9.562933318965394 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.LastIndexOfAnyValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1954199543828072 > 1.1019177159566864.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/22/2023 11:33:11 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.827050217878849 (T) = (0 -2.1457380417579004) / Math.Sqrt((0.07933831564085288 / (16)) + (0.03585522261724333 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -1.2391944896973803 = (0.9582633628434347 - 2.1457380417579004) / 0.9582633628434347 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyTwoValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0018950524230226 > 0.8685038306648426.
IsChangePoint: Marked as a change because one of 3/20/2023 8:31:51 AM, 3/21/2023 3:33:39 AM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -9.97843760146294 (T) = (0 -2.359844533752022) / Math.Sqrt((0.2215020835669603 / (17)) + (0.05983073878399343 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.6393280570974116 = (0.8941080770183794 - 2.359844533752022) / 0.8941080770183794 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.StartsWith(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.1768619359113295 > 1.4652142920287246.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.68811514502241 (T) = (0 -4.334785936357216) / Math.Sqrt((0.3451631762818653 / (19)) + (0.05662095975827794 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.9514433452720803 = (1.4687003710578126 - 4.334785936357216) / 1.4687003710578126 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceCompareToDifferent(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5670758525096673 > 2.3497110580823843.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -49.29504532011284 (T) = (0 -3.53859934377945) / Math.Sqrt((0.0077075002388398055 / (19)) + (0.002122127128667747 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5895350676459101 = (2.2261851379095963 - 3.53859934377945) / 2.2261851379095963 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.EndsWith(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.892845491299793 > 5.987619108136513.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.13799593211403 (T) = (0 -9.527127331043065) / Math.Sqrt((0.79563818004913 / (19)) + (0.3132513178569327 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5999504545651563 = (5.954638972637689 - 9.527127331043065) / 5.954638972637689 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.LastIndexOfValue(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3534490319568357 > 1.2847661867380868.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.184825689779037 (T) = (0 -3.1137322972328376) / Math.Sqrt((0.01832604308564544 / (16)) + (0.13382819207016045 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -1.5174399751278078 = (1.2368645640000837 - 3.1137322972328376) / 1.2368645640000837 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.EndsWith(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.239309093651265 > 2.3092170313739206.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.896595816660392 (T) = (0 -4.628308651363693) / Math.Sqrt((0.48914061718166796 / (19)) + (0.05547339484076228 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.0462065933205278 = (2.2618970471857396 - 4.628308651363693) / 2.2618970471857396 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceEqual(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.035285425287208 > 8.841545763957436.
IsChangePoint: Marked as a change because one of 4/6/2023 7:14:22 PM, 4/12/2023 8:39:35 PM, 5/18/2023 11:09:10 AM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.459424160464597 (T) = (0 -11.028934166686938) / Math.Sqrt((0.37877971331173976 / (17)) + (0.47770656495359026 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.3992612936591354 = (7.881969019414341 - 11.028934166686938) / 7.881969019414341 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceCompareTo(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.291028357638432 > 6.135634308011395.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 7/5/2023 5:05:14 PM, 7/17/2023 4:07:27 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.08334727108446 (T) = (0 -8.150489938000794) / Math.Sqrt((0.01259521688513354 / (16)) + (0.09033058307996915 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.4051314564653119 = (5.80051773839283 - 8.150489938000794) / 5.80051773839283 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfValue(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.736379740791244 > 4.33958010957269.
IsChangePoint: Marked as a change because one of 4/28/2023 10:02:43 PM, 7/18/2023 9:59:03 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.064581765295426 (T) = (0 -6.053989032392431) / Math.Sqrt((0.0322032525524162 / (18)) + (0.05797118203710422 / (7))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (7) - 2, .025) and -0.4620313676520189 = (4.140806528737455 - 6.053989032392431) / 4.140806528737455 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.BinarySearch(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.197145775805524 > 6.1288026563940035.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.119381623354164 (T) = (0 -8.267471128380256) / Math.Sqrt((0.2918825744548887 / (16)) + (0.049182209465000126 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.3726418135654368 = (6.02303605112065 - 8.267471128380256) / 6.02303605112065 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyFourValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.957273358372396 > 8.597124090728732.
IsChangePoint: Marked as a change because one of 4/28/2023 1:33:49 PM, 5/3/2023 7:05:50 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -25.71892966673469 (T) = (0 -10.159507139123942) / Math.Sqrt((0.08091294859824656 / (17)) + (0.01302435314718624 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.25940733861461623 = (8.066895298783702 - 10.159507139123942) / 8.066895298783702 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.LastIndexOfValue(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.688492473915527 > 5.905859287721766.
IsChangePoint: Marked as a change because one of 7/28/2023 12:13:51 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.571263197973556 (T) = (0 -7.531766238008984) / Math.Sqrt((0.739079706110298 / (17)) + (0.11062671782260725 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.3832711080822272 = (5.444895215407959 - 7.531766238008984) / 5.444895215407959 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfAnyThreeValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1452669642038464 > 0.9962536570818211.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -5.329230538286128 (T) = (0 -2.6043726793717283) / Math.Sqrt((0.9665074488891647 / (17)) + (0.08435139739580173 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.1604503128834673 = (1.205476776689104 - 2.6043726793717283) / 1.205476776689104 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.SequenceCompareToDifferent(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.503466073728391 > 2.361868508037061.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/18/2023 11:09:10 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -36.96251356921574 (T) = (0 -3.5857870153005122) / Math.Sqrt((0.007836676709992663 / (19)) + (0.006290672480251621 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5955305430140294 = (2.2473947810029373 - 3.5857870153005122) / 2.2473947810029373 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.IndexOfValue(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.5780378070231205 > 1.2277490881523598.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 7/4/2023 9:23:46 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.09303456761019 (T) = (0 -3.0934810182191472) / Math.Sqrt((0.29199650576897324 / (19)) + (0.053641707729087086 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.4585925161222544 = (1.2582325041395037 - 3.0934810182191472) / 1.2582325041395037 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.BinarySearch(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.659241529218784 > 11.812511070143385.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/5/2023 3:18:41 PM, 5/11/2023 10:01:37 AM, 7/5/2023 5:05:14 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -27.84722785325773 (T) = (0 -13.638555839714437) / Math.Sqrt((0.12049925595890779 / (19)) + (0.0015259414542389018 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.19814154252343147 = (11.38309236068218 - 13.638555839714437) / 11.38309236068218 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Int32>.StartsWith(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.734812515214545 > 3.4387642294799696.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/13/2023 7:10:54 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -23.171960307249844 (T) = (0 -6.72634470993952) / Math.Sqrt((0.1791150255016811 / (19)) + (0.0869394557275716 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.0374482068450281 = (3.3013573976220036 - 6.72634470993952) / 3.3013573976220036 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
11.61 ns 13.47 ns 1.16 0.04 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

System.Tests.Perf_String.IndexOfAny

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.472607238335204 > 12.191492304511604.
IsChangePoint: Marked as a change because one of 6/3/2023 4:12:47 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -58.82897178525698 (T) = (0 -13.453037011835063) / Math.Sqrt((0.01601436267770822 / (17)) + (0.0006909655340339632 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.1642416215251633 = (11.55519332336831 - 13.453037011835063) / 11.55519332336831 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.Span<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.96 ns 6.19 ns 1.56 0.21 False
0.76 ns 1.99 ns 2.62 0.65 False
3.30 ns 6.11 ns 1.85 0.62 False
4.86 ns 7.31 ns 1.50 0.22 False
12.15 ns 14.14 ns 1.16 0.05 False
1.47 ns 4.15 ns 2.81 0.62 False
3.99 ns 6.27 ns 1.57 0.18 False
1.50 ns 3.42 ns 2.28 0.48 False
4.73 ns 6.93 ns 1.47 0.19 False
20.21 ns 22.58 ns 1.12 0.07 False
26.15 ns 28.17 ns 1.08 0.05 False
4.97 ns 7.28 ns 1.47 0.41 False
4.09 ns 6.64 ns 1.62 0.48 False
3.23 ns 4.41 ns 1.36 0.53 False
7.03 ns 9.42 ns 1.34 0.12 False
15.65 ns 19.48 ns 1.24 0.08 False
6.59 ns 8.56 ns 1.30 0.04 False
2.41 ns 4.54 ns 1.88 0.49 False
1.87 ns 3.83 ns 2.05 0.42 False
0.87 ns 2.29 ns 2.63 0.60 False
4.89 ns 6.26 ns 1.28 0.08 False
20.21 ns 22.29 ns 1.10 0.06 False
0.80 ns 3.38 ns 4.24 0.80 False
4.05 ns 5.37 ns 1.33 0.35 False
0.99 ns 2.72 ns 2.74 0.54 False
3.08 ns 4.43 ns 1.44 0.53 False
9.50 ns 11.44 ns 1.20 0.09 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.Span&lt;Byte&gt;*'

Payloads

Baseline
Compare

System.Memory.Span<Byte>.LastIndexOfValue(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.187015161496405 > 4.151894439737054.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -91.95599474919011 (T) = (0 -6.191560737115268) / Math.Sqrt((0.011884734176419777 / (19)) + (3.2635970104984396E-05 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5944629467302789 = (3.883163763580791 - 6.191560737115268) / 3.883163763580791 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfValue(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9881484850233335 > 1.0372391698639845.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 4/26/2023 2:24:59 PM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.188416203056882 (T) = (0 -2.4235308937314994) / Math.Sqrt((0.018482205576931903 / (16)) + (0.09450106982399235 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -1.5564322563347133 = (0.9480129534925517 - 2.4235308937314994) / 0.9480129534925517 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.SequenceEqual(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.1055269970119275 > 3.4505453902589007.
IsChangePoint: Marked as a change because one of 4/28/2023 1:33:49 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.955881252461708 (T) = (0 -6.257624545590291) / Math.Sqrt((0.043879133799954055 / (19)) + (0.15486690240451212 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.8133235002740902 = (3.450914602190083 - 6.257624545590291) / 3.450914602190083 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.311650466604767 > 5.209805193232945.
IsChangePoint: Marked as a change because one of 4/28/2023 1:33:49 PM, 7/1/2023 8:07:46 AM, 7/17/2023 4:07:27 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.140318180448872 (T) = (0 -7.168383861665388) / Math.Sqrt((0.05561728398893562 / (16)) + (0.04705674878582608 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.4532995945947443 = (4.932488723128219 - 7.168383861665388) / 4.932488723128219 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.BinarySearch(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.138262346103819 > 12.59288890653537.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 5/3/2023 7:05:50 AM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -76.02101855098444 (T) = (0 -14.209890932311733) / Math.Sqrt((0.010706125367838859 / (17)) + (0.0010105353515551814 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.17488210700665516 = (12.094737716719045 - 14.209890932311733) / 12.094737716719045 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.EndsWith(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.147180963441823 > 1.328882996489127.
IsChangePoint: Marked as a change because one of 4/23/2023 8:38:14 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.25917767014944 (T) = (0 -4.034413724638489) / Math.Sqrt((0.04178784641996408 / (17)) + (0.14501226171647433 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.8673070740638336 = (1.4070392952089763 - 4.034413724638489) / 1.4070392952089763 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.272737335808927 > 4.152136754055853.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -34.917677776097214 (T) = (0 -6.237003655706963) / Math.Sqrt((0.07362003419330544 / (17)) + (0.0022872976190835234 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.6182964118338552 = (3.8540551719070946 - 6.237003655706963) / 3.8540551719070946 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.421194408501824 > 1.5650889975368072.
IsChangePoint: Marked as a change because one of 5/18/2023 7:10:13 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.571404294379983 (T) = (0 -3.269235418288801) / Math.Sqrt((0.007025700021284147 / (16)) + (0.10220639581166042 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -1.2056179559735265 = (1.4822310497766191 - 3.269235418288801) / 1.4822310497766191 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyFiveValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.932642632686252 > 4.955840110987614.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/28/2023 1:33:49 PM, 5/3/2023 4:46:09 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -85.90010721749928 (T) = (0 -6.908408500127208) / Math.Sqrt((0.008599306908508651 / (19)) + (0.0016799050685552058 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.4863888163480915 = (4.647780193274379 - 6.908408500127208) / 4.647780193274379 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyTwoValues(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.57992466378432 > 21.22002907305235.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 7/1/2023 8:07:46 AM, 7/17/2023 4:07:27 PM, 7/29/2023 7:04:01 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.60509093380093 (T) = (0 -22.728784721708504) / Math.Sqrt((0.06151905867164064 / (17)) + (0.07515514075919431 / (6))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (6) - 2, .025) and -0.12308949538546934 = (20.237732447054434 - 22.728784721708504) / 20.237732447054434 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.168576845681994 > 27.44174240375824.
IsChangePoint: Marked as a change because one of 4/7/2023 2:04:00 PM, 4/17/2023 5:51:51 AM, 7/1/2023 8:07:46 AM, 7/17/2023 4:07:27 PM, 7/29/2023 7:04:01 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.870077013561877 (T) = (0 -28.07916311109767) / Math.Sqrt((0.7558735445327076 / (17)) + (0.09455950529292496 / (6))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (6) - 2, .025) and -0.08403662906522526 = (25.90241174351331 - 28.07916311109767) / 25.90241174351331 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.SequenceCompareTo(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.277422658468873 > 5.1939652929873885.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -75.33390295389043 (T) = (0 -7.264779972269623) / Math.Sqrt((0.013497155508504447 / (19)) + (0.0017164004788812921 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.471786625238953 = (4.936027986454996 - 7.264779972269623) / 4.936027986454996 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.EndsWith(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.64048946855143 > 4.3209732837136885.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/26/2023 12:55:36 AM, 7/28/2023 12:13:51 PM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -6.577139236438457 (T) = (0 -6.355971104191192) / Math.Sqrt((0.3523624730005867 / (13)) + (0.7397003004237397 / (10))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (10) - 2, .025) and -0.49028057096057326 = (4.26494931762708 - 6.355971104191192) / 4.26494931762708 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.SequenceCompareToDifferent(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.406785568325087 > 3.3827777558345247.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -5.651543695682814 (T) = (0 -4.236198199221586) / Math.Sqrt((0.6069633327727042 / (17)) + (0.0820621355487767 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.4095190128922724 = (3.0054211120778636 - 4.236198199221586) / 3.0054211120778636 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.423198800665425 > 7.353465115624603.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/3/2023 7:05:50 AM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.346696438990328 (T) = (0 -9.281760187833518) / Math.Sqrt((0.04710341044900057 / (19)) + (0.07745036943067021 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.3424811353735335 = (6.913885002377296 - 9.281760187833518) / 6.913885002377296 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.EndsWith(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.475626369251447 > 16.406949248350376.
IsChangePoint: Marked as a change because one of 5/3/2023 7:05:50 AM, 6/22/2023 12:06:43 AM, 6/26/2023 6:52:13 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -25.22026725832289 (T) = (0 -19.30973079663524) / Math.Sqrt((0.026580182574752895 / (16)) + (0.1421091123613701 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.24001742022378383 = (15.572144779345475 - 19.30973079663524) / 15.572144779345475 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.BinarySearch(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.555049855360325 > 6.7330018923673105.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 5/3/2023 7:05:50 AM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -57.46393474199531 (T) = (0 -8.55433335480778) / Math.Sqrt((0.01780735975074715 / (17)) + (0.0024890684853960112 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.3362238707376365 = (6.40187137959564 - 8.55433335480778) / 6.40187137959564 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfValue(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.544315580140912 > 2.5029050432340045.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/3/2023 1:18:01 AM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.004542544706865 (T) = (0 -4.476742413543063) / Math.Sqrt((0.007119148831246694 / (16)) + (0.0952406690294934 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.911048712419615 = (2.342557981096659 - 4.476742413543063) / 2.342557981096659 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyThreeValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.831874813577578 > 1.9624328456727036.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.95691464015381 (T) = (0 -3.6845749620449992) / Math.Sqrt((0.05208720687125761 / (16)) + (0.08230950168288165 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.9896371603905721 = (1.8518828635678002 - 3.6845749620449992) / 1.8518828635678002 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.LastIndexOfValue(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.2883879993730156 > 0.9017048853403683.
IsChangePoint: Marked as a change because one of 4/12/2023 2:59:13 PM, 4/29/2023 4:23:29 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.078949426468053 (T) = (0 -2.2191930533733433) / Math.Sqrt((0.1356117496815394 / (19)) + (0.07004337655859305 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.4670912865772807 = (0.8995180135600661 - 2.2191930533733433) / 0.8995180135600661 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.258268166990321 > 5.1393616202354275.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -4.563109748420824 (T) = (0 -6.859897297194002) / Math.Sqrt((1.900838481814489 / (18)) + (0.14850965696408516 / (7))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (7) - 2, .025) and -0.3104145970709617 = (5.234906046168321 - 6.859897297194002) / 5.234906046168321 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.29378566166474 > 21.217651465736985.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 7/1/2023 8:07:46 AM, 7/17/2023 4:07:27 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -10.972373641474965 (T) = (0 -22.092735299756097) / Math.Sqrt((0.013856383878675799 / (17)) + (0.2038571646372442 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.09401150295052411 = (20.1942440643197 - 22.092735299756097) / 20.1942440643197 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.StartsWith(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.3759936935933337 > 1.0615025022724018.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.820663770358214 (T) = (0 -3.531767573029415) / Math.Sqrt((0.07957704600067103 / (17)) + (0.13648096421110695 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -2.295314259040684 = (1.071754405013125 - 3.531767573029415) / 1.071754405013125 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyFourValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.373185531101125 > 4.165392394474293.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 4/28/2023 1:33:49 PM, 5/4/2023 2:44:24 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -9.806499456049766 (T) = (0 -5.851480029860282) / Math.Sqrt((0.014363369038651859 / (16)) + (0.2443778712547459 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.4644170480219472 = (3.995774316998108 - 5.851480029860282) / 3.995774316998108 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.LastIndexOfAnyValues(Size: 4)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.7221499002579437 > 1.1248625983180325.
IsChangePoint: Marked as a change because one of 5/27/2023 6:38:16 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -5.785402611284813 (T) = (0 -2.6981282672217843) / Math.Sqrt((0.9320153715469774 / (19)) + (0.05532470705370279 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.048064766377549 = (1.3174037811284722 - 2.6981282672217843) / 1.3174037811284722 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.SequenceCompareToDifferent(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.425776903435779 > 3.4579681518297223.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -5.705490648259747 (T) = (0 -4.431628642730719) / Math.Sqrt((0.8096801719543198 / (17)) + (0.005511986013458495 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.39525351966566635 = (3.1762174975861273 - 4.431628642730719) / 3.1762174975861273 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.Span<Byte>.IndexOfAnyFiveValues(Size: 33)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.43615266749576 > 9.315380774217791.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/28/2023 1:33:49 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -52.196098910030194 (T) = (0 -11.458970930985405) / Math.Sqrt((0.028704396213919545 / (19)) + (0.006454757145373265 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.2897789155119328 = (8.884445848176364 - 11.458970930985405) / 8.884445848176364 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.52 ns 11.59 ns 2.10 0.24 False
13.92 ns 18.32 ns 1.32 0.04 False
5.24 ns 11.77 ns 2.24 0.25 False
12.36 ns 17.47 ns 1.41 0.06 False
12.92 ns 17.43 ns 1.35 0.03 False
13.27 ns 18.06 ns 1.36 0.06 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int16*'

Payloads

Baseline
Compare

System.Tests.Perf_Int16.Parse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.592055439008904 > 5.748691948098936.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -37.01333865588657 (T) = (0 -11.343285700376908) / Math.Sqrt((0.05256939056802559 / (16)) + (0.15473682059527213 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -1.083097649043878 = (5.445393165118 - 11.343285700376908) / 5.445393165118 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.Parse(value: "-32768")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.315836208039755 > 14.614847238808172.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.654910732555432 (T) = (0 -18.085634691096935) / Math.Sqrt((0.0070854304238085935 / (16)) + (0.23662984285607913 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.3017688373738457 = (13.89312308902894 - 18.085634691096935) / 13.89312308902894 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.TryParse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.766793680443966 > 5.46620593453078.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -50.535537747492 (T) = (0 -11.529002796485416) / Math.Sqrt((0.034219211875473346 / (16)) + (0.09618030795640513 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -1.233880923517782 = (5.160974640640305 - 11.529002796485416) / 5.160974640640305 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.TryParse(value: "32767")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.469349381467804 > 13.07852299900314.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -57.003589687721565 (T) = (0 -17.390012285633308) / Math.Sqrt((0.07294003338937638 / (17)) + (0.023895552731291476 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.4039385916688419 = (12.386590402762593 - 17.390012285633308) / 12.386590402762593 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.Parse(value: "32767")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.429680218200925 > 13.495365362332539.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -75.07329137808392 (T) = (0 -17.438558655596506) / Math.Sqrt((0.0487028076679374 / (16)) + (0.00523289363505765 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.36069502247182267 = (12.81592007584317 - 17.438558655596506) / 12.81592007584317 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int16.TryParse(value: "-32768")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 18.06006280384652 > 13.947653987243354.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -28.88598745335942 (T) = (0 -17.966157937565182) / Math.Sqrt((0.045759735064728524 / (15)) + (0.1674045942196917 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.3587327428398208 = (13.222731278275516 - 17.966157937565182) / 13.222731278275516 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Perf_Ascii

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.59 ns 8.32 ns 1.81 0.36 False
3.26 ns 7.05 ns 2.16 0.45 False
7.21 ns 10.88 ns 1.51 0.37 False
3.59 ns 6.86 ns 1.91 0.47 False
7.67 ns 10.62 ns 1.38 0.44 False
3.66 ns 6.57 ns 1.79 0.53 False
6.74 ns 10.42 ns 1.55 0.48 False
5.03 ns 7.35 ns 1.46 0.43 False
3.71 ns 7.02 ns 1.89 0.58 False
4.99 ns 8.62 ns 1.73 0.51 False
7.58 ns 10.77 ns 1.42 0.35 False
19.94 ns 21.96 ns 1.10 0.08 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'

Payloads

Baseline
Compare

System.Text.Perf_Ascii.EqualsIgnoreCase_ExactlyTheSame_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.316813001572278 > 4.849451704520222.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -97.99734146503275 (T) = (0 -8.34373252582524) / Math.Sqrt((0.022480426593968938 / (16)) + (0.00027155356213377535 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.8059770600637649 = (4.620065620064212 - 8.34373252582524) / 4.620065620064212 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.Equals_Bytes_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.052250501323483 > 3.9196768211717194.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -37.77654655918529 (T) = (0 -6.96058776044101) / Math.Sqrt((0.07640413750236252 / (16)) + (0.020533222079593946 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.9102217370262002 = (3.643863759647679 - 6.96058776044101) / 3.643863759647679 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.EqualsIgnoreCase_DifferentCase_Bytes_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.87905933778028 > 7.814270599641315.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -59.1495454026214 (T) = (0 -10.957704235824243) / Math.Sqrt((0.05895238775640031 / (19)) + (0.0016529642993918763 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.4533042064168082 = (7.5398558591122455 - 10.957704235824243) / 7.5398558591122455 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.Equals_Bytes(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.857223031018391 > 3.749607874734527.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -63.74505587831804 (T) = (0 -6.872276457767613) / Math.Sqrt((0.04014580233970058 / (16)) + (0.0010001748240911908 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.9145156266752378 = (3.5895640453465822 - 6.872276457767613) / 3.5895640453465822 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.IsValid_Chars(Size: 128)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.618607730818216 > 8.06025362491325.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -53.00344931605472 (T) = (0 -10.589675063012887) / Math.Sqrt((0.0489411880302908 / (16)) + (0.001080366873668623 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.39609938900500824 = (7.5851870908417816 - 10.589675063012887) / 7.5851870908417816 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.IsValid_Bytes(Size: 128)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.5650015568654805 > 3.821698594867359.
IsChangePoint: Marked as a change because one of 7/1/2023 8:07:46 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -21.09147248101716 (T) = (0 -6.4065301036569755) / Math.Sqrt((0.047954471794021715 / (16)) + (0.10593055258562324 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.7962544528605378 = (3.5666049948850884 - 6.4065301036569755) / 3.5666049948850884 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.EqualsIgnoreCase_DifferentCase_Bytes(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.417377917051121 > 7.498633837652712.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.564561111127691 (T) = (0 -10.255752992551452) / Math.Sqrt((0.44201412073702867 / (19)) + (0.14113046097685028 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.42033084334754683 = (7.220678928847233 - 10.255752992551452) / 7.220678928847233 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.EqualsIgnoreCase_ExactlyTheSame_Bytes_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.351265315588484 > 4.857514770801359.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -21.23931098158487 (T) = (0 -8.249150319600055) / Math.Sqrt((0.1054163121298544 / (19)) + (0.15745417421436894 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.7579475003527211 = (4.692489575453712 - 8.249150319600055) / 4.692489575453712 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.Equals_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.020502647233946 > 3.644227873809582.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -43.61522471832997 (T) = (0 -6.940688759496875) / Math.Sqrt((0.044176328652589504 / (16)) + (0.02191163989568956 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.9317125678330572 = (3.5930235559231005 - 6.940688759496875) / 3.5930235559231005 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.EqualsIgnoreCase_ExactlyTheSame_Bytes(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.624114551487036 > 5.244736793081396.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -106.69433597019926 (T) = (0 -8.604687375119289) / Math.Sqrt((0.016216807974474137 / (16)) + (0.0009093264726681914 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.7220332677566298 = (4.996818317179785 - 8.604687375119289) / 4.996818317179785 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.EqualsIgnoreCase_DifferentCase_Chars(Size: 6)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.770756357382004 > 7.965364883432468.
IsChangePoint: Marked as a change because one of 6/22/2023 12:06:43 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -24.09366078423478 (T) = (0 -10.709681336853048) / Math.Sqrt((0.038691294280793445 / (19)) + (0.11082873174365648 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.4300632620605164 = (7.488956342687897 - 10.709681336853048) / 7.488956342687897 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.Equals_Chars(Size: 128)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.96352755679505 > 21.32947984332262.
IsChangePoint: Marked as a change because one of 5/11/2023 10:01:37 AM, 7/14/2023 7:13:27 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -27.69465763644427 (T) = (0 -22.009025123310803) / Math.Sqrt((0.04614910167249079 / (17)) + (0.010748130092342855 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.08936516519910212 = (20.203533054307126 - 22.009025123310803) / 20.203533054307126 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Vector3

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.05 ns 1.40 ns 26.50 0.86 False
4.21 ns 6.69 ns 1.59 0.17 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector3*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Vector3.EqualsBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.4028569700859812 > 0.20064151225965365.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -24.355532791778195 (T) = (0 -1.4437925244950822) / Math.Sqrt((0.03746069763269618 / (17)) + (0.0005994496743675775 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -4.1844301197623555 = (0.2784862542541634 - 1.4437925244950822) / 0.2784862542541634 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Vector3.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.686913485774433 > 4.4232527829197705.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -83.9091581606501 (T) = (0 -6.648518238503762) / Math.Sqrt((0.007936766733411834 / (16)) + (0.002803462379042159 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.6074430037326155 = (4.136083346697427 - 6.648518238503762) / 4.136083346697427 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt32

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.15 ns 14.66 ns 1.45 0.12 False
15.98 ns 19.21 ns 1.20 0.02 False
5.18 ns 8.47 ns 1.63 0.38 False
16.98 ns 20.77 ns 1.22 0.05 False
5.25 ns 10.27 ns 1.95 0.29 False
3.22 ns 5.75 ns 1.79 0.52 False
7.11 ns 10.14 ns 1.43 0.28 False
10.81 ns 14.05 ns 1.30 0.12 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt32*'

Payloads

Baseline
Compare

System.Tests.Perf_UInt32.TryParse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.664526661292898 > 11.027969190593476.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -34.06231634118844 (T) = (0 -14.689620783054567) / Math.Sqrt((0.2884554055544923 / (19)) + (0.0009046319475090326 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.40237488680591776 = (10.474817341111972 - 14.689620783054567) / 10.474817341111972 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParseHex(value: "FFFFFFFF")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.20750606000785 > 16.508809041402714.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -55.73119053861728 (T) = (0 -19.231595703992138) / Math.Sqrt((0.048786869717456445 / (17)) + (0.006027755107475999 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.21507761155072258 = (15.827462806633507 - 19.231595703992138) / 15.827462806633507 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParseHex(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.467990905679315 > 5.543100531650572.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -66.93489122751295 (T) = (0 -8.435077441368788) / Math.Sqrt((0.040342978175026106 / (19)) + (0.0014297717021418148 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.6203192166637143 = (5.205812135424071 - 8.435077441368788) / 5.205812135424071 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParse(value: "4294967295")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.765901967122655 > 17.70905836683909.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.699851405750344 (T) = (0 -20.36389936266486) / Math.Sqrt((0.056263883692133615 / (19)) + (0.2431605126022105 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.21698172064443486 = (16.733118515437894 - 20.36389936266486) / 16.733118515437894 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.269298419459854 > 5.472189897507747.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -79.46490244654014 (T) = (0 -10.156464845737869) / Math.Sqrt((0.06177048829935191 / (19)) + (0.005584367144676511 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.9914361517015138 = (5.100070538068734 - 10.156464845737869) / 5.100070538068734 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryFormat(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.746363283957177 > 3.390370695064834.
IsChangePoint: Marked as a change because one of 4/28/2023 1:33:49 PM, 5/3/2023 1:18:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.00753466124357 (T) = (0 -5.591670443488785) / Math.Sqrt((0.028807566026326673 / (16)) + (0.10537731251075146 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.7184766241407218 = (3.2538530725052777 - 5.591670443488785) / 3.2538530725052777 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryFormat(value: 4294967295)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.140476823451099 > 7.46560722520906.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/15/2023 9:18:21 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -21.036459487027013 (T) = (0 -9.955074919675372) / Math.Sqrt((0.03292589072486582 / (16)) + (0.11943593545417237 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.41281974094786683 = (7.046245625784128 - 9.955074919675372) / 7.046245625784128 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt32.TryParseHex(value: "3039")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.052014312094322 > 11.14988196804022.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.6506333026294 (T) = (0 -14.030086621522088) / Math.Sqrt((0.4148824101010883 / (18)) + (0.005975887294656635 / (7))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (7) - 2, .025) and -0.3326216691268031 = (10.528184365120872 - 14.030086621522088) / 10.528184365120872 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Globalization.Tests.StringSearch

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.04 ns 11.65 ns 1.45 0.33 False
7.48 ns 10.77 ns 1.44 0.39 False
7.27 ns 10.58 ns 1.46 0.42 False
6.57 ns 9.92 ns 1.51 0.51 False
8.08 ns 10.90 ns 1.35 0.35 False
17.39 ns 21.00 ns 1.21 0.04 False
7.31 ns 11.60 ns 1.59 0.32 False
7.75 ns 8.92 ns 1.15 0.51 False
17.05 ns 19.91 ns 1.17 0.09 False
7.62 ns 9.76 ns 1.28 0.44 False
8.04 ns 11.21 ns 1.39 0.37 False
7.70 ns 10.71 ns 1.39 0.44 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.StringSearch*'

Payloads

Baseline
Compare

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, None, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.652705661166145 > 8.479418656899108.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 5/19/2023 4:22:12 AM, 7/17/2023 4:07:27 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.710812723904667 (T) = (0 -11.711341577349057) / Math.Sqrt((0.07041963893306527 / (15)) + (0.18194807852064418 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.44883727190957173 = (8.083269118217448 - 11.711341577349057) / 8.083269118217448 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreCase, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.769460515771865 > 7.885762824755993.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -38.83434528020149 (T) = (0 -10.94348621060657) / Math.Sqrt((0.0008730558059269015 / (16)) + (0.054373715297173435 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.45741297792764657 = (7.508843667748553 - 10.94348621060657) / 7.508843667748553 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, IgnoreNonSpace, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.579486768281344 > 7.633422959810049.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -47.38626516541756 (T) = (0 -10.5602527641188) / Math.Sqrt((0.061982594311039724 / (16)) + (0.0028737073009159793 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.4158579013677772 = (7.458554106253996 - 10.5602527641188) / 7.458554106253996 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (en-US, None, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.920080455337603 > 6.91140896096496.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.365729963690487 (T) = (0 -9.887669401166406) / Math.Sqrt((0.003831397075320445 / (17)) + (0.2794091846036189 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.49614579307954365 = (6.608760621392678 - 9.887669401166406) / 6.608760621392678 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, IgnoreCase, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.902503214212185 > 8.322820950938361.
IsChangePoint: Marked as a change because one of 3/31/2023 7:02:21 PM, 4/1/2023 8:54:51 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -54.18255228614697 (T) = (0 -10.894061564567926) / Math.Sqrt((0.003436505841231288 / (17)) + (0.01951819403832021 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.3735906743651086 = (7.931082940413309 - 10.894061564567926) / 7.931082940413309 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsSuffix_SecondHalf(Options: (en-US, Ordinal, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.001169372169993 > 18.101404098197293.
IsChangePoint: Marked as a change because one of 3/5/2023 5:50:49 PM, 4/25/2023 6:46:56 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.807889688095036 (T) = (0 -20.343634033388366) / Math.Sqrt((0.09479256998043456 / (16)) + (0.2774301358354789 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.1839388319749254 = (17.18301105087769 - 20.343634033388366) / 17.18301105087769 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, None, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.598215860501963 > 7.958292153748862.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -35.39195174870933 (T) = (0 -11.168101331020903) / Math.Sqrt((0.05930271124621438 / (16)) + (0.039254810284809386 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.4405971040086296 = (7.752411343840937 - 11.168101331020903) / 7.752411343840937 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, None, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.91830236947145 > 7.505184286618962.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -4.760247152744189 (T) = (0 -9.409537241469906) / Math.Sqrt((2.1011027359561467 / (17)) + (0.2065183082780313 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.2467972117783425 = (7.546966862436927 - 9.409537241469906) / 7.546966862436927 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_FirstHalf(Options: (en-US, Ordinal, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 19.909490577525744 > 17.89913839097987.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 4/25/2023 1:02:54 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -21.887465332927743 (T) = (0 -20.105030390372058) / Math.Sqrt((0.0767738741397182 / (19)) + (0.11812696231492648 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.18686679815948481 = (16.93958447700249 - 20.105030390372058) / 16.93958447700249 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (en-US, IgnoreNonSpace, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.756293899739887 > 7.188717391847004.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.805330739283438 (T) = (0 -9.871235920344699) / Math.Sqrt((0.12158715856472918 / (17)) + (0.07592068903565632 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.3942299388532539 = (7.0800630837577145 - 9.871235920344699) / 7.0800630837577145 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsSuffix_DifferentLastChar(Options: (, IgnoreCase, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.211497084149789 > 8.389900079187703.
IsChangePoint: Marked as a change because one of 5/19/2023 1:23:34 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -34.65931491736956 (T) = (0 -11.031886183579257) / Math.Sqrt((0.01451750775241068 / (16)) + (0.04655457595569473 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.37576680559352876 = (8.018718098682369 - 11.031886183579257) / 8.018718098682369 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.StringSearch.IsPrefix_DifferentFirstChar(Options: (, IgnoreCase, False))

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.712750524873021 > 7.92583375930838.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -23.00960411323486 (T) = (0 -10.809193052011498) / Math.Sqrt((0.293007980139749 / (17)) + (0.031686129067785104 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.45778610856593016 = (7.414800421335363 - 10.809193052011498) / 7.414800421335363 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
18.93 ns 21.75 ns 1.15 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt32&gt;*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_VectorOf<UInt32>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.74654654784288 > 19.87769855886319.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -18.787314994119306 (T) = (0 -21.58157611123798) / Math.Sqrt((0.009371544734202162 / (19)) + (0.13902303673217795 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.14180624955803908 = (18.901259403328364 - 21.58157611123798) / 18.901259403328364 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Boolean

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.68 ns 2.09 ns 3.08 0.65 False
4.00 ns 7.38 ns 1.84 0.44 False
0.64 ns 3.32 ns 5.22 0.61 False
6.26 ns 10.01 ns 1.60 0.40 False
6.77 ns 10.28 ns 1.52 0.35 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Boolean*'

Payloads

Baseline
Compare

System.Tests.Perf_Boolean.TryParse(value: "false")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.0892105422551386 > 0.6048916167882386.
IsChangePoint: Marked as a change because one of 5/15/2023 9:18:21 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -24.564222728916477 (T) = (0 -2.0804324720298353) / Math.Sqrt((0.06863853608613105 / (17)) + (0.0012150024051867595 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -3.2773768551697957 = (0.4863804482215187 - 2.0804324720298353) / 0.4863804482215187 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Boolean.TryParse(value: "Bogus")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.381993600670239 > 4.225979350985756.
IsChangePoint: Marked as a change because one of 5/15/2023 5:11:34 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -33.4025329402122 (T) = (0 -7.503516010295051) / Math.Sqrt((0.05249083326406735 / (16)) + (0.05451853376087723 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.8809329771384035 = (3.989252196381118 - 7.503516010295051) / 3.989252196381118 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Boolean.TryParse(value: "False")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.322017274555686 > 0.56130918185936.
IsChangePoint: Marked as a change because one of 5/15/2023 5:11:34 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.349351074438822 (T) = (0 -2.506956731020132) / Math.Sqrt((0.18577924110318766 / (16)) + (0.2892977367430288 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -3.278885942802811 = (0.5858900574895888 - 2.506956731020132) / 0.5858900574895888 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Boolean.TryParse(value: " True ")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.00891691037454 > 6.616378396646717.
IsChangePoint: Marked as a change because one of 5/15/2023 5:11:34 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.878922907700897 (T) = (0 -10.252297108228502) / Math.Sqrt((0.33156689016855306 / (16)) + (0.20378887355298503 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.637478969694195 = (6.261025208856975 - 10.252297108228502) / 6.261025208856975 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Boolean.Parse(value: " True ")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.275455906194965 > 7.049617414492973.
IsChangePoint: Marked as a change because one of 5/15/2023 5:11:34 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -26.36809259558785 (T) = (0 -10.419260801143405) / Math.Sqrt((0.18349967323583147 / (15)) + (0.06319089244703906 / (7))) is less than -2.0859634472534285 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (15) + (7) - 2, .025) and -0.584792484704848 = (6.574526887085718 - 10.419260801143405) / 6.574526887085718 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Version

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
18.77 ns 22.69 ns 1.21 0.04 False
11.08 ns 14.81 ns 1.34 0.15 False
26.12 ns 28.69 ns 1.10 0.04 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Version*'

Payloads

Baseline
Compare

System.Tests.Perf_Version.TryFormat3

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 22.686403783623327 > 20.216420676080705.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.844218418971463 (T) = (0 -22.335977497143634) / Math.Sqrt((0.4731770491935596 / (17)) + (0.22706049279208373 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.14966788022770744 = (19.428199988260687 - 22.335977497143634) / 19.428199988260687 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Version.TryFormat2

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.807591811075696 > 12.183257386839692.
IsChangePoint: Marked as a change because one of 4/19/2023 6:20:25 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -25.81732199766078 (T) = (0 -14.918428212978565) / Math.Sqrt((0.29060686981582495 / (17)) + (0.0053068244659200195 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.30077480781474564 = (11.468878489460433 - 14.918428212978565) / 11.468878489460433 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Version.TryFormat4

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.68856169008346 > 27.489984764349273.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.92406137551621 (T) = (0 -28.94475546208048) / Math.Sqrt((0.869896356682829 / (17)) + (0.2832308371066112 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.10293576115994399 = (26.243373804145797 - 28.94475546208048) / 26.243373804145797 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Hashing

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.98 ns 8.15 ns 1.36 0.15 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Hashing*'

Payloads

Baseline
Compare

System.Hashing.GetStringHashCode(BytesCount: 10)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.1454663970443 > 6.2518564468913045.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 7/17/2023 4:07:27 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.678861192604113 (T) = (0 -8.022681498356574) / Math.Sqrt((0.008555842657083439 / (16)) + (0.10675329262056166 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.3535450353143291 = (5.927162590857935 - 8.022681498356574) / 5.927162590857935 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.80 ns 6.97 ns 1.83 0.45 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

System.IO.Tests.Perf_Path.HasExtension

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.967624007898962 > 4.466616359090291.
IsChangePoint: Marked as a change because one of 4/18/2023 7:36:21 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.955221273790675 (T) = (0 -7.586395350303298) / Math.Sqrt((0.4417267542808922 / (19)) + (0.20389958583910803 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.7270887300686801 = (4.392591543343355 - 7.586395350303298) / 4.392591543343355 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Byte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.45 ns 13.61 ns 1.61 0.15 False
5.45 ns 11.54 ns 2.12 0.19 False
9.41 ns 13.66 ns 1.45 0.12 False
4.66 ns 10.66 ns 2.29 0.26 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Byte*'

Payloads

Baseline
Compare

System.Tests.Perf_Byte.TryParse(value: "255")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.608738771192678 > 9.47166518202215.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -71.91501734501061 (T) = (0 -13.507186057695526) / Math.Sqrt((0.06749428976235927 / (19)) + (0.004141277241508621 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5214619804355294 = (8.877767720379708 - 13.507186057695526) / 8.877767720379708 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Byte.Parse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.540475277974407 > 5.863038770295621.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -106.54224334370701 (T) = (0 -11.546068484550599) / Math.Sqrt((0.06336270625548461 / (19)) + (0.00046878214728222027 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.1654481652851432 = (5.3319532970812205 - 11.546068484550599) / 5.3319532970812205 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Byte.Parse(value: "255")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.655318131281799 > 9.880352359650114.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -27.24021928856786 (T) = (0 -13.568277250237147) / Math.Sqrt((0.1554827613103637 / (19)) + (0.10829364477564825 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.44669865904411626 = (9.378786083345219 - 13.568277250237147) / 9.378786083345219 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Byte.TryParse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.65977931800921 > 5.4303461403303475.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -32.344398327272486 (T) = (0 -11.530670129905971) / Math.Sqrt((0.3065893921472978 / (19)) + (0.15132200203265614 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.1979710905384655 = (5.2460517699899105 - 11.530670129905971) / 5.2460517699899105 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_UInt16

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
11.17 ns 16.37 ns 1.47 0.06 False
5.02 ns 11.46 ns 2.28 0.20 False
5.25 ns 11.58 ns 2.20 0.25 False
12.29 ns 16.95 ns 1.38 0.06 False
12.10 ns 17.09 ns 1.41 0.04 False
11.36 ns 16.28 ns 1.43 0.07 False

graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_UInt16*'

Payloads

Baseline
Compare

System.Tests.Perf_UInt16.TryParse(value: "65535")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.36790275596187 > 12.086688889358008.
IsChangePoint: Marked as a change because one of 4/25/2023 6:51:20 AM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -70.83796057633188 (T) = (0 -16.36902551947404) / Math.Sqrt((0.061501718231620296 / (16)) + (0.007429014324442364 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.434901987479599 = (11.407765591171968 - 16.36902551947404) / 11.407765591171968 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.460752888491916 > 5.776752897584276.
IsChangePoint: Marked as a change because one of 5/2/2023 4:25:00 AM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -111.32841166278328 (T) = (0 -11.517797005601574) / Math.Sqrt((0.03729277081994731 / (17)) + (0.005107790774307216 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.0947301249032408 = (5.498463438641577 - 11.517797005601574) / 5.498463438641577 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "0")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.5799627317988 > 5.463783069910395.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -119.43706249844855 (T) = (0 -11.672427886038333) / Math.Sqrt((0.034785611167391435 / (17)) + (0.007027508999178109 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.2994662393443859 = (5.076146666700498 - 11.672427886038333) / 5.076146666700498 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.94698276624066 > 12.694856831835965.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -79.04327056250168 (T) = (0 -16.88729123740814) / Math.Sqrt((0.053304538658462276 / (17)) + (0.0033772173996077007 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.39186634660183217 = (12.132839678635499 - 16.88729123740814) / 12.132839678635499 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.Parse(value: "65535")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 17.09476067610599 > 12.734655438659885.
IsChangePoint: Marked as a change because one of 4/25/2023 6:51:20 AM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -73.34706985516755 (T) = (0 -16.9833457660215) / Math.Sqrt((0.04941875306706286 / (17)) + (0.01031292240629385 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.4002289763054973 = (12.128977512543727 - 16.9833457660215) / 12.128977512543727 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_UInt16.TryParse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 16.278929043830008 > 11.911767655406562.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -31.6871356772025 (T) = (0 -16.118360664753673) / Math.Sqrt((0.027941089750329655 / (16)) + (0.14252787882151496 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.4130315623573805 = (11.406936047389618 - 16.118360664753673) / 11.406936047389618 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Memory.MemoryMarshal<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.79 ns 5.28 ns 1.39 0.54 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.MemoryMarshal&lt;Byte&gt;*'

Payloads

Baseline
Compare

System.Memory.MemoryMarshal<Byte>.TryGetArray

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.283594681211536 > 3.9205074119560503.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.900104577919327 (T) = (0 -5.565141977972516) / Math.Sqrt((0.08626602923774093 / (19)) + (0.14263274673457724 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5095111830231819 = (3.686717952514202 - 5.565141977972516) / 3.686717952514202 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in PerfLabTests.CastingPerf

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.95 ns 10.79 ns 1.55 0.46 False
2.38 ns 8.22 ns 3.46 0.67 False
2.90 ns 8.59 ns 2.96 0.54 False
6.86 ns 11.39 ns 1.66 0.52 False
2.65 ns 8.24 ns 3.11 0.60 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'PerfLabTests.CastingPerf*'

Payloads

Baseline
Compare

PerfLabTests.CastingPerf.CheckArrayIsNonvariantGenericInterfaceNo

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.787584952796182 > 8.598686468044543.
IsChangePoint: Marked as a change because one of 7/10/2023 2:17:11 PM, 7/11/2023 7:34:35 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -5.96053938657658 (T) = (0 -11.77601457090619) / Math.Sqrt((0.614885980191244 / (18)) + (2.873791003771561 / (7))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (7) - 2, .025) and -0.5095188627912879 = (7.801170863894258 - 11.77601457090619) / 7.801170863894258 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.CheckArrayIsArrayByVariance

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.218824445614716 > 2.9807965963589216.
IsChangePoint: Marked as a change because one of 6/9/2023 7:03:36 PM, 6/12/2023 3:15:58 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.195135276593039 (T) = (0 -7.8550885291468395) / Math.Sqrt((0.9528240109523749 / (17)) + (0.3300159856932547 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.3838861972270509 = (3.295076979045359 - 7.8550885291468395) / 3.295076979045359 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.CheckArrayIsVariantGenericInterfaceNo

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.586011507887285 > 3.119435066353328.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -23.044796029275638 (T) = (0 -8.803020355575322) / Math.Sqrt((0.32697253169750595 / (17)) + (0.3311213392809932 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -2.0793991809209826 = (2.8586811382285706 - 8.803020355575322) / 2.8586811382285706 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.CheckArrayIsNonvariantGenericInterface

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 11.387564922427249 > 7.786533995301257.
IsChangePoint: Marked as a change because one of 3/14/2023 1:07:29 AM, 3/18/2023 7:37:20 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -5.909280139645045 (T) = (0 -11.853600359173104) / Math.Sqrt((0.5079557035423208 / (19)) + (3.2523446039870922 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5371551932971376 = (7.711388160975208 - 11.853600359173104) / 7.711388160975208 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

PerfLabTests.CastingPerf.CheckListIsVariantGenericInterface

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.241124136580668 > 2.8625100462666686.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -9.711689256190537 (T) = (0 -8.41988692680386) / Math.Sqrt((4.6968516764353705 / (17)) + (0.08854256594006078 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.6315433831467565 = (3.199600272876937 - 8.41988692680386) / 3.199600272876937 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_DateTime

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.00 ns 1.19 ns ∞ 0.79 False
1.85 ns 3.82 ns 2.06 0.45 True
1.92 ns 3.60 ns 1.87 0.46 False
3.16 ns 4.72 ns 1.49 0.34 True

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_DateTime*'

Payloads

Baseline
Compare

System.Tests.Perf_DateTime.ObjectEquals

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.1899355780976428 > 0.010148945799766265.
IsChangePoint: Marked as a change because one of 3/16/2023 2:30:29 AM, 4/3/2023 11:26:06 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.506148322934498 (T) = (0 -1.2599325859008204) / Math.Sqrt((0.23726759741308026 / (19)) + (0.02094887489976302 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -5.249875611200316 = (0.2015932258944342 - 1.2599325859008204) / 0.2015932258944342 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_DateTime.DayOfYear

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.820246827761978 > 1.9621580650484862.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -93.96744253173712 (T) = (0 -3.807285328349655) / Math.Sqrt((0.005323510687064622 / (19)) + (0.0011340320858705816 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -1.0789957075480496 = (1.8313098552954377 - 3.807285328349655) / 1.8313098552954377 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_DateTime.Year

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.5975323358615277 > 2.031159759866353.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.753815900579855 (T) = (0 -3.438441679915665) / Math.Sqrt((0.1477156706211031 / (19)) + (0.1437851683947485 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.7494644812358572 = (1.9654252582977165 - 3.438441679915665) / 1.9654252582977165 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_DateTime.Day

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.7221674848048485 > 3.323249517464076.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.0587710311494 (T) = (0 -5.396674894571829) / Math.Sqrt((0.0070063154206139655 / (19)) + (0.08847143342293055 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.7358452252500706 = (3.1089608774275193 - 5.396674894571829) / 3.1089608774275193 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateFor<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
205.89 ns 219.27 ns 1.06 0.16 False
9.08 μs 10.51 μs 1.16 0.13 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateFor&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.IterateFor<Int32>.ImmutableArray(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 219.26682144694294 > 216.91653819474004.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -9.874542950713387 (T) = (0 -219.4566790600695) / Math.Sqrt((19.097861891658347 / (16)) + (0.4083270102623311 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.05301464984822771 = (208.4080018181134 - 219.4566790600695) / 208.4080018181134 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateFor<Int32>.ImmutableSortedSet(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.51096294489445 > 10.020599528288088.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/27/2023 8:47:55 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -4.5236044274576495 (T) = (0 -10490.254843525774) / Math.Sqrt((226920.4395238736 / (13)) + (142304.16696067626 / (13))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (13) + (13) - 2, .025) and -0.07836808798265231 = (9727.898071566942 - 10490.254843525774) / 9727.898071566942 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.90 ns 13.17 ns 1.21 0.04 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt64&gt;*'

Payloads

Baseline
Compare

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt64>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.171149766722284 > 11.42560600382228.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.760013419643366 (T) = (0 -13.138201122386945) / Math.Sqrt((0.00759097415172338 / (16)) + (0.16909264947881794 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.21405541793722102 = (10.821747449313161 - 13.138201122386945) / 10.821747449313161 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.IterateForEach<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
207.26 ns 219.26 ns 1.06 0.15 False
207.35 ns 219.44 ns 1.06 0.17 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.IterateForEach&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.IterateForEach<Int32>.Array(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 219.26188167740767 > 217.70608298037814.
IsChangePoint: Marked as a change because one of 3/22/2023 1:13:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -8.53371649478737 (T) = (0 -219.73061612541477) / Math.Sqrt((24.370255850466258 / (17)) + (0.7059425562817311 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.050539187436030746 = (209.1598473938837 - 219.73061612541477) / 209.1598473938837 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.IterateForEach<Int32>.ImmutableArray(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 219.43813620276558 > 217.69182596896403.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -9.914507164927322 (T) = (0 -219.77948143624533) / Math.Sqrt((19.5866048949962 / (16)) + (0.7181729916967942 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.05480925925469736 = (208.35945409839897 - 219.77948143624533) / 208.35945409839897 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<UInt64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.81 ns 13.13 ns 1.21 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;UInt64&gt;*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_VectorOf<UInt64>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.132560087259154 > 11.37255648238143.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.836314815201852 (T) = (0 -13.276517972601786) / Math.Sqrt((0.003118500829822143 / (16)) + (0.09367468394249383 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.22379310249401288 = (10.848662200779758 - 13.276517972601786) / 10.848662200779758 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Linq.Tests.Perf_Enumerable

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
7.21 ns 8.65 ns 1.20 0.09 False
22.94 ns 26.59 ns 1.16 0.03 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Linq.Tests.Perf_Enumerable*'

Payloads

Baseline
Compare

System.Linq.Tests.Perf_Enumerable.ElementAt(input: IList)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.64802639567617 > 6.861772932300975.
IsChangePoint: Marked as a change because one of 5/2/2023 12:51:17 PM, 5/10/2023 2:36:50 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.63103664222392 (T) = (0 -8.499534706472893) / Math.Sqrt((0.04677341913051398 / (16)) + (0.14051201348262107 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.2909202673465677 = (6.584089599850601 - 8.499534706472893) / 6.584089599850601 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Linq.Tests.Perf_Enumerable.Contains_ElementNotFound(input: ICollection)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.591955287687284 > 24.13971344401597.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -67.04539458673412 (T) = (0 -26.618268149478308) / Math.Sqrt((0.03657098927031306 / (19)) + (0.008367673097475999 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.16373101190019027 = (22.87321372145514 - 26.618268149478308) / 22.87321372145514 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Concurrent.IsEmpty<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.62 ns 5.13 ns 3.16 0.53 True
1.62 ns 5.14 ns 3.18 0.52 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;String&gt;*'

Payloads

Baseline
Compare

System.Collections.Concurrent.IsEmpty<String>.Stack(Size: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.130870936776132 > 1.7060789886699712.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -48.04462393435391 (T) = (0 -5.078291250595013) / Math.Sqrt((0.08854653813572007 / (19)) + (0.005040507006681999 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -2.2675587477311048 = (1.5541545363556897 - 5.078291250595013) / 1.5541545363556897 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Concurrent.IsEmpty<String>.Stack(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.143066999847259 > 1.4699132288879182.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.565604147414955 (T) = (0 -4.854227240202701) / Math.Sqrt((0.12694934171690814 / (19)) + (0.2324665751044731 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -2.1406316215699643 = (1.5456213351682841 - 4.854227240202701) / 1.5456213351682841 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Array

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
8.71 ns 10.53 ns 1.21 0.08 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Array*'

Payloads

Baseline
Compare

System.Tests.Perf_Array.IndexOfChar

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 10.533910883524456 > 9.12587594649153.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 5/3/2023 7:05:50 AM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -43.759568548565255 (T) = (0 -10.465285051731936) / Math.Sqrt((0.013781398555452195 / (17)) + (0.006607151330548222 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.21233844362117074 = (8.632313119159083 - 10.465285051731936) / 8.632313119159083 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_SByte

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
9.83 ns 14.36 ns 1.46 0.09 False
10.93 ns 15.24 ns 1.39 0.07 False
10.84 ns 15.05 ns 1.39 0.07 False
9.53 ns 14.43 ns 1.51 0.09 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_SByte*'

Payloads

Baseline
Compare

System.Tests.Perf_SByte.TryParse(value: "127")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.36455624377442 > 10.468831129189189.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -65.51978424614389 (T) = (0 -14.329079087820746) / Math.Sqrt((0.07331292079982744 / (17)) + (0.003827003363994791 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.4678711509442689 = (9.761809869076705 - 14.329079087820746) / 9.761809869076705 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.Parse(value: "-128")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.235652334445236 > 11.460913617503387.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -66.45527582970075 (T) = (0 -15.281601692433705) / Math.Sqrt((0.07276622968099038 / (17)) + (0.0012538923933024322 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.4092536299306891 = (10.843755423347957 - 15.281601692433705) / 10.843755423347957 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.TryParse(value: "-128")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.053050119512468 > 11.221559425308305.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.46642985409894 (T) = (0 -14.739764801195701) / Math.Sqrt((0.06506150465536656 / (17)) + (0.20768586473914777 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.38688765830594257 = (10.62794431324023 - 14.739764801195701) / 10.62794431324023 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_SByte.Parse(value: "127")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.426841856080467 > 10.463677612407729.
IsChangePoint: Marked as a change because one of 4/17/2023 8:43:19 PM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -61.34511922189521 (T) = (0 -14.370409688105655) / Math.Sqrt((0.0832039980395425 / (17)) + (0.0024112847819993967 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.4471321260831504 = (9.930267892677513 - 14.370409688105655) / 9.930267892677513 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Threading.Tests.Perf_SpinLock

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
13.91 ns 15.82 ns 1.14 0.05 False
1.91 ns 8.25 ns 4.32 0.62 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_SpinLock*'

Payloads

Baseline
Compare

System.Threading.Tests.Perf_SpinLock.EnterExit

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 15.818833159155213 > 14.983903819044144.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -25.689485459147377 (T) = (0 -15.826772049871057) / Math.Sqrt((0.033396281610100144 / (19)) + (0.017029112087048208 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.11740993488995882 = (14.163801086509633 - 15.826772049871057) / 14.163801086509633 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tests.Perf_SpinLock.TryEnter_Fail

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.251347727937617 > 1.892717179962672.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -37.25396583224519 (T) = (0 -7.96600067693832) / Math.Sqrt((0.20259772859521724 / (19)) + (0.12278772422105631 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -3.659742746534829 = (1.7095365796453368 - 7.96600067693832) / 1.7095365796453368 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Concurrent.IsEmpty<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.11 ns 5.03 ns 2.38 0.63 False
1.62 ns 4.28 ns 2.63 0.58 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.Concurrent.IsEmpty<Int32>.Stack(Size: 0)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.028753885820161 > 1.6619815942501797.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -58.57523079291292 (T) = (0 -5.061293185605657) / Math.Sqrt((0.05545032333748396 / (16)) + (0.0009748958001788015 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -2.2772319800906935 = (1.5443805065839713 - 5.061293185605657) / 1.5443805065839713 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Concurrent.IsEmpty<Int32>.Stack(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.27599159181499 > 1.7200545538467258.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -21.70112258147582 (T) = (0 -4.873719654309132) / Math.Sqrt((0.05026214634012704 / (19)) + (0.14762517781450665 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -2.184541916801393 = (1.5304303669534918 - 4.873719654309132) / 1.5304303669534918 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
10.90 ns 13.73 ns 1.26 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int64&gt;*'

Payloads

Baseline
Compare

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int64>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.726646738016129 > 11.371493074604164.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.398036562925864 (T) = (0 -13.254749281563507) / Math.Sqrt((0.00809470867742197 / (16)) + (0.2659989907101052 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.22480836956535305 = (10.821896396958172 - 13.254749281563507) / 10.821896396958172 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Net.Primitives.Tests.IPAddressPerformanceTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.32 ns 2.18 ns 6.74 0.79 False
1.57 ns 4.27 ns 2.73 0.72 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Net.Primitives.Tests.IPAddressPerformanceTests*'

Payloads

Baseline
Compare

System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address: 143.24.20.36)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 2.1755970457750795 > 0.5567084963581987.
IsChangePoint: Marked as a change because one of 4/11/2023 2:19:19 PM, 4/12/2023 8:39:35 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.749683004056187 (T) = (0 -2.0615378124696124) / Math.Sqrt((0.1424425368407384 / (16)) + (0.032654822790504016 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -1.9750914952103076 = (0.6929325756160932 - 2.0615378124696124) / 0.6929325756160932 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Net.Primitives.Tests.IPAddressPerformanceTests.TryWriteBytes(address: 1020:3040:5060:7080:9010:1112:1314:1516)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.272178049553287 > 1.6607193302480099.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.158572285511793 (T) = (0 -4.942885693213201) / Math.Sqrt((0.44558444915107526 / (17)) + (0.21323558245606974 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -2.143767888712336 = (1.572280737060957 - 4.942885693213201) / 1.572280737060957 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Random

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
4.30 ns 5.83 ns 1.36 0.21 False
4.04 ns 6.58 ns 1.63 0.22 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Random*'

Payloads

Baseline
Compare

System.Tests.Perf_Random.Next_int_int

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.832341392234637 > 4.456111779128644.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.10876796099224 (T) = (0 -6.608898248014735) / Math.Sqrt((0.014517661306823021 / (16)) + (0.11947960497078601 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.5315906287796152 = (4.3150552920794265 - 6.608898248014735) / 4.3150552920794265 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Random.Next_int

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.578696476569882 > 4.242753160488067.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -92.12876310299619 (T) = (0 -6.578570773211099) / Math.Sqrt((0.011946667206396937 / (16)) + (4.993297553967038E-07 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.6199332832336307 = (4.061013401786079 - 6.578570773211099) / 4.061013401786079 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
18.73 ns 21.69 ns 1.16 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Int32>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.686829161526294 > 19.931423109477844.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -47.87809628166654 (T) = (0 -21.870850580377752) / Math.Sqrt((0.016466711570875904 / (18)) + (0.019619420907203877 / (7))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (7) - 2, .025) and -0.1540356606905652 = (18.951624568768022 - 21.870850580377752) / 18.951624568768022 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.16 ns 9.36 ns 1.82 0.41 False
18.09 ns 21.60 ns 1.19 0.07 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

System.Tests.Perf_Enum.IsDefined_Generic_Flags

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 9.360741001142328 > 5.988497355243008.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.255541439397469 (T) = (0 -9.229678480993174) / Math.Sqrt((0.6855834517075652 / (19)) + (0.15923217347057142 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.5990466176652977 = (5.771988370463551 - 9.229678480993174) / 5.771988370463551 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.60305602328772 > 18.99952599445685.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 5/26/2023 9:55:14 AM, 6/8/2023 5:31:08 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -9.122625622556603 (T) = (0 -21.790882760925335) / Math.Sqrt((0.4314179130279856 / (16)) + (1.068811621052575 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.21572180581344172 = (17.924234522013048 - 21.790882760925335) / 17.924234522013048 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
19.00 ns 21.91 ns 1.15 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_VectorOf<Int32>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.914596068512193 > 19.794265738419952.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -83.19627067646113 (T) = (0 -21.851168316410703) / Math.Sqrt((0.010640174259782345 / (19)) + (0.004993166633171078 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.15722246867120032 = (18.882426592962428 - 21.851168316410703) / 18.882426592962428 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
3.58 ns 6.03 ns 1.68 0.47 False
0.09 ns 1.61 ns 16.91 0.71 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark*'

Payloads

Baseline
Compare

Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.ForEach_Array

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.026141069031126 > 3.763952690554397.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -137.58967404137704 (T) = (0 -6.026078228253213) / Math.Sqrt((0.005448135248581542 / (17)) + (6.418956356453433E-08 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.6913292411187819 = (3.5629244039245 - 6.026078228253213) / 3.5629244039245 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Microsoft.Extensions.Primitives.Performance.StringValuesBenchmark.ForEach_String

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.6056906246214744 > 0.47977380464314257.
IsChangePoint: Marked as a change because one of 5/4/2023 2:44:24 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -9.846962040225442 (T) = (0 -1.4982479462224745) / Math.Sqrt((0.20290249499065888 / (18)) + (0.007359409857133406 / (7))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (7) - 2, .025) and -2.6983258606348746 = (0.40511517986283585 - 1.4982479462224745) / 0.40511517986283585 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Type

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.95 ns 1.97 ns 2.08 0.72 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Type*'

Payloads

Baseline
Compare

System.Tests.Perf_Type.op_Equality

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 1.9679653276673568 > 0.9811474894159211.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -23.880714071951047 (T) = (0 -1.9598858185738202) / Math.Sqrt((0.03128982339148866 / (17)) + (0.0010762237297272327 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.1936785815456286 = (0.8934243307389718 - 1.9598858185738202) / 0.8934243307389718 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
19.02 ns 21.03 ns 1.11 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;UInt32&gt;*'

Payloads

Baseline
Compare

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<UInt32>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 21.025182690316363 > 19.799874594189404.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.77630120900178 (T) = (0 -21.798102515438615) / Math.Sqrt((0.026166953137600913 / (18)) + (0.1260627831818485 / (7))) is less than -2.0686576104106944 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (18) + (7) - 2, .025) and -0.1533615254069771 = (18.899626903842574 - 21.798102515438615) / 18.899626903842574 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Vector4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.52 ns 8.31 ns 1.27 0.03 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector4*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Vector4.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.306245446171786 > 6.953149331797143.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -14.942143696627303 (T) = (0 -8.896706413418636) / Math.Sqrt((0.008373270269593615 / (19)) + (0.15116951422816366 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.33212176625617174 = (6.678598487600846 - 8.896706413418636) / 6.678598487600846 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_VectorOf<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
11.10 ns 13.26 ns 1.19 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_VectorOf&lt;Double&gt;*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_VectorOf<Double>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 13.261371695309535 > 11.635613691895427.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -21.476382319074794 (T) = (0 -13.492895754200205) / Math.Sqrt((0.010732310493827486 / (19)) + (0.08958357202039091 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.2254788616353776 = (11.01030476869605 - 13.492895754200205) / 11.01030476869605 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Vector2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
2.85 ns 5.41 ns 1.90 0.35 True

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Vector2*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Vector2.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 5.413235670364192 > 3.206825283986489.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -84.88109935865116 (T) = (0 -5.3726463824002435) / Math.Sqrt((0.007879614722838706 / (19)) + (0.0025131657232585853 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.7839971587957265 = (3.011577880553917 - 5.3726463824002435) / 3.011577880553917 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Threading.Tests.Perf_SemaphoreSlim

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
64.48 ns 69.76 ns 1.08 0.01 False
61.12 ns 66.84 ns 1.09 0.03 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Threading.Tests.Perf_SemaphoreSlim*'

Payloads

Baseline
Compare

System.Threading.Tests.Perf_SemaphoreSlim.ReleaseWait

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 69.76433174337599 > 67.63442759814113.
IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/22/2023 10:35:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.955570137849016 (T) = (0 -70.45065248472773) / Math.Sqrt((0.19830922989249758 / (16)) + (1.6655246414404867 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.10133109976491872 = (63.968639857501124 - 70.45065248472773) / 63.968639857501124 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Threading.Tests.Perf_SemaphoreSlim.ReleaseWaitAsync

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 66.83542347577195 > 63.12208160177135.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -22.920359660255706 (T) = (0 -67.43966289259954) / Math.Sqrt((0.505879064897621 / (16)) + (0.47835425058281267 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.12038516169903235 = (60.1932845936028 - 67.43966289259954) / 60.1932845936028 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Tests.ReadOnlySequenceTests<Byte>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.33 ns 6.95 ns 1.30 0.37 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Tests.ReadOnlySequenceTests&lt;Byte&gt;*'

Payloads

Baseline
Compare

System.Buffers.Tests.ReadOnlySequenceTests<Byte>.IterateTryGetArray

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 6.954121206570552 > 5.494042035178991.
IsChangePoint: Marked as a change because one of 4/3/2023 11:26:06 PM, 5/9/2023 11:02:55 PM, 5/10/2023 9:01:12 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -38.88783952235097 (T) = (0 -7.055539100392638) / Math.Sqrt((0.0236195788218408 / (19)) + (0.005562061779936403 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.3312085476140879 = (5.300100508698064 - 7.055539100392638) / 5.300100508698064 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
11.05 ns 12.51 ns 1.13 0.10 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

System.Text.Tests.Perf_Encoding.GetByteCount(size: 16, encName: "utf-8")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.509660100752122 > 11.826947087077194.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 4/28/2023 1:33:49 PM, 5/3/2023 7:05:50 AM, 7/1/2023 8:07:46 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -9.961100333673516 (T) = (0 -13.165845201337605) / Math.Sqrt((0.745652919329642 / (19)) + (0.08657640087315854 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.20756170744286298 = (10.902834298395936 - 13.165845201337605) / 10.902834298395936 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Buffers.Binary.Tests.BinaryReadAndWriteTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.79 ns 8.96 ns 1.55 0.42 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Binary.Tests.BinaryReadAndWriteTests*'

Payloads

Baseline
Compare

System.Buffers.Binary.Tests.BinaryReadAndWriteTests.ReadStructFieldByFieldBE

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.956091446434224 > 6.125355511639744.
IsChangePoint: Marked as a change because one of 6/28/2023 11:03:41 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.48997223982143 (T) = (0 -8.887966761440932) / Math.Sqrt((0.04441084637604019 / (16)) + (0.1496990217939396 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.5171147958147276 = (5.858466864841218 - 8.887966761440932) / 5.858466864841218 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Collections.Concurrent.Count<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
5.40 ns 7.95 ns 1.47 0.16 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.Count&lt;Int32&gt;*'

Payloads

Baseline
Compare

System.Collections.Concurrent.Count<Int32>.Queue(Size: 512)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.945457758404879 > 5.668531744557306.
IsChangePoint: Marked as a change because one of 5/18/2023 7:10:13 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -53.31420501234164 (T) = (0 -7.986686614349596) / Math.Sqrt((0.03307566947720964 / (17)) + (0.0018799775937103549 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.4579636466250136 = (5.477973770359558 - 7.986686614349596) / 5.477973770359558 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Int

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
18.84 ns 20.27 ns 1.08 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Int*'

Payloads

Baseline
Compare

System.Runtime.Intrinsics.Tests.Perf_Vector128Int.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 20.272302430185896 > 19.683873483655372.
IsChangePoint: Marked as a change because one of 4/6/2023 11:23:56 PM, 4/12/2023 8:39:35 PM, 5/3/2023 7:05:50 AM, 7/4/2023 5:42:46 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -38.4498498589054 (T) = (0 -20.32549983585152) / Math.Sqrt((0.015322811035581621 / (16)) + (0.005200417661304651 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -0.08461115218090749 = (18.739895671348705 - 20.32549983585152) / 18.739895671348705 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Matrix3x2

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
0.28 ns 3.47 ns 12.46 0.64 False
0.06 ns 3.40 ns 55.53 0.80 False
0.72 ns 3.47 ns 4.82 0.64 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix3x2*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Matrix3x2.InequalityOperatorBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.4700621374734686 > 0.5277352382574204.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -15.596103122820551 (T) = (0 -3.376798630504573) / Math.Sqrt((0.04684965264280953 / (19)) + (0.20639195892863701 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -4.73263092395563 = (0.5890486715957138 - 3.376798630504573) / 0.5890486715957138 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.InvertBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.4028907559314336 > 0.1077119277042917.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.318918730708837 (T) = (0 -3.037258041684304) / Math.Sqrt((0.010361460117210574 / (16)) + (0.2184799027181731 / (7))) is less than -2.079613844716807 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (16) + (7) - 2, .025) and -23.399695412078707 = (0.1244793424831342 - 3.037258041684304) / 0.1244793424831342 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Numerics.Tests.Perf_Matrix3x2.EqualityOperatorBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 3.4670076656918454 > 0.7552882183833579.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.250589530366117 (T) = (0 -3.281084250984995) / Math.Sqrt((0.1818444942107349 / (19)) + (0.2350283674090677 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -3.455381392139621 = (0.7364317355128402 - 3.281084250984995) / 0.7364317355128402 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
11.09 ns 12.49 ns 1.13 0.02 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Runtime.Intrinsics.Tests.Perf_Vector128Of&lt;Double&gt;*'

Payloads

Baseline
Compare

System.Runtime.Intrinsics.Tests.Perf_Vector128Of<Double>.GetHashCodeBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 12.491865034637714 > 11.639376131626006.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -9.678437957605436 (T) = (0 -13.267089374595786) / Math.Sqrt((0.008305364107158575 / (19)) + (0.3744710332266689 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.20397351677984535 = (11.0194196049096 - 13.267089374595786) / 11.0194196049096 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Numerics.Tests.Perf_Matrix4x4

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
6.53 ns 8.40 ns 1.29 0.39 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Numerics.Tests.Perf_Matrix4x4*'

Payloads

Baseline
Compare

System.Numerics.Tests.Perf_Matrix4x4.EqualsBenchmark

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 8.398176840658378 > 6.878347687538018.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -19.06863339056721 (T) = (0 -8.393042444248966) / Math.Sqrt((0.16353792756640448 / (17)) + (0.00010300508423824983 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.28701147265608645 = (6.521342367622968 - 8.393042444248966) / 6.521342367622968 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@dotnet-issue-labeler dotnet-issue-labeler bot added the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Aug 3, 2023
@EgorBo EgorBo transferred this issue from dotnet/perf-autofiling-issues Aug 3, 2023
@EgorBo EgorBo self-assigned this Aug 3, 2023
@EgorBo EgorBo added tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI and removed untriaged New issue has not been triaged by the area owner labels Aug 3, 2023
@ghost
Copy link

ghost commented Aug 3, 2023

Tagging subscribers to this area: @JulieLeeMSFT, @jakobbotsch
See info in area-owners.md if you want to be subscribed.

Issue Details

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Int64

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
23.85 ns 26.14 ns 1.10 0.05 False
33.49 ns 37.63 ns 1.12 0.02 False
33.23 ns 37.92 ns 1.14 0.02 False
11.17 ns 14.99 ns 1.34 0.04 False
33.40 ns 36.51 ns 1.09 0.03 False
6.22 ns 7.81 ns 1.25 0.40 False
11.29 ns 14.34 ns 1.27 0.05 False

graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Int64*'

Payloads

Baseline
Compare

System.Tests.Perf_Int64.TryFormat(value: 9223372036854775807)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 26.137705630589398 > 24.946168217632582.
IsChangePoint: Marked as a change because one of 5/14/2023 7:30:23 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.96988176081783 (T) = (0 -26.177414265844696) / Math.Sqrt((0.1795955695337053 / (17)) + (0.0053610676933496864 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.09321733472089504 = (23.945297457735563 - 26.177414265844696) / 23.945297457735563 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParse(value: "-9223372036854775808")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.63239881623019 > 35.19169745471809.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -13.655672490042395 (T) = (0 -37.27632669811096) / Math.Sqrt((0.05575448746761843 / (19)) + (0.5155246679550679 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.11281388266078136 = (33.49735951260943 - 37.27632669811096) / 33.49735951260943 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParseSpan(value: "-9223372036854775808")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 37.91541186721144 > 35.10850531701371.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.05495389754118 (T) = (0 -37.17139801700723) / Math.Sqrt((0.07819416609189558 / (17)) + (0.5994983827453598 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.10793876735167553 = (33.55004727008392 - 37.17139801700723) / 33.55004727008392 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParse(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.98854804202657 > 11.723113429994191.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -26.061266785816077 (T) = (0 -14.895645118514304) / Math.Sqrt((0.037213479000615945 / (19)) + (0.13681616706416241 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.3451033461516067 = (11.073978189951967 - 14.895645118514304) / 11.073978189951967 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParse(value: "9223372036854775807")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 36.50534067023052 > 35.07997456237158.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.417045291246943 (T) = (0 -36.63475704991735) / Math.Sqrt((0.2969835311564679 / (19)) + (0.18037131662308017 / (7))) is less than -2.0638985616205927 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (19) + (7) - 2, .025) and -0.10032586393426689 = (33.29446144156609 - 36.63475704991735) / 33.29446144156609 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryFormat(value: 12345)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 7.809253575213003 > 6.5500550657364585.
IsChangePoint: Marked as a change because one of 4/26/2023 8:38:54 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -11.896320532104834 (T) = (0 -7.676588644599028) / Math.Sqrt((0.06498843870042768 / (17)) + (0.09454463207941372 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.25628422057555245 = (6.1105508760446625 - 7.676588644599028) / 6.1105508760446625 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Int64.TryParseSpan(value: "12345")

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 14.33778883668515 > 11.681744153510063.
IsChangePoint: Marked as a change because one of 7/6/2023 2:30:01 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -20.206760882260394 (T) = (0 -14.71737208357051) / Math.Sqrt((0.025067926714099274 / (17)) + (0.21036756619539698 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.3223766431694294 = (11.129485808442888 - 14.71737208357051) / 11.129485808442888 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline d5c4a4e6b7e78d4dc9fc5ff560c802a4b901b2ed
Compare 836d1664384b7f7eb3e8b3439038cb06d95846b1
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Regressions in System.Tests.Perf_Char

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio
1.85 ns 4.45 ns 2.41 0.55 False
25.30 ns 28.12 ns 1.11 0.17 False
1.82 ns 4.36 ns 2.40 0.52 False
1.54 ns 4.46 ns 2.89 0.53 False
1.87 ns 4.34 ns 2.33 0.51 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Char*'

Payloads

Baseline
Compare

System.Tests.Perf_Char.Char_ToUpper(c: 'A', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.454784779761455 > 1.940471593226209.
IsChangePoint: Marked as a change because one of 4/27/2023 8:46:44 AM, 5/18/2023 7:10:13 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -17.404654951633336 (T) = (0 -4.29344463753373) / Math.Sqrt((0.03263198650029867 / (17)) + (0.12907222072202787 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.3719410274745631 = (1.810097547874121 - 4.29344463753373) / 1.810097547874121 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Char.Char_ToLower(c: 'İ', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 28.124586897956824 > 26.59612595016098.
IsChangePoint: Marked as a change because one of 4/25/2023 1:02:54 PM, 5/10/2023 2:36:50 PM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -6.765847488000034 (T) = (0 -27.603716528515747) / Math.Sqrt((0.4517760996729567 / (17)) + (0.3938400422609239 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -0.07589988531982853 = (25.65639880174362 - 27.603716528515747) / 25.65639880174362 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Char.Char_ToLower(c: 'A', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.355267051395538 > 2.0041186461607876.
IsChangePoint: Marked as a change because one of 4/26/2023 12:55:36 AM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -52.16133250015398 (T) = (0 -4.44309144718427) / Math.Sqrt((0.03545241092210663 / (17)) + (0.002638495898547744 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.3955452567372602 = (1.8547307485377145 - 4.44309144718427) / 1.8547307485377145 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Char.Char_ToUpper(c: 'a', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.460913033845659 > 1.8259304854261336.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -12.024422375425454 (T) = (0 -4.144715129014192) / Math.Sqrt((0.0336722002161439 / (17)) + (0.2511553105099265 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.2961849969320636 = (1.8050440772637886 - 4.144715129014192) / 1.8050440772637886 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Char.Char_ToLower(c: 'a', cultureName: en-US)

ETL Files

Histogram

Description of detection logic

IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 4.341934270862805 > 1.9843361565410813.
IsChangePoint: Marked as a change because one of 7/29/2023 11:09:12 AM, 8/2/2023 4:26:52 PM falls between 7/24/2023 8:03:13 PM and 8/2/2023 4:26:52 PM.
IsRegressionStdDev: Marked as regression because -16.620088663019327 (T) = (0 -4.288372340468756) / Math.Sqrt((0.014158487864279378 / (17)) + (0.13863464484267096 / (7))) is less than -2.073873067894501 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (17) + (7) - 2, .025) and -1.256143690366358 = (1.9007532005961907 - 4.288372340468756) / 1.9007532005961907 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Author: performanceautofiler[bot]
Assignees: EgorBo
Labels:

arch-arm64, os-linux, tenet-performance, tenet-performance-benchmarks, area-CodeGen-coreclr, runtime-coreclr, needs-area-label
Milestone: -

@EgorBo
Copy link
Member

EgorBo commented Aug 3, 2023 •
edited
Loading

Same on other OSes:

@JulieLeeMSFT JulieLeeMSFT added this to the 8.0.0 milestone Aug 3, 2023
@JulieLeeMSFT JulieLeeMSFT removed the needs-area-label An area label is needed to ensure this gets routed to the appropriate area owners label Aug 3, 2023
@tannergooding
Copy link
Member

@EgorBo this is coming from BDNs own codegen changing due to a jit change rather than any of the tests code changing, correct?

@EgorBo
Copy link
Member

EgorBo commented Aug 3, 2023

@EgorBo this is coming from BDNs own codegen changing due to a jit change rather than any of the tests code changing, correct?

Yes, BDN uses volatile to consume [Benchmark]'s return value every iteration: https://github.com/dotnet/BenchmarkDotNet/blob/fd5b76691505d53f3e934f8502629c006539e82c/src/BenchmarkDotNet/Engines/Consumer.cs#L22-L155

@AndyAyersMS pointed me to the fact that all these "regressions" have smaller "overhead invocation" counter:

image

And presumably final result is "realTime - overhead" and overhead is now smaller here

@tannergooding
Copy link
Member

Thanks!

CC. @adamsitnik as an FYI. This is another case where the benchmark isn't changing, but rather changes to BDN itself or to its codegen produced by the JIT are influencing the result, sometimes significantly.

@AndyAyersMS
Copy link
Member

There is still a puzzle here -- the same consumption sequence should exist in both the overhead delegate and the benchmark delegate, so in principle if both change the same way, the net result (benchmark - overhead) should remain the same. But that doesn't seem to be the case.

@adamsitnik
Copy link
Member

cc @timcassell @AndreyAkinshin who introduced some changes to this are of BDN recently

@timcassell
Copy link

The only codegen change in BDN recently was dotnet/BenchmarkDotNet#2335, which shouldn't have any effect. We know the overhead measurements are currently flawed for non-void benchmarks (dotnet/BenchmarkDotNet#2305).

@EgorBo
Copy link
Member

EgorBo commented Aug 4, 2023 •
edited
Loading

The only codegen change in BDN recently was dotnet/BenchmarkDotNet#2335, which shouldn't have any effect. We know the overhead measurements are currently flawed for non-void benchmarks (dotnet/BenchmarkDotNet#2305).

We don't think it's BDN update's fault (or rather - anyone's fault). It's just that BDN uses volatile writes (those Consume) methods every iteration (or even more than that if a benchmark is unrolled?) which are fairly expensive on arm64 because of explicit memory barriers - my PR relaxed them to be less expensive (even more cheaper on armv8.3+ with LRCPC which the CPU in question does support - ampere) so the overhead metric signifacantly decreased:
image

Many of the benchmarks in ^ list (all of them?) don't use volatile anywhere so the Consume is the only culprit. The question is only in how exactly BDN measures overhead, esp in cases where it decides to unroll benchmark?

@timcassell
Copy link

The question is only in how exactly BDN measures overhead, esp in cases where it decides to unroll benchmark?

It does exactly the same number of volatile writes in the overhead as it does in the workload (once for each delegate invocation).

private void OverheadActionUnroll(System.Int64 invokeCount)
{
    $LoadArguments$
    for (System.Int64 i = 0; i < invokeCount; i++)
    {
        consumer.Consume(overheadDelegate($PassArguments$));@Unroll@
    }
}

private void WorkloadActionUnroll(System.Int64 invokeCount)
{
    $LoadArguments$
    for (System.Int64  i = 0; i < invokeCount; i++)
    {
        consumer.Consume(workloadDelegate($PassArguments$)$ConsumeField$);@Unroll@
    }
}

@EgorBo
Copy link
Member

EgorBo commented Aug 9, 2023 •
edited
Loading

So, a minimal repro:

// Licensed to the .NET Foundation under one or more agreements.
// The .NET Foundation licenses this file to you under the MIT license.

using BenchmarkDotNet.Attributes;
using BenchmarkDotNet.Running;
using System;
using System.Runtime.CompilerServices;
using System.Threading;

public class Tests
{
    public static int Main(string[] args)
    {
        BenchmarkSwitcher.FromAssembly(typeof(Tests).Assembly).Run(args);
        return 100;
    }

    public class Perf_Boolean
    {
        [Benchmark]
        [Arguments("Bogus")]
        public bool TryParse(string value) => bool.TryParse(value, out _);
    }
}

Running on Azure Linux arm64 Ampere:

Method Job Toolchain value Mean
TryParse Job-LNEXZL /Core_Root/corerun Bogus 5.455 ns
TryParse Job-QLLZJH /Core_Root_MyPR/corerun Bogus 8.504 ns
/Core_Root/corerun 
WorkloadPilot    1: 16 op, 840.00 ns, 52.5000 ns/op
WorkloadPilot    2: 32 op, 1080.00 ns, 33.7500 ns/op
WorkloadPilot    3: 64 op, 1520.00 ns, 23.7500 ns/op
WorkloadPilot    4: 128 op, 2080.00 ns, 16.2500 ns/op
WorkloadPilot    5: 256 op, 3480.00 ns, 13.5938 ns/op
WorkloadPilot    6: 512 op, 6400.00 ns, 12.5000 ns/op
WorkloadPilot    7: 1024 op, 12160.00 ns, 11.8750 ns/op
WorkloadPilot    8: 2048 op, 23800.00 ns, 11.6211 ns/op
WorkloadPilot    9: 4096 op, 46959.00 ns, 11.4646 ns/op
WorkloadPilot   10: 8192 op, 93360.00 ns, 11.3965 ns/op
WorkloadPilot   11: 16384 op, 186599.00 ns, 11.3891 ns/op
WorkloadPilot   12: 32768 op, 371199.00 ns, 11.3281 ns/op
WorkloadPilot   13: 65536 op, 742917.00 ns, 11.3360 ns/op
WorkloadPilot   14: 131072 op, 1488035.00 ns, 11.3528 ns/op
WorkloadPilot   15: 262144 op, 2985229.00 ns, 11.3877 ns/op
WorkloadPilot   16: 524288 op, 5979978.00 ns, 11.4059 ns/op
WorkloadPilot   17: 1048576 op, 11949155.00 ns, 11.3956 ns/op
WorkloadPilot   18: 2097152 op, 23776032.00 ns, 11.3373 ns/op
WorkloadPilot   19: 4194304 op, 47534864.00 ns, 11.3332 ns/op
WorkloadPilot   20: 8388608 op, 95069049.00 ns, 11.3331 ns/op
WorkloadPilot   21: 16777216 op, 190049097.00 ns, 11.3278 ns/op
WorkloadPilot   22: 33554432 op, 380285954.00 ns, 11.3334 ns/op
WorkloadPilot   23: 67108864 op, 760518428.00 ns, 11.3326 ns/op

OverheadWarmup   1: 67108864 op, 394571462.00 ns, 5.8796 ns/op
OverheadWarmup   2: 67108864 op, 394859900.00 ns, 5.8839 ns/op
OverheadWarmup   3: 67108864 op, 395143739.00 ns, 5.8881 ns/op
OverheadWarmup   4: 67108864 op, 395124939.00 ns, 5.8878 ns/op
OverheadWarmup   5: 67108864 op, 394588701.00 ns, 5.8798 ns/op
OverheadWarmup   6: 67108864 op, 394606872.00 ns, 5.8801 ns/op
OverheadWarmup   7: 67108864 op, 394551017.00 ns, 5.8793 ns/op

OverheadActual   1: 67108864 op, 425405472.00 ns, 6.3390 ns/op
OverheadActual   2: 67108864 op, 425451872.00 ns, 6.3397 ns/op
OverheadActual   3: 67108864 op, 426058230.00 ns, 6.3488 ns/op
OverheadActual   4: 67108864 op, 425385792.00 ns, 6.3387 ns/op
OverheadActual   5: 67108864 op, 425503511.00 ns, 6.3405 ns/op
OverheadActual   6: 67108864 op, 425486151.00 ns, 6.3402 ns/op
OverheadActual   7: 67108864 op, 425498786.00 ns, 6.3404 ns/op
OverheadActual   8: 67108864 op, 425467506.00 ns, 6.3400 ns/op
OverheadActual   9: 67108864 op, 425439026.00 ns, 6.3395 ns/op
OverheadActual  10: 67108864 op, 426120344.00 ns, 6.3497 ns/op
OverheadActual  11: 67108864 op, 425472865.00 ns, 6.3400 ns/op
OverheadActual  12: 67108864 op, 425405106.00 ns, 6.3390 ns/op
OverheadActual  13: 67108864 op, 425402706.00 ns, 6.3390 ns/op
OverheadActual  14: 67108864 op, 425412546.00 ns, 6.3391 ns/op
OverheadActual  15: 67108864 op, 425417626.00 ns, 6.3392 ns/op

WorkloadWarmup   1: 67108864 op, 760198601.00 ns, 11.3278 ns/op
WorkloadWarmup   2: 67108864 op, 760888280.00 ns, 11.3381 ns/op
WorkloadWarmup   3: 67108864 op, 760210962.00 ns, 11.3280 ns/op

// BeforeActualRun
WorkloadActual   1: 67108864 op, 761919156.00 ns, 11.3535 ns/op
WorkloadActual   2: 67108864 op, 761905636.00 ns, 11.3533 ns/op
WorkloadActual   3: 67108864 op, 762967846.00 ns, 11.3691 ns/op

// AfterActualRun
WorkloadResult   1: 67108864 op, 336467284.00 ns, 5.0138 ns/op
WorkloadResult   2: 67108864 op, 336453764.00 ns, 5.0136 ns/op
WorkloadResult   3: 67108864 op, 337515974.00 ns, 5.0294 ns/op
/Core_Root_MyPR/corerun 
WorkloadPilot    1: 16 op, 840.00 ns, 52.5000 ns/op
WorkloadPilot    2: 32 op, 1000.00 ns, 31.2500 ns/op
WorkloadPilot    3: 64 op, 1480.00 ns, 23.1250 ns/op
WorkloadPilot    4: 128 op, 2080.00 ns, 16.2500 ns/op
WorkloadPilot    5: 256 op, 3320.00 ns, 12.9688 ns/op
WorkloadPilot    6: 512 op, 6040.00 ns, 11.7969 ns/op
WorkloadPilot    7: 1024 op, 11400.00 ns, 11.1328 ns/op
WorkloadPilot    8: 2048 op, 22200.00 ns, 10.8398 ns/op
WorkloadPilot    9: 4096 op, 43919.00 ns, 10.7224 ns/op
WorkloadPilot   10: 8192 op, 87279.00 ns, 10.6542 ns/op
WorkloadPilot   11: 16384 op, 173519.00 ns, 10.5908 ns/op
WorkloadPilot   12: 32768 op, 347679.00 ns, 10.6103 ns/op
WorkloadPilot   13: 65536 op, 697518.00 ns, 10.6433 ns/op
WorkloadPilot   14: 131072 op, 1396754.00 ns, 10.6564 ns/op
WorkloadPilot   15: 262144 op, 2762669.00 ns, 10.5387 ns/op
WorkloadPilot   16: 524288 op, 5533177.00 ns, 10.5537 ns/op
WorkloadPilot   17: 1048576 op, 11132114.00 ns, 10.6164 ns/op
WorkloadPilot   18: 2097152 op, 22166070.00 ns, 10.5696 ns/op
WorkloadPilot   19: 4194304 op, 44297939.00 ns, 10.5615 ns/op
WorkloadPilot   20: 8388608 op, 88554719.00 ns, 10.5565 ns/op
WorkloadPilot   21: 16777216 op, 177089438.00 ns, 10.5554 ns/op
WorkloadPilot   22: 33554432 op, 354360875.00 ns, 10.5608 ns/op
WorkloadPilot   23: 67108864 op, 708517351.00 ns, 10.5577 ns/op

OverheadWarmup   1: 67108864 op, 138628955.00 ns, 2.0657 ns/op
OverheadWarmup   2: 67108864 op, 138113477.00 ns, 2.0581 ns/op
OverheadWarmup   3: 67108864 op, 137740878.00 ns, 2.0525 ns/op
OverheadWarmup   4: 67108864 op, 137714359.00 ns, 2.0521 ns/op
OverheadWarmup   5: 67108864 op, 137756398.00 ns, 2.0527 ns/op
OverheadWarmup   6: 67108864 op, 137753838.00 ns, 2.0527 ns/op
OverheadWarmup   7: 67108864 op, 137696679.00 ns, 2.0518 ns/op
OverheadWarmup   8: 67108864 op, 137707438.00 ns, 2.0520 ns/op
OverheadWarmup   9: 67108864 op, 137781238.00 ns, 2.0531 ns/op
OverheadWarmup  10: 67108864 op, 137813798.00 ns, 2.0536 ns/op

OverheadActual   1: 67108864 op, 133939173.00 ns, 1.9958 ns/op
OverheadActual   2: 67108864 op, 134086333.00 ns, 1.9980 ns/op
OverheadActual   3: 67108864 op, 133854455.00 ns, 1.9946 ns/op
OverheadActual   4: 67108864 op, 133825534.00 ns, 1.9942 ns/op
OverheadActual   5: 67108864 op, 133832214.00 ns, 1.9943 ns/op
OverheadActual   6: 67108864 op, 133911774.00 ns, 1.9954 ns/op
OverheadActual   7: 67108864 op, 133848974.00 ns, 1.9945 ns/op
OverheadActual   8: 67108864 op, 133844614.00 ns, 1.9944 ns/op
OverheadActual   9: 67108864 op, 133848214.00 ns, 1.9945 ns/op
OverheadActual  10: 67108864 op, 133838424.00 ns, 1.9943 ns/op
OverheadActual  11: 67108864 op, 133824581.00 ns, 1.9941 ns/op
OverheadActual  12: 67108864 op, 133845501.00 ns, 1.9945 ns/op
OverheadActual  13: 67108864 op, 133827341.00 ns, 1.9942 ns/op
OverheadActual  14: 67108864 op, 134434019.00 ns, 2.0032 ns/op
OverheadActual  15: 67108864 op, 133925260.00 ns, 1.9956 ns/op

WorkloadWarmup   1: 67108864 op, 708506010.00 ns, 10.5576 ns/op
WorkloadWarmup   2: 67108864 op, 708312570.00 ns, 10.5547 ns/op
WorkloadWarmup   3: 67108864 op, 708455537.00 ns, 10.5568 ns/op

// BeforeActualRun
WorkloadActual   1: 67108864 op, 711698139.00 ns, 10.6051 ns/op
WorkloadActual   2: 67108864 op, 712404898.00 ns, 10.6157 ns/op
WorkloadActual   3: 67108864 op, 711609940.00 ns, 10.6038 ns/op

// AfterActualRun
WorkloadResult   1: 67108864 op, 577849925.00 ns, 8.6106 ns/op
WorkloadResult   2: 67108864 op, 578556684.00 ns, 8.6212 ns/op
WorkloadResult   3: 67108864 op, 577761726.00 ns, 8.6093 ns/op

So OverheadActual jumps from 6.3390 ns/op to 1.9956 ns/op in my PR (#70794)
WorkloadActual and WorkloadPilot are also better with my PR. Also, when I benchmark it with Stopwatch I am also seeing improvements with my PR as expected.

I think for BDN we migth want to introduce a better way to consume a value similar to Java - "Blackhole" API intrinsified in JIT. I am going to close this one since there is no sign this is a real regression, we may spend some time investigating BDN internals but I'd rather invest into the "Blackhole" API for it, e.g. https://javadoc.io/doc/org.openjdk.jmh/jmh-core/1.23/org/openjdk/jmh/infra/Blackhole.html

@EgorBo EgorBo closed this as completed Aug 9, 2023
@timcassell
Copy link

timcassell commented Aug 9, 2023 •
edited
Loading

I think for BDN we migth want to introduce a better way to consume a value

I have a PR for that already awaiting review. dotnet/BenchmarkDotNet#2336

"Blackhole" API intrinsified in JIT

That would probably be useful in general.

@EgorBo
Copy link
Member

EgorBo commented Aug 9, 2023

I think for BDN we migth want to introduce a better way to consume a value

I have a PR for that already awaiting review. dotnet/BenchmarkDotNet#2336

Nice!! Thanks

"Blackhole" API intrinsified in JIT

That would probably be useful in general.

Going to put up a proposal once I have a moment!

@timcassell timcassell mentioned this issue Aug 9, 2023
Open
@DrewScoggins DrewScoggins mentioned this issue Sep 5, 2023
Closed
@ghost ghost locked as resolved and limited conversation to collaborators Sep 9, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Assignees

@EgorBo EgorBo

Labels
arch-arm64 area-CodeGen-coreclr CLR JIT compiler in src/coreclr/src/jit and related components such as SuperPMI os-linux Linux OS (any supported distro) runtime-coreclr specific to the CoreCLR runtime tenet-performance Performance related issue tenet-performance-benchmarks Issue from performance benchmark
Projects
None yet
Milestone
8.0.0
Development

No branches or pull requests
6 participants
@EgorBo @adamsitnik @AndyAyersMS @tannergooding @timcassell @JulieLeeMSFT