Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Perf] Linux/arm64: 76 Improvements on 5/26/2023 11:50:09 PM #18442

Closed
performanceautofiler bot opened this issue Jun 1, 2023 · 4 comments
Closed

[Perf] Linux/arm64: 76 Improvements on 5/26/2023 11:50:09 PM #18442

performanceautofiler bot opened this issue Jun 1, 2023 · 4 comments
Assignees
@kunalspathak
Labels
ampere arch-arm64 branch-refs/heads/main kind-micro os-linux perf-improvement runtime-coreclr untriaged

Comments

@performanceautofiler
Copy link

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueTrue<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 4.34 μs 2.66 μs 0.61 0.07 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueTrue&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueTrue<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.6587000405566936 < 4.020497015962609.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 106.24698962368225 (T) = (0 -2662.481796997044) / Math.Sqrt((325283.66798450175 / (299)) + (15.401457536166719 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.5685082316481518 = (6170.41156350866 - 2662.481796997044) / 6170.41156350866 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_String

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Join_List - Duration of single invocation 76.76 ns 56.25 ns 0.73 0.02 False
Insert - Duration of single invocation 17.24 ns 14.89 ns 0.86 0.34 False
Insert - Duration of single invocation 17.47 ns 15.13 ns 0.87 0.30 False
Replace_String - Duration of single invocation 75.28 ns 65.32 ns 0.87 0.02 False
Replace_String - Duration of single invocation 64.83 ns 50.76 ns 0.78 0.01 False
Join_Array - Duration of single invocation 71.03 ns 53.61 ns 0.75 0.02 False
Split - Duration of single invocation 29.17 ns 25.79 ns 0.88 0.60 False
TrimEnd_CharArr - Duration of single invocation 6.46 ns 4.02 ns 0.62 0.61 False
Trim_CharArr - Duration of single invocation 21.80 ns 18.11 ns 0.83 0.23 False
Replace_String - Duration of single invocation 14.81 ns 13.68 ns 0.92 0.42 False
TrimEnd_CharArr - Duration of single invocation 16.83 ns 13.80 ns 0.82 0.38 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_String*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_String.Join_List

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 56.245747887522235 < 73.17127825979739.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/10/2023 2:36:50 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 36.79645184186472 (T) = (0 -57.74622950246155) / Math.Sqrt((8.078659892172405 / (299)) + (2.4341340242363176 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.28061732208061163 = (80.27192101633062 - 57.74622950246155) / 80.27192101633062 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Insert(s1: "dzsdzsDDZSDZSDZSddsz", i: 7, s2: "Test")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 14.889916391006071 < 16.970749119080274.
IsChangePoint: Marked as a change because one of 5/12/2023 9:12:45 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 22.241398877906995 (T) = (0 -15.151285904124583) / Math.Sqrt((0.5200921570918419 / (299)) + (0.24727862309348728 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.22034247612538327 = (19.433258116753823 - 15.151285904124583) / 19.433258116753823 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Insert(s1: "Test", i: 2, s2: " Test")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 15.131049511320368 < 16.237741146439824.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 11.751073418407405 (T) = (0 -14.105150045231325) / Math.Sqrt((0.5678718732268835 / (299)) + (0.7791914646560727 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2189423333727507 = (18.059037953164147 - 14.105150045231325) / 18.059037953164147 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence. This is another very nice sentence.", oldValue: "a", newValue: "")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 65.31507270329064 < 73.21774808561386.
IsChangePoint: Marked as a change because one of 2/9/2023 1:41:04 PM, 4/10/2023 5:02:03 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 30.007132982532692 (T) = (0 -65.73230446148013) / Math.Sqrt((1.7098104991266878 / (299)) + (0.5815431781799041 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.11974342811787225 = (74.67402864250597 - 65.73230446148013) / 74.67402864250597 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "nice", newValue: "bad")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 50.75531471908362 < 61.42992552633629.
IsChangePoint: Marked as a change because one of 3/25/2023 12:18:49 AM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 46.066436344606 (T) = (0 -51.798767466487234) / Math.Sqrt((2.2178667551194824 / (299)) + (0.5995125565533262 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.21340445588286855 = (65.85184451384829 - 51.798767466487234) / 65.85184451384829 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Join_Array

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 53.60701845494741 < 67.41896221689832.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/10/2023 2:36:50 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 86.6713116335031 (T) = (0 -53.47061066593985) / Math.Sqrt((7.2854290489762965 / (299)) + (0.24204569612331375 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.28239885387596025 = (74.51299507358546 - 53.47061066593985) / 74.51299507358546 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Split(s: "ABCDEFGHIJKLMNOPQRSTUVWXYZ", arr: [' '], options: RemoveEmptyEntries)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 25.792080188908738 < 28.030365188215058.
IsChangePoint: Marked as a change because one of 5/25/2023 4:37:39 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 14.04463400283629 (T) = (0 -21.57429692100618) / Math.Sqrt((7.127249602434368 / (299)) + (2.633183405815701 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.26803484867373173 = (29.474486431376008 - 21.57429692100618) / 29.474486431376008 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.TrimEnd_CharArr(s: "Test", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.023530163114769 < 6.060809466263494.
IsChangePoint: Marked as a change because one of 3/13/2023 3:41:35 AM, 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 10.222917690636049 (T) = (0 -4.2182912304996165) / Math.Sqrt((0.33717449920522197 / (299)) + (0.07725661100310259 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.21091452790194087 = (5.3457976095844435 - 4.2182912304996165) / 5.3457976095844435 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Trim_CharArr(s: " Te st ", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 18.10930601087656 < 21.551383348352303.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 68.7833737937753 (T) = (0 -18.301119761782672) / Math.Sqrt((0.940869639009483 / (299)) + (0.030479753737373858 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.24557350445288634 = (24.258320551839333 - 18.301119761782672) / 24.258320551839333 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.Replace_String(text: "This is a very nice sentence", oldValue: "bad", newValue: "nice")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.678822096596123 < 14.37467641805541.
IsChangePoint: Marked as a change because one of 3/12/2023 3:57:11 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 10.731921336759036 (T) = (0 -13.257700327281878) / Math.Sqrt((0.44076531889135806 / (299)) + (0.29661751518070284 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.14493795075311428 = (15.504957025000563 - 13.257700327281878) / 15.504957025000563 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_String.TrimEnd_CharArr(s: "Test ", c: [' ', ' '])

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.797577727088433 < 16.108826786706654.
IsChangePoint: Marked as a change because one of 3/17/2023 1:20:12 AM, 4/6/2023 7:14:22 PM, 5/19/2023 1:23:34 PM, 5/26/2023 11:50:09 PM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 58.57743117945626 (T) = (0 -13.709216607652529) / Math.Sqrt((0.959816577743954 / (299)) + (0.017314204712335954 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.2501528331580073 = (18.282681076717765 - 13.709216607652529) / 18.282681076717765 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
SerializeToWriter - Duration of single invocation 8.45 μs 7.57 μs 0.90 0.00 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Serialization.Tests.WriteJson&lt;Dictionary&lt;String, String&gt;&gt;*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Serialization.Tests.WriteJson<Dictionary<String, String>>.SerializeToWriter(Mode: SourceGen)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 7.570754687499999 < 8.086524604397491.
IsChangePoint: Marked as a change because one of 4/4/2023 10:11:10 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 62.595633891528024 (T) = (0 -7637.263561740691) / Math.Sqrt((46058.965239424644 / (299)) + (3002.0896934732373 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.16519385551498225 = (9148.54737497414 - 7637.263561740691) / 9148.54737497414 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Depth

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ReadSpanEmptyLoop - Duration of single invocation 26.18 μs 23.61 μs 0.90 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Depth*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Depth.ReadSpanEmptyLoop(Depth: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.611570645080324 < 24.87845537042183.
IsChangePoint: Marked as a change because one of 2/8/2023 10:48:55 AM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 56.72562303075586 (T) = (0 -23294.900378629183) / Math.Sqrt((259911.48763009036 / (299)) + (45850.33883517565 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.17338241176844613 = (28180.988053334 - 23294.900378629183) / 28180.988053334 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Single

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithFormat - Duration of single invocation 141.85 ns 125.82 ns 0.89 0.01 False
ToString - Duration of single invocation 129.78 ns 117.35 ns 0.90 0.02 False
ToStringWithCultureInfo - Duration of single invocation 161.17 ns 149.20 ns 0.93 0.00 False
ToStringWithFormat - Duration of single invocation 130.75 ns 119.94 ns 0.92 0.01 False
ToString - Duration of single invocation 169.80 ns 156.16 ns 0.92 0.01 False
ToStringWithCultureInfo - Duration of single invocation 163.17 ns 149.41 ns 0.92 0.01 False
ToString - Duration of single invocation 169.10 ns 154.37 ns 0.91 0.01 False
ToStringWithFormat - Duration of single invocation 144.17 ns 129.16 ns 0.90 0.01 False
ToStringWithFormat - Duration of single invocation 136.01 ns 115.36 ns 0.85 0.01 False
ToStringWithCultureInfo - Duration of single invocation 121.62 ns 111.96 ns 0.92 0.02 False
ToStringWithFormat - Duration of single invocation 167.75 ns 155.98 ns 0.93 0.00 False
ToStringWithFormat - Duration of single invocation 143.96 ns 124.97 ns 0.87 0.01 False
ToStringWithFormat - Duration of single invocation 169.79 ns 157.84 ns 0.93 0.01 False
ToStringWithFormat - Duration of single invocation 199.73 ns 185.20 ns 0.93 0.01 False

graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Single*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 125.81875453800951 < 134.7949299419356.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 60.17547659199214 (T) = (0 -123.13249297723561) / Math.Sqrt((39.606212975074385 / (299)) + (2.1698729320632775 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.24532253822822075 = (163.15909671948293 - 123.13249297723561) / 163.15909671948293 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToString(value: 12345)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 117.34976851929741 < 122.73987558885398.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 34.67597100403856 (T) = (0 -116.63656794142443) / Math.Sqrt((18.836502313353648 / (299)) + (2.1656410905286694 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.15356077879819086 = (137.79674313274265 - 116.63656794142443) / 137.79674313274265 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithCultureInfo(value: 3.4028235E+38, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 149.20406142550334 < 152.65846786284573.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 45.13928193007844 (T) = (0 -148.11316997212285) / Math.Sqrt((22.845841901897845 / (299)) + (1.6347977775216211 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1450585092556771 = (173.2436331323371 - 148.11316997212285) / 173.2436331323371 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 119.94042773097229 < 123.23485145994849.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 26.724167225563722 (T) = (0 -117.01780158342773) / Math.Sqrt((10.4507469978758 / (299)) + (3.857326591858879 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1488085634941723 = (137.4753041029056 - 117.01780158342773) / 137.4753041029056 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToString(value: -3.4028235E+38)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 156.16296559345327 < 161.61531678171232.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 29.98158027510642 (T) = (0 -158.34198454509533) / Math.Sqrt((16.89563500963488 / (299)) + (3.7823542790288247 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12761374978752757 = (181.50444772201607 - 158.34198454509533) / 181.50444772201607 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithCultureInfo(value: -3.4028235E+38, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 149.40531701037125 < 155.11458340343535.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 42.245862843777 (T) = (0 -150.1381790767014) / Math.Sqrt((23.905802893880086 / (299)) + (2.152549640329011 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.14904366880649325 = (176.43464602481475 - 150.1381790767014) / 176.43464602481475 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToString(value: 3.4028235E+38)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 154.3661193631469 < 158.4399217550498.
IsChangePoint: Marked as a change because one of 4/6/2023 1:55:21 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 24.506950188693747 (T) = (0 -155.81274307305742) / Math.Sqrt((14.679173357105155 / (299)) + (5.613391151681571 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12671013251714477 = (178.42041786442277 - 155.81274307305742) / 178.42041786442277 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 129.1566832036885 < 136.80986981271818.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 54.559944017173486 (T) = (0 -127.02430211409809) / Math.Sqrt((38.55757257354011 / (299)) + (2.53277284630626 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.23130478167298726 = (165.24664013203258 - 127.02430211409809) / 165.24664013203258 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "R")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 115.3556860038889 < 122.0673652080635.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 59.628231158606546 (T) = (0 -116.22790913720976) / Math.Sqrt((14.770010764771587 / (299)) + (0.7355204640827795 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.16780086075898076 = (139.6635776903251 - 116.22790913720976) / 139.6635776903251 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithCultureInfo(value: 12345, culture: zh)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 111.96140055378301 < 115.70500720041284.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 38.07320004756897 (T) = (0 -109.5470403696337) / Math.Sqrt((28.316314624182677 / (299)) + (1.6860363363297914 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1675889460570247 = (131.60209712584893 - 109.5470403696337) / 131.60209712584893 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 3.4028235E+38, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 155.98251798071 < 159.52417387654324.
IsChangePoint: Marked as a change because one of 3/21/2023 8:47:40 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 33.63982025038331 (T) = (0 -156.446390338957) / Math.Sqrt((13.042284872700979 / (299)) + (2.565276224370769 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12103184147567392 = (177.98868914843317 - 156.446390338957) / 177.98868914843317 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: 12345, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 124.97107819994652 < 136.6727446628087.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 69.03946153077132 (T) = (0 -125.69579019948573) / Math.Sqrt((53.532186765545774 / (299)) + (1.6733083573277305 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.26207221665607117 = (170.33616708384892 - 125.69579019948573) / 170.33616708384892 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 157.8377227334116 < 161.53974233036328.
IsChangePoint: Marked as a change because one of 3/17/2023 1:20:12 AM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 33.50986886022601 (T) = (0 -158.67703604104346) / Math.Sqrt((15.957757308887398 / (299)) + (2.8529244841410404 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1253971148713654 = (181.42752412451236 - 158.67703604104346) / 181.42752412451236 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Single.ToStringWithFormat(value: -3.4028235E+38, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 185.2021853654857 < 188.62650583269382.
IsChangePoint: Marked as a change because one of 4/11/2023 11:54:05 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 44.096306469734536 (T) = (0 -185.10546012844955) / Math.Sqrt((30.231389458173897 / (299)) + (2.35923327457161 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.136207137522002 = (214.2938060375145 - 185.10546012844955) / 214.2938060375145 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Tests.DictionarySequentialKeys

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryGetValue_17_Int_Int - Duration of single invocation 6.40 ns 3.98 ns 0.62 0.03 False
TryGetValue_3k_Int_Int - Duration of single invocation 6.55 ns 3.93 ns 0.60 0.04 True
TryGetValue_3k_Int_32ByteRefsValue - Duration of single invocation 6.87 ns 4.20 ns 0.61 0.01 True

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Tests.DictionarySequentialKeys*'

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_17_Int_Int

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.9811549579611847 < 6.086996265334447.
IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 105.5351872890146 (T) = (0 -3.948256538038262) / Math.Sqrt((0.24160031796350936 / (299)) + (0.001244819386533139 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.4563046981094315 = (7.261891953653375 - 3.948256538038262) / 7.261891953653375 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_Int

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 3.934204547719453 < 6.179720890164743.
IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 126.34980833281068 (T) = (0 -3.9390551828310527) / Math.Sqrt((0.18835594657326435 / (299)) + (2.241730668220475E-05 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.44663308685286796 = (7.11834244015185 - 3.9390551828310527) / 7.11834244015185 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Collections.Tests.DictionarySequentialKeys.TryGetValue_3k_Int_32ByteRefsValue

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.20177850417003 < 6.493196382219968.
IsChangePoint: Marked as a change because one of 5/4/2023 12:54:08 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 204.05605727051815 (T) = (0 -4.208280619380092) / Math.Sqrt((0.0803021817736371 / (299)) + (1.1783174519058593E-05 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.44355750541333855 = (7.562831128679527 - 4.208280619380092) / 7.562831128679527 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Enum

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Parse_Flags - Duration of single invocation 230.14 ns 197.79 ns 0.86 0.02 False
ToString_Flags - Duration of single invocation 69.50 ns 65.03 ns 0.94 0.04 False
ToString_Flags - Duration of single invocation 46.34 ns 40.63 ns 0.88 0.19 False
TryParseGeneric_Flags - Duration of single invocation 111.58 ns 99.28 ns 0.89 0.01 False
TryParseGeneric_Flags - Duration of single invocation 18.01 ns 13.66 ns 0.76 0.04 False

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Enum*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Enum.Parse_Flags(text: "Red, Orange, Yellow, Green, Blue")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 197.79228510934985 < 216.89344761699954.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 28.721982419113264 (T) = (0 -198.5157391472748) / Math.Sqrt((162.29671553236898 / (299)) + (2.4274558337193284 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12007609302971303 = (225.60557518069368 - 198.5157391472748) / 225.60557518069368 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Flags(value: Red, Orange, Yellow, Green, Blue)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 65.03162710547089 < 66.65337060106367.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/8/2023 3:52:55 PM, 5/10/2023 2:36:50 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 11.373370257737188 (T) = (0 -63.936232563545715) / Math.Sqrt((5.482494665736114 / (299)) + (3.2506050885416733 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.10999590224743178 = (71.83813279623882 - 63.936232563545715) / 71.83813279623882 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.ToString_Flags(value: Yellow, Blue)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 40.63091313309406 < 43.986132531237224.
IsChangePoint: Marked as a change because one of 5/26/2023 11:50:09 PM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 9.294158405853867 (T) = (0 -41.87864662612731) / Math.Sqrt((1.5253719336111 / (299)) + (1.5511305618294613 / (6))) is greater than 1.967824097752068 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (6) - 2, .975) and 0.10229258892178125 = (46.650663801279855 - 41.87864662612731) / 46.650663801279855 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red, Orange, Yellow, Green, Blue")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 99.28311628234353 < 106.06312548259318.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 28.64679830088369 (T) = (0 -98.2881120703786) / Math.Sqrt((37.38691631776144 / (299)) + (0.2072581694671072 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.10283050732681415 = (109.55356025038432 - 98.2881120703786) / 109.55356025038432 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Enum.TryParseGeneric_Flags(text: "Red")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.657398864225689 < 16.83934460642889.
IsChangePoint: Marked as a change because one of 4/1/2023 8:54:51 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 95.77870037595491 (T) = (0 -13.736698724929342) / Math.Sqrt((1.1820411801641548 / (299)) + (0.0024764803733635828 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.31393721428356874 = (20.022509617082044 - 13.736698724929342) / 20.022509617082044 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Tests.Perf_Encoding

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetChars - Duration of single invocation 32.22 ns 30.19 ns 0.94 0.17 False
GetString - Duration of single invocation 23.91 ns 19.79 ns 0.83 0.24 False

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_Encoding*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_Encoding.GetChars(size: 16, encName: "utf-8")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 30.185389046656375 < 30.404477334184865.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 19.888174696986106 (T) = (0 -29.38741301989777) / Math.Sqrt((0.837337483230479 / (299)) + (0.3050825650654505 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12721120035258976 = (33.670703647628976 - 29.38741301989777) / 33.670703647628976 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_Encoding.GetString(size: 16, encName: "utf-8")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 19.794844050740757 < 22.496124437493624.
IsChangePoint: Marked as a change because one of 4/26/2023 5:34:01 AM, 5/25/2023 4:37:39 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 24.32937120796461 (T) = (0 -20.082635710555394) / Math.Sqrt((1.602023744284254 / (299)) + (0.5794884141734045 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.24238974471370328 = (26.507872049549114 - 20.082635710555394) / 26.507872049549114 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Booleans

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteBooleans - Duration of single invocation 2.10 ms 1.81 ms 0.86 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Booleans*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Booleans.WriteBooleans(Formatted: True, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.805014604761905 < 1.996454988576348.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 41.64802050897728 (T) = (0 -1803707.7420892664) / Math.Sqrt((2188014235.6866384 / (299)) + (691149930.4398233 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.19210746731837972 = (2232608.508092355 - 1803707.7420892664) / 2232608.508092355 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Memory.ReadOnlySpan

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
IndexOfString - Duration of single invocation 12.03 ns 9.95 ns 0.83 0.00 True
IndexOfString - Duration of single invocation 16.29 ns 13.31 ns 0.82 0.01 False
IndexOfString - Duration of single invocation 25.85 ns 23.81 ns 0.92 0.00 True
IndexOfString - Duration of single invocation 13.65 ns 11.50 ns 0.84 0.00 True
IndexOfString - Duration of single invocation 12.88 ns 10.92 ns 0.85 0.01 False
IndexOfString - Duration of single invocation 23.65 ns 20.54 ns 0.87 0.01 True

graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Memory.ReadOnlySpan*'

Payloads

Baseline
Compare

Histogram

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello Worldbbbbbbbbbbbbbbcbbbbbbbbbbbbbbbbbbba!", value: "y", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 9.94674865732791 < 11.435587602717833.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 439.3904277882019 (T) = (0 -9.933873104406933) / Math.Sqrt((0.0013298771909583363 / (299)) + (0.00012261191242314554 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1717008088302902 = (11.993097675706395 - 9.933873104406933) / 11.993097675706395 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "StrIng", value: "string", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 13.310819342509523 < 15.721746003713891.
IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 107.83612953860191 (T) = (0 -13.275436757110608) / Math.Sqrt((0.18664971441268405 / (299)) + (0.003399085284773432 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2129764025017251 = (16.867901800288404 - 13.275436757110608) / 16.867901800288404 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "~", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 23.81270296259891 < 24.570244270768004.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 25.489407898179753 (T) = (0 -23.89228628336798) / Math.Sqrt((0.007836021453246059 / (299)) + (0.041619131617426273 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.07616395413237873 = (25.862041636326847 - 23.89228628336798) / 25.862041636326847 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "だだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだすだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだだ", value: "す", comparisonType: Ordinal)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 11.496704295359725 < 12.920462627296303.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 206.10715511524157 (T) = (0 -11.492420114104421) / Math.Sqrt((0.005823329512111117 / (299)) + (0.0006636896534415759 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.16088170217230652 = (13.695828280536794 - 11.492420114104421) / 13.695828280536794 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "Hello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello WorldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylongHello Worldbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbareallyreallylong!xyz", value: "w", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 10.91918385134226 < 12.240991006079884.
IsChangePoint: Marked as a change because one of 5/1/2023 10:17:01 AM, 5/4/2023 7:16:27 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 32.201311171907534 (T) = (0 -10.914246779191924) / Math.Sqrt((1.1310006803374602 / (299)) + (0.00048584526040012935 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1547736234022604 = (12.912809019430584 - 10.914246779191924) / 12.912809019430584 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Memory.ReadOnlySpan.IndexOfString(input: "More Test's", value: "Tests", comparisonType: OrdinalIgnoreCase)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.544219490818328 < 22.607383609446888.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 219.94988745822167 (T) = (0 -20.575803613255893) / Math.Sqrt((0.04305521096597374 / (299)) + (0.0007055634349964678 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.14328537116975085 = (24.01710315294828 - 20.575803613255893) / 24.01710315294828 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Double

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToStringWithFormat - Duration of single invocation 144.28 ns 124.16 ns 0.86 0.01 False
ToStringWithFormat - Duration of single invocation 145.12 ns 125.16 ns 0.86 0.01 False
ToStringWithFormat - Duration of single invocation 128.72 ns 115.39 ns 0.90 0.01 False
ToStringWithFormat - Duration of single invocation 198.38 ns 180.84 ns 0.91 0.01 False
ToString - Duration of single invocation 129.36 ns 113.89 ns 0.88 0.01 False
ToStringWithFormat - Duration of single invocation 129.52 ns 117.62 ns 0.91 0.01 False
ToStringWithFormat - Duration of single invocation 142.90 ns 122.15 ns 0.85 0.01 False
ToStringWithFormat - Duration of single invocation 195.85 ns 181.28 ns 0.93 0.02 False

graph
graph
graph
graph
graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Double*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 124.15957711870776 < 137.0829343921643.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 85.94109631430392 (T) = (0 -124.62730335270614) / Math.Sqrt((32.20288220664642 / (299)) + (0.5126125650445144 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.22679780137672542 = (161.1833276917879 - 124.62730335270614) / 161.1833276917879 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 125.15689864663442 < 137.27664609577633.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 55.66008431948039 (T) = (0 -125.54307044087922) / Math.Sqrt((27.358054236080385 / (299)) + (1.5437938261497706 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.19849985384919072 = (156.63511858830927 - 125.54307044087922) / 156.63511858830927 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "G")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 115.39217827349295 < 122.2824773866119.
IsChangePoint: Marked as a change because one of 4/11/2023 5:52:15 AM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 48.286229776823305 (T) = (0 -115.34883948071004) / Math.Sqrt((9.836030817550952 / (299)) + (0.9873107252310729 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1486366084939029 = (135.48719692616 - 115.34883948071004) / 135.48719692616 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: -1.7976931348623157E+308, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 180.83868706035648 < 188.32385549281318.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 58.96207058556403 (T) = (0 -181.76959357937093) / Math.Sqrt((21.799082008461447 / (299)) + (1.277450547349157 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.14084267451044524 = (211.56729761431896 - 181.76959357937093) / 211.56729761431896 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToString(value: 12345)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 113.88618565276776 < 121.91622039965736.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 22.274831503753866 (T) = (0 -116.04362576513242) / Math.Sqrt((15.827173431631708 / (299)) + (5.119347781022038 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1452925013546559 = (135.76998674874625 - 116.04362576513242) / 135.76998674874625 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 12345, format: "R")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 117.62255286261959 < 122.53617535182367.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 42.98552830242255 (T) = (0 -116.07430450998399) / Math.Sqrt((13.976436460768271 / (299)) + (1.3854351221404129 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.15481779644286822 = (137.33642760278224 - 116.07430450998399) / 137.33642760278224 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "E")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 122.1516236303039 < 135.59241321325086.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 91.6721258208091 (T) = (0 -122.32459133628375) / Math.Sqrt((24.451116780194162 / (299)) + (0.3024741195938969 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.20945181233688556 = (154.73388370907423 - 122.32459133628375) / 154.73388370907423 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Tests.Perf_Double.ToStringWithFormat(value: 1.7976931348623157E+308, format: "G17")

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 181.2802555623308 < 185.6352504761963.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 48.22322870976487 (T) = (0 -182.5584091853551) / Math.Sqrt((21.10107939169729 / (299)) + (1.5517342083241152 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12495684461224718 = (208.62789230601894 - 182.5584091853551) / 208.62789230601894 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Sort<Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
List - Duration of single invocation 6.63 μs 5.72 μs 0.86 0.19 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Sort&lt;Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Sort<Int32>.List(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.716101559999999 < 6.142326575862745.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 1:23:34 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 7.752188601329285 (T) = (0 -5710.9978178732545) / Math.Sqrt((81940.77399999455 / (299)) + (35224.34406400208 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.08998093152478513 = (6275.690274756917 - 5710.9978178732545) / 6275.690274756917 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.IO.Tests.Perf_Path

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
GetFileNameWithoutExtension - Duration of single invocation 39.59 ns 30.91 ns 0.78 0.18 False
GetExtension - Duration of single invocation 20.40 ns 16.91 ns 0.83 0.37 False
GetFileName - Duration of single invocation 27.46 ns 20.94 ns 0.76 0.19 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.IO.Tests.Perf_Path*'

Payloads

Baseline
Compare

Histogram

System.IO.Tests.Perf_Path.GetFileNameWithoutExtension

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 30.91037392893681 < 37.64405328985286.
IsChangePoint: Marked as a change because one of 4/17/2023 5:51:51 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 7.05075217649051 (T) = (0 -33.33442732158331) / Math.Sqrt((2.0446655008196672 / (299)) + (6.04604013365132 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.16482366859333625 = (39.9130411962694 - 33.33442732158331) / 39.9130411962694 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetExtension

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 16.909729898905525 < 19.171776781940554.
IsChangePoint: Marked as a change because one of 4/17/2023 5:51:51 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 3.3756061997708873 (T) = (0 -17.621050496541034) / Math.Sqrt((1.3254840448972225 / (299)) + (3.163500284507266 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1145835203543643 = (19.901425940923744 - 17.621050496541034) / 19.901425940923744 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.IO.Tests.Perf_Path.GetFileName

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 20.938892278833833 < 27.346024396942695.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 8.925471708531818 (T) = (0 -22.422511675523815) / Math.Sqrt((0.7210823133828523 / (299)) + (4.08219137122362 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.23348598605696222 = (29.252578906130875 - 22.422511675523815) / 29.252578906130875 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Tests.Perf_StringBuilder

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString_MultipleSegments - Duration of single invocation 35.71 ns 32.05 ns 0.90 0.19 False
Insert_Strings - Duration of single invocation 796.30 ns 679.41 ns 0.85 0.56 False
Insert_Primitives - Duration of single invocation 38.22 μs 28.15 μs 0.74 0.21 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Tests.Perf_StringBuilder*'

Payloads

Baseline
Compare

Histogram

System.Text.Tests.Perf_StringBuilder.ToString_MultipleSegments(length: 100)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 32.0466123298704 < 33.7123484009934.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 9.244212358313373 (T) = (0 -31.829297079150802) / Math.Sqrt((5.065324101126338 / (299)) + (1.9394681341093498 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.13605333575495435 = (36.84173849662888 - 31.829297079150802) / 36.84173849662888 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.Insert_Strings

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 679.4122080903417 < 759.0945720104479.
IsChangePoint: Marked as a change because one of 4/10/2023 1:33:42 PM, 5/25/2023 4:37:39 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 11.786895292927252 (T) = (0 -654.6743110394232) / Math.Sqrt((2101.247063344755 / (299)) + (1458.397156258783 / (9))) is greater than 1.9677467375895035 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (9) - 2, .975) and 0.18969579067540135 = (807.9364558467548 - 654.6743110394232) / 807.9364558467548 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Text.Tests.Perf_StringBuilder.Insert_Primitives

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 28.148561726248435 < 35.53966994729241.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 51.763628205442316 (T) = (0 -27803.315535053076) / Math.Sqrt((1082715.699886491 / (299)) + (266027.9955412405 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.2752806743658853 = (38364.252961966675 - 27803.315535053076) / 38364.252961966675 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@performanceautofiler
Copy link
Author

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Tests.Perf_Decimal

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 75.94 ns 67.89 ns 0.89 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Tests.Perf_Decimal*'

Payloads

Baseline
Compare

Histogram

System.Tests.Perf_Decimal.ToString(value: 123456.789)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 67.89230905010486 < 72.23406293338185.
IsChangePoint: Marked as a change because one of 4/9/2023 9:22:58 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 75.0488790205758 (T) = (0 -68.30575891404627) / Math.Sqrt((4.590494519210652 / (299)) + (0.11922122626142699 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.16508171688866685 = (81.81131051472968 - 68.30575891404627) / 81.81131051472968 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.TryGetValueFalse<Int32, Int32>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Dictionary - Duration of single invocation 4.44 μs 2.36 μs 0.53 0.07 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.TryGetValueFalse&lt;Int32, Int32&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.TryGetValueFalse<Int32, Int32>.Dictionary(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 2.356920658648771 < 4.071879893270733.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 96.78175309420051 (T) = (0 -2353.056366961214) / Math.Sqrt((522203.16043791774 / (299)) + (31.4445096318028 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.6324999553475895 = (6402.873689952297 - 2353.056366961214) / 6402.873689952297 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Globalization.Tests.Perf_NumberCultureInfo

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToString - Duration of single invocation 140.51 ns 128.24 ns 0.91 0.01 False
ToString - Duration of single invocation 139.96 ns 128.15 ns 0.92 0.01 False
ToString - Duration of single invocation 141.74 ns 127.83 ns 0.90 0.01 False
ToString - Duration of single invocation 141.84 ns 128.36 ns 0.90 0.01 False

graph
graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Globalization.Tests.Perf_NumberCultureInfo*'

Payloads

Baseline
Compare

Histogram

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: ja)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 128.23578693301423 < 134.61172977365626.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 40.06075453833989 (T) = (0 -129.28781636021444) / Math.Sqrt((37.132014631985896 / (299)) + (1.3420994377937288 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1483269667502141 = (151.80452041187948 - 129.28781636021444) / 151.80452041187948 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: )

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 128.1530370999429 < 133.60976212908608.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 44.02265094124336 (T) = (0 -129.58095366440236) / Math.Sqrt((36.972571329338344 / (299)) + (0.9144922358833316 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.14626135180728211 = (151.78058758229196 - 129.58095366440236) / 151.78058758229196 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: da)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 127.8299102573086 < 135.10912103048085.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 25.502437898417604 (T) = (0 -129.63455797411376) / Math.Sqrt((36.845799530854 / (299)) + (4.435388322683928 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.146135802278339 = (151.82104873352645 - 129.63455797411376) / 151.82104873352645 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Globalization.Tests.Perf_NumberCultureInfo.ToString(culturestring: fr)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 128.3642128031155 < 134.7536621816329.
IsChangePoint: Marked as a change because one of 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 46.57575901398694 (T) = (0 -128.77605625172495) / Math.Sqrt((36.53043149178451 / (299)) + (0.8520330413119933 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.15154881990369606 = (151.77780321680754 - 128.77605625172495) / 151.77780321680754 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Collections.Concurrent.IsEmpty<String>

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
Queue - Duration of single invocation 13.41 ns 8.99 ns 0.67 0.28 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Collections.Concurrent.IsEmpty&lt;String&gt;*'

Payloads

Baseline
Compare

Histogram

System.Collections.Concurrent.IsEmpty<String>.Queue(Size: 512)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 8.98823545197615 < 13.214216797407712.
IsChangePoint: Marked as a change because one of 3/14/2023 8:42:09 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 24.30221115853871 (T) = (0 -9.159360134449638) / Math.Sqrt((3.6971580380844724 / (299)) + (0.21860466615413587 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.3564918659196691 = (14.233479966092005 - 9.159360134449638) / 14.233479966092005 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Perf_Ascii

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
ToUtf16 - Duration of single invocation 6.78 ns 5.29 ns 0.78 0.14 True
FromUtf16 - Duration of single invocation 7.14 ns 5.33 ns 0.75 0.12 True

graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Perf_Ascii*'

Payloads

Baseline
Compare

Histogram

System.Text.Perf_Ascii.ToUtf16(Size: 6)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.288379531708405 < 6.486069745530598.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 9.416827706215843 (T) = (0 -5.146868403256832) / Math.Sqrt((0.08137263105994924 / (41)) + (0.21136995935369343 / (7))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .975) and 0.2471093716688236 = (6.836143537428736 - 5.146868403256832) / 6.836143537428736 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Text.Perf_Ascii.FromUtf16(Size: 6)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 5.327967309359419 < 6.6055536439279745.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 26.05908618874475 (T) = (0 -5.389403231144205) / Math.Sqrt((0.13488889346858968 / (41)) + (0.00254610302073831 / (7))) is greater than 2.01289559891803 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (41) + (7) - 2, .975) and 0.22616823921048523 = (6.964567111649147 - 5.389403231144205) / 6.964567111649147 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Buffers.Text.Tests.Utf8ParserTests

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
TryParseDecimal - Duration of single invocation 48.14 ns 41.23 ns 0.86 0.01 True
TryParseDateTimeOffset - Duration of single invocation 7.13 ns 4.49 ns 0.63 0.22 False
TryParseUInt16 - Duration of single invocation 3.27 ns 1.84 ns 0.56 0.49 False

graph
graph
graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Buffers.Text.Tests.Utf8ParserTests*'

Payloads

Baseline
Compare

Histogram

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDecimal(value: 123456.789)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 41.225263713945616 < 45.72344228441166.
IsChangePoint: Marked as a change because one of 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 32.81321666722707 (T) = (0 -40.769687291633076) / Math.Sqrt((0.424017276572474 / (299)) + (0.3470854772947645 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.15380619812504598 = (48.18008262563214 - 40.769687291633076) / 48.18008262563214 is greater than 0.05.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseDateTimeOffset(value: 12/30/2017 3:45:22 AM -08:00)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.486369338153327 < 6.721330124699414.
IsChangePoint: Marked as a change because one of 3/21/2023 2:53:21 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 14.28524150528259 (T) = (0 -4.579899206894662) / Math.Sqrt((0.30412306866953787 / (299)) + (0.2853682595176802 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.3893439791945757 = (7.4999656940318165 - 4.579899206894662) / 7.4999656940318165 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

System.Buffers.Text.Tests.Utf8ParserTests.TryParseUInt16(value: 0)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 1.844978119865182 < 3.09546709370837.
IsChangePoint: Marked as a change because one of 3/8/2023 10:34:21 AM, 3/9/2023 2:25:45 AM, 4/25/2023 6:46:56 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 14.93618528718548 (T) = (0 -2.223370056003945) / Math.Sqrt((0.13205329462488247 / (299)) + (0.0370789121030631 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.3372654939288007 = (3.3548427547321378 - 2.223370056003945) / 3.3548427547321378 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in BenchmarksGame.ReverseComplement_1

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
RunBench - Duration of single invocation 712.70 μs 664.61 μs 0.93 0.04 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'BenchmarksGame.ReverseComplement_1*'

Payloads

Baseline
Compare

Histogram

BenchmarksGame.ReverseComplement_1.RunBench

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 664.6138962339743 < 675.9478104747199.
IsChangePoint: Marked as a change because one of 3/16/2023 8:14:39 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 45.9715009196058 (T) = (0 -666386.3852735806) / Math.Sqrt((216409039.4332032 / (299)) + (4125415.817684515 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.07326093220745156 = (719065.817372827 - 666386.3852735806) / 719065.817372827 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Strings

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteStringsUtf8 - Duration of single invocation 4.71 ms 4.46 ms 0.95 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Strings*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Strings.WriteStringsUtf8(Formatted: False, SkipValidation: True, Escaped: NoneEscaped)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.456810794968555 < 4.4762148717948715.
IsChangePoint: Marked as a change because one of 5/10/2023 2:36:50 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 39.95759674932748 (T) = (0 -4462387.596928555) / Math.Sqrt((6318850258.492485 / (299)) + (925545398.8630592 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.09981823899226495 = (4957207.299926854 - 4462387.596928555) / 4957207.299926854 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_DateTimes

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDateTimes - Duration of single invocation 5.26 ms 4.96 ms 0.94 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_DateTimes*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_DateTimes.WriteDateTimes(Formatted: False, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.962705447222222 < 5.005347229486586.
IsChangePoint: Marked as a change because one of 5/19/2023 4:22:12 AM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 97.09859451327934 (T) = (0 -4943520.207837301) / Math.Sqrt((13517363509.627825 / (299)) + (87427087.60890335 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.12982652774823322 = (5681074.366752237 - 4943520.207837301) / 5681074.366752237 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Run Information

Name Value
Architecture arm64
OS ubuntu 20.04
Queue AmpereUbuntu
Baseline 1a609335296e0e3a2ce2b8248365faf60d580bd6
Compare d58754bcbd414461adfc5ffb8f2b6c89f078cd12
Diff Diff
Configs CompilationMode:tiered, RunKind:micro

Improvements in System.Text.Json.Tests.Perf_Deep

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
WriteDeepUtf8 - Duration of single invocation 5.18 ms 4.90 ms 0.95 0.01 False

graph
Test Report

Repro

General Docs link: https://github.com/dotnet/performance/blob/main/docs/benchmarking-workflow-dotnet-runtime.md

Payloads

Baseline
Compare

git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net8.0 --filter 'System.Text.Json.Tests.Perf_Deep*'

Payloads

Baseline
Compare

Histogram

System.Text.Json.Tests.Perf_Deep.WriteDeepUtf8(Formatted: False, SkipValidation: True)

Description of detection logic

IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsRegressionBase: Marked as not a regression because the compare was not 5% greater than the baseline, or the value was too small.
IsImprovementBase: Marked as improvement because the compare was 5% less than the baseline, and the value was not too small.
IsImprovementCheck: Marked as improvement because the three check build points were 0.05 less than the baseline.
IsImprovementWindowed: Marked as improvement because 4.901952923076924 < 4.94914519289855.
IsChangePoint: Marked as a change because one of 4/12/2023 8:39:35 PM, 4/24/2023 7:54:29 PM, 5/26/2023 9:55:14 AM, 5/31/2023 10:16:51 PM falls between 5/22/2023 11:33:11 PM and 5/31/2023 10:16:51 PM.
IsImprovementStdDev: Marked as improvement because 19.300648751667627 (T) = (0 -4830276.150781751) / Math.Sqrt((37438528581.56682 / (299)) + (8675293219.410686 / (7))) is greater than 1.9677981407170237 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (299) + (7) - 2, .975) and 0.1286179809984229 = (5543235.969358473 - 4830276.150781751) / 5543235.969358473 is greater than 0.05.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.

JIT Disasms

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

@EgorBo
Copy link
Member

EgorBo commented Jun 1, 2023

dotnet/runtime#86551

@EgorBo EgorBo mentioned this issue Jun 1, 2023
Closed
@EgorBo EgorBo closed this as completed Jun 1, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kunalspathak kunalspathak

Labels
ampere arch-arm64 branch-refs/heads/main kind-micro os-linux perf-improvement runtime-coreclr untriaged
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@EgorBo @kunalspathak