[Perf] Regressions in System.Buffers.Text.Tests.Base64Tests #72866
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | b5fd812c63f2d548395b33e556f0e68cf7598dcf |
| Compare | 02ffbcdb785d405e3d4e8457c05ab08f6b745e51 |
| Diff | Diff |
Regressions in System.Perf_Convert
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ToBase64CharArray - Duration of single invocation | 889.24 ns | 988.56 ns | 1.11 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Perf_Convert*'Details
Payloads
Histogram
System.Perf_Convert.ToBase64CharArray(binaryDataSize: 1024, formattingOptions: None)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 988.5578792093395 > 934.2436747398181.
IsChangePoint: Marked as a change because one of 7/24/2022 8:36:55 PM, 7/26/2022 6:20:16 AM falls between 7/17/2022 2:02:19 PM and 7/26/2022 6:20:16 AM.
IsRegressionStdDev: Marked as regression because -157.90484221330118 (T) = (0 -985.1798912135928) / Math.Sqrt((1.4283183589673323 / (69)) + (2.4194046780736205 / (7))) is less than -1.9925434951804422 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (69) + (7) - 2, .025) and -0.10743179921890782 = (889.6077319691004 - 985.1798912135928) / 889.6077319691004 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
| Architecture | x64 |
|---|---|
| OS | ubuntu 18.04 |
| Baseline | b5fd812c63f2d548395b33e556f0e68cf7598dcf |
| Compare | 02ffbcdb785d405e3d4e8457c05ab08f6b745e51 |
| Diff | Diff |
Regressions in System.Buffers.Text.Tests.Base64Tests
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| ConvertToBase64CharArray - Duration of single invocation | 866.07 ns | 958.87 ns | 1.11 | 0.00 | False |
Repro
git clone https://github.com/dotnet/performance.git
python3 .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Buffers.Text.Tests.Base64Tests*'Details
Payloads
Histogram
System.Buffers.Text.Tests.Base64Tests.ConvertToBase64CharArray(NumberOfBytes: 1000)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 958.8704418275734 > 910.0775226681224.
IsChangePoint: Marked as a change because one of 7/24/2022 8:36:55 PM, 7/26/2022 6:20:16 AM falls between 7/17/2022 2:02:19 PM and 7/26/2022 6:20:16 AM.
IsRegressionStdDev: Marked as regression because -368.28363235682093 (T) = (0 -958.7521004129097) / Math.Sqrt((0.49035060946694337 / (70)) + (0.385901776692054 / (7))) is less than -1.9921021540016632 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (70) + (7) - 2, .025) and -0.10588923110301522 = (866.9512944408087 - 958.7521004129097) / 866.9512944408087 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked not as a regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository