-
Notifications
You must be signed in to change notification settings - Fork 5.3k
Closed
Labels
arch-x64runtime-coreclrspecific to the CoreCLR runtimespecific to the CoreCLR runtimetenet-performancePerformance related issuePerformance related issuetenet-performance-benchmarksIssue from performance benchmarkIssue from performance benchmarkuntriagedNew issue has not been triaged by the area ownerNew issue has not been triaged by the area owner
Description
Run Information
| Architecture | x64 |
|---|---|
| OS | Windows 10.0.18362 |
| Baseline | a5fda753afce966b9d31c76559636eb515bfa05a |
| Compare | 1df36c702a46363336b6ea5d0d9558d513e7be8e |
| Diff | Diff |
Regressions in System.Collections.Tests.Perf_BitArray
| Benchmark | Baseline | Test | Test/Base | Test Quality | Edge Detector | Baseline IR | Compare IR | IR Ratio | Baseline ETL | Compare ETL |
|---|---|---|---|---|---|---|---|---|---|---|
| BitArrayBoolArrayCtor - Duration of single invocation | 31.59 ns | 40.88 ns | 1.29 | 0.13 | True |
Repro
git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'Details
Payloads
Histogram
System.Collections.Tests.Perf_BitArray.BitArrayBoolArrayCtor(Size: 512)
Description of detection logic
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.87508460982235 > 33.235490963584866.
IsChangePoint: Marked as a change because one of 2/2/2022 4:36:37 PM, 2/8/2022 12:37:33 AM falls between 1/30/2022 3:20:30 AM and 2/8/2022 12:37:33 AM.
IsRegressionStdDev: Marked as regression because -22.70736782183897 (T) = (0 -40.01058690490022) / Math.Sqrt((5.070068259802953 / (46)) + (0.15899135968215822 / (32))) is less than -1.9916726096441384 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (32) - 2, .025) and -0.2385693211100522 = (32.30387368955759 - 40.01058690490022) / 32.30387368955759 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.
Docs
Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository
Reactions are currently unavailable
Metadata
Metadata
Assignees
Labels
arch-x64runtime-coreclrspecific to the CoreCLR runtimespecific to the CoreCLR runtimetenet-performancePerformance related issuePerformance related issuetenet-performance-benchmarksIssue from performance benchmarkIssue from performance benchmarkuntriagedNew issue has not been triaged by the area ownerNew issue has not been triaged by the area owner
