Skip to content

Regressions in System.Collections.Tests.Perf_BitArray #64990

@performanceautofiler

Description

@performanceautofiler

Run Information

Architecture x64
OS Windows 10.0.18362
Baseline a5fda753afce966b9d31c76559636eb515bfa05a
Compare 1df36c702a46363336b6ea5d0d9558d513e7be8e
Diff Diff

Regressions in System.Collections.Tests.Perf_BitArray

Benchmark Baseline Test Test/Base Test Quality Edge Detector Baseline IR Compare IR IR Ratio Baseline ETL Compare ETL
BitArrayBoolArrayCtor - Duration of single invocation 31.59 ns 40.88 ns 1.29 0.13 True

graph
Test Report

Repro

git clone https://github.com/dotnet/performance.git
py .\performance\scripts\benchmarks_ci.py -f net6.0 --filter 'System.Collections.Tests.Perf_BitArray*'
Details

Payloads

Baseline
Compare

Histogram

System.Collections.Tests.Perf_BitArray.BitArrayBoolArrayCtor(Size: 512)


Description of detection logic

IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsRegressionBase: Marked as regression because the compare was 5% greater than the baseline, and the value was not too small.
IsRegressionChecked: Marked as regression because the three check build points were 0.05 greater than the baseline.
IsRegressionWindowed: Marked as regression because 40.87508460982235 > 33.235490963584866.
IsChangePoint: Marked as a change because one of 2/2/2022 4:36:37 PM, 2/8/2022 12:37:33 AM falls between 1/30/2022 3:20:30 AM and 2/8/2022 12:37:33 AM.
IsRegressionStdDev: Marked as regression because -22.70736782183897 (T) = (0 -40.01058690490022) / Math.Sqrt((5.070068259802953 / (46)) + (0.15899135968215822 / (32))) is less than -1.9916726096441384 = MathNet.Numerics.Distributions.StudentT.InvCDF(0, 1, (46) + (32) - 2, .025) and -0.2385693211100522 = (32.30387368955759 - 40.01058690490022) / 32.30387368955759 is less than -0.05.
IsImprovementBase: Marked as not an improvement because the compare was not 5% less than the baseline, or the value was too small.
IsChangeEdgeDetector: Marked as regression because Edge Detector said so.

Docs

Profiling workflow for dotnet/runtime repository
Benchmarking workflow for dotnet/runtime repository

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions