Upload code coverage reports to Datadog by nikita-tkachenko-datadog · Pull Request #9425 · DataDog/dd-trace-java

nikita-tkachenko-datadog · 2025-08-26T15:57:16Z

What Does This Do

Implements uploading code coverage reports to Datadog's Code Coverage directly from the tracer.
If Jacoco coverage data is detected, it is either uploaded as is or, if there were tests skipped by ITR, is augmented with the coverage of the skipped tests (received from the backend) and is uploaded as an LCOV report (as it is simpler that reconstruct a Jacoco report).

Motivation

Simplify onboarding to Code Coverage for existing Test Optimization customers.

Contributor Checklist

Format the title according the contribution guidelines
Assign the type: and (comp: or inst:) labels in addition to any usefull labels
Don't use close, fix or any linking keywords when referencing an issue.
Use solves instead, and assign the PR milestone to the issue
Update the CODEOWNERS file on source file addition, move, or deletion
Update the public documentation in case of new configuration flag or behavior

Jira ticket: SDTEST-2562

datadog-official · 2025-08-26T16:15:28Z

🎯 Code Coverage
• Patch Coverage: 45.03%
• Total Coverage: 57.45% (-0.16%)

View detailed report

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: c0d713c | Docs | Was this helpful? Give us feedback!}

pr-commenter · 2025-08-26T16:53:16Z

Benchmarks

Startup

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/coverage-reports-upload
git_commit_date	1756292489	1756298913
git_commit_sha	`8799a82`	`c0d713c`
release_version	1.53.0-SNAPSHOT~8799a82b0a	1.53.0-SNAPSHOT~c0d713c226

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1756300743	1756300743
ci_job_id	1101084366	1101084366
ci_pipeline_id	74846592	74846592
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-rjfv8mzf 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-rjfv8mzf 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux
module	Agent	Agent
parent	None	None

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 46 metrics, 13 unstable metrics.

Startup time reports for petclinic

gantt
    title petclinic - global startup overhead: candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.047 s) : 0, 1046768
Total [baseline] (10.723 s) : 0, 10722565
Agent [candidate] (1.043 s) : 0, 1043361
Total [candidate] (10.734 s) : 0, 10734430
section appsec
Agent [baseline] (1.221 s) : 0, 1221397
Total [baseline] (10.862 s) : 0, 10862195
Agent [candidate] (1.224 s) : 0, 1224239
Total [candidate] (10.823 s) : 0, 10823245
section iast
Agent [baseline] (1.179 s) : 0, 1178936
Total [baseline] (10.922 s) : 0, 10921724
Agent [candidate] (1.187 s) : 0, 1186925
Total [candidate] (10.925 s) : 0, 10925448
section profiling
Agent [baseline] (1.197 s) : 0, 1196840
Total [baseline] (10.923 s) : 0, 10923049
Agent [candidate] (1.209 s) : 0, 1208977
Total [candidate] (10.992 s) : 0, 10992256

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.047 s	-
Agent	appsec	1.221 s	174.629 ms (16.7%)
Agent	iast	1.179 s	132.167 ms (12.6%)
Agent	profiling	1.197 s	150.072 ms (14.3%)
Total	tracing	10.723 s	-
Total	appsec	10.862 s	139.631 ms (1.3%)
Total	iast	10.922 s	199.159 ms (1.9%)
Total	profiling	10.923 s	200.484 ms (1.9%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.043 s	-
Agent	appsec	1.224 s	180.878 ms (17.3%)
Agent	iast	1.187 s	143.564 ms (13.8%)
Agent	profiling	1.209 s	165.616 ms (15.9%)
Total	tracing	10.734 s	-
Total	appsec	10.823 s	88.816 ms (0.8%)
Total	iast	10.925 s	191.018 ms (1.8%)
Total	profiling	10.992 s	257.826 ms (2.4%)

gantt
    title petclinic - break down per module: candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.453 ms) : 0, 1453
crashtracking [candidate] (1.455 ms) : 0, 1455
BytebuddyAgent [baseline] (731.343 ms) : 0, 731343
BytebuddyAgent [candidate] (731.091 ms) : 0, 731091
GlobalTracer [baseline] (241.93 ms) : 0, 241930
GlobalTracer [candidate] (241.737 ms) : 0, 241737
AppSec [baseline] (30.098 ms) : 0, 30098
AppSec [candidate] (30.029 ms) : 0, 30029
Debugger [baseline] (6.035 ms) : 0, 6035
Debugger [candidate] (6.042 ms) : 0, 6042
Remote Config [baseline] (667.13 µs) : 0, 667
Remote Config [candidate] (654.56 µs) : 0, 655
Telemetry [baseline] (14.279 ms) : 0, 14279
Telemetry [candidate] (11.366 ms) : 0, 11366
section appsec
crashtracking [baseline] (1.445 ms) : 0, 1445
crashtracking [candidate] (1.44 ms) : 0, 1440
BytebuddyAgent [baseline] (754.505 ms) : 0, 754505
BytebuddyAgent [candidate] (756.219 ms) : 0, 756219
GlobalTracer [baseline] (235.043 ms) : 0, 235043
GlobalTracer [candidate] (235.314 ms) : 0, 235314
AppSec [baseline] (169.428 ms) : 0, 169428
AppSec [candidate] (169.393 ms) : 0, 169393
Debugger [baseline] (6.499 ms) : 0, 6499
Debugger [candidate] (6.541 ms) : 0, 6541
Remote Config [baseline] (667.361 µs) : 0, 667
Remote Config [candidate] (640.53 µs) : 0, 641
Telemetry [baseline] (9.235 ms) : 0, 9235
Telemetry [candidate] (9.927 ms) : 0, 9927
IAST [baseline] (23.556 ms) : 0, 23556
IAST [candidate] (23.644 ms) : 0, 23644
section iast
crashtracking [baseline] (1.448 ms) : 0, 1448
crashtracking [candidate] (1.456 ms) : 0, 1456
BytebuddyAgent [baseline] (850.829 ms) : 0, 850829
BytebuddyAgent [candidate] (856.998 ms) : 0, 856998
GlobalTracer [baseline] (233.217 ms) : 0, 233217
GlobalTracer [candidate] (235.072 ms) : 0, 235072
AppSec [baseline] (26.696 ms) : 0, 26696
AppSec [candidate] (27.897 ms) : 0, 27897
Debugger [baseline] (5.765 ms) : 0, 5765
Debugger [candidate] (6.655 ms) : 0, 6655
Remote Config [baseline] (595.431 µs) : 0, 595
Remote Config [candidate] (606.27 µs) : 0, 606
Telemetry [baseline] (8.326 ms) : 0, 8326
Telemetry [candidate] (8.358 ms) : 0, 8358
IAST [baseline] (30.988 ms) : 0, 30988
IAST [candidate] (28.667 ms) : 0, 28667
section profiling
crashtracking [baseline] (1.429 ms) : 0, 1429
crashtracking [candidate] (1.463 ms) : 0, 1463
BytebuddyAgent [baseline] (761.859 ms) : 0, 761859
BytebuddyAgent [candidate] (771.738 ms) : 0, 771738
GlobalTracer [baseline] (222.279 ms) : 0, 222279
GlobalTracer [candidate] (223.648 ms) : 0, 223648
AppSec [baseline] (30.957 ms) : 0, 30957
AppSec [candidate] (31.161 ms) : 0, 31161
Debugger [baseline] (6.255 ms) : 0, 6255
Debugger [candidate] (6.337 ms) : 0, 6337
Remote Config [baseline] (691.259 µs) : 0, 691
Remote Config [candidate] (707.222 µs) : 0, 707
Telemetry [baseline] (15.451 ms) : 0, 15451
Telemetry [candidate] (15.488 ms) : 0, 15488
ProfilingAgent [baseline] (108.289 ms) : 0, 108289
ProfilingAgent [candidate] (107.998 ms) : 0, 107998
Profiling [baseline] (108.944 ms) : 0, 108944
Profiling [candidate] (108.636 ms) : 0, 108636

Startup time reports for insecure-bank

gantt
    title insecure-bank - global startup overhead: candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a

    dateFormat X
    axisFormat %s
section tracing
Agent [baseline] (1.045 s) : 0, 1044681
Total [baseline] (8.622 s) : 0, 8622303
Agent [candidate] (1.049 s) : 0, 1048794
Total [candidate] (8.647 s) : 0, 8647487
section iast
Agent [baseline] (1.187 s) : 0, 1186978
Total [baseline] (9.335 s) : 0, 9334835
Agent [candidate] (1.197 s) : 0, 1197160
Total [candidate] (9.334 s) : 0, 9334103

baseline results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.045 s	-
Agent	iast	1.187 s	142.298 ms (13.6%)
Total	tracing	8.622 s	-
Total	iast	9.335 s	712.532 ms (8.3%)

candidate results

Module	Variant	Duration	Δ tracing
Agent	tracing	1.049 s	-
Agent	iast	1.197 s	148.365 ms (14.1%)
Total	tracing	8.647 s	-
Total	iast	9.334 s	686.616 ms (7.9%)

gantt
    title insecure-bank - break down per module: candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a

    dateFormat X
    axisFormat %s
section tracing
crashtracking [baseline] (1.457 ms) : 0, 1457
crashtracking [candidate] (1.449 ms) : 0, 1449
BytebuddyAgent [baseline] (732.648 ms) : 0, 732648
BytebuddyAgent [candidate] (735.247 ms) : 0, 735247
GlobalTracer [baseline] (242.658 ms) : 0, 242658
GlobalTracer [candidate] (242.873 ms) : 0, 242873
AppSec [baseline] (30.143 ms) : 0, 30143
AppSec [candidate] (30.129 ms) : 0, 30129
Debugger [baseline] (6.059 ms) : 0, 6059
Debugger [candidate] (6.053 ms) : 0, 6053
Remote Config [baseline] (668.869 µs) : 0, 669
Remote Config [candidate] (664.892 µs) : 0, 665
Telemetry [baseline] (10.071 ms) : 0, 10071
Telemetry [candidate] (11.405 ms) : 0, 11405
section iast
crashtracking [baseline] (1.464 ms) : 0, 1464
crashtracking [candidate] (1.482 ms) : 0, 1482
BytebuddyAgent [baseline] (857.403 ms) : 0, 857403
BytebuddyAgent [candidate] (867.381 ms) : 0, 867381
GlobalTracer [baseline] (233.975 ms) : 0, 233975
GlobalTracer [candidate] (234.114 ms) : 0, 234114
AppSec [baseline] (30.355 ms) : 0, 30355
AppSec [candidate] (28.381 ms) : 0, 28381
Debugger [baseline] (5.759 ms) : 0, 5759
Debugger [candidate] (6.67 ms) : 0, 6670
Remote Config [baseline] (599.192 µs) : 0, 599
Remote Config [candidate] (614.097 µs) : 0, 614
Telemetry [baseline] (8.431 ms) : 0, 8431
Telemetry [candidate] (8.447 ms) : 0, 8447
IAST [baseline] (27.867 ms) : 0, 27867
IAST [candidate] (28.803 ms) : 0, 28803

Load

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/coverage-reports-upload
git_commit_date	1756292489	1756298913
git_commit_sha	`8799a82`	`c0d713c`
release_version	1.53.0-SNAPSHOT~8799a82b0a	1.53.0-SNAPSHOT~c0d713c226

See matching parameters

	Baseline	Candidate
application	insecure-bank	insecure-bank
ci_job_date	1756300351	1756300351
ci_job_id	1101084367	1101084367
ci_pipeline_id	74846592	74846592
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-i1wtri04 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-i1wtri04 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 1 performance improvements and 2 performance regressions! Performance is the same for 9 metrics, 12 unstable metrics.

scenario	Δ mean http_req_duration	Δ mean throughput	candidate mean http_req_duration	candidate mean throughput	baseline mean http_req_duration	baseline mean throughput
scenario:load:insecure-bank:profiling:high_load	better [-613.995µs; -307.033µs] or [-6.702%; -3.352%]	unstable [-37.874op/s; +90.562op/s] or [-7.476%; +17.875%]	8.700ms	532.969op/s	9.161ms	506.625op/s
scenario:load:petclinic:tracing:high_load	worse [+1.951ms; +2.754ms] or [+4.476%; +6.320%]	unstable [-12.954op/s; +1.854op/s] or [-12.062%; +1.726%]	45.931ms	101.850op/s	43.579ms	107.400op/s
scenario:load:petclinic:profiling:high_load	worse [+1.347ms; +2.415ms] or [+2.787%; +4.997%]	unstable [-10.755op/s; +3.505op/s] or [-11.107%; +3.620%]	50.210ms	93.213op/s	48.329ms	96.838op/s

Request duration reports for insecure-bank

gantt
    title insecure-bank - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a
    dateFormat X
    axisFormat %s
section baseline
no_agent (4.503 ms) : 4453, 4554
.   : milestone, 4503,
iast (9.249 ms) : 9099, 9399
.   : milestone, 9249,
iast_FULL (14.178 ms) : 13892, 14463
.   : milestone, 14178,
iast_GLOBAL (10.34 ms) : 10159, 10521
.   : milestone, 10340,
profiling (9.161 ms) : 9015, 9307
.   : milestone, 9161,
tracing (7.649 ms) : 7542, 7756
.   : milestone, 7649,
section candidate
no_agent (4.418 ms) : 4362, 4473
.   : milestone, 4418,
iast (9.486 ms) : 9333, 9639
.   : milestone, 9486,
iast_FULL (13.838 ms) : 13558, 14118
.   : milestone, 13838,
iast_GLOBAL (10.573 ms) : 10388, 10758
.   : milestone, 10573,
profiling (8.7 ms) : 8561, 8840
.   : milestone, 8700,
tracing (7.609 ms) : 7493, 7725
.   : milestone, 7609,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.503 ms [4.453 ms, 4.554 ms]	-
iast	9.249 ms [9.099 ms, 9.399 ms]	4.745 ms (105.4%)
iast_FULL	14.178 ms [13.892 ms, 14.463 ms]	9.674 ms (214.8%)
iast_GLOBAL	10.34 ms [10.159 ms, 10.521 ms]	5.836 ms (129.6%)
profiling	9.161 ms [9.015 ms, 9.307 ms]	4.657 ms (103.4%)
tracing	7.649 ms [7.542 ms, 7.756 ms]	3.145 ms (69.8%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	4.418 ms [4.362 ms, 4.473 ms]	-
iast	9.486 ms [9.333 ms, 9.639 ms]	5.068 ms (114.7%)
iast_FULL	13.838 ms [13.558 ms, 14.118 ms]	9.42 ms (213.2%)
iast_GLOBAL	10.573 ms [10.388 ms, 10.758 ms]	6.155 ms (139.3%)
profiling	8.7 ms [8.561 ms, 8.84 ms]	4.283 ms (96.9%)
tracing	7.609 ms [7.493 ms, 7.725 ms]	3.191 ms (72.2%)

Request duration reports for petclinic

gantt
    title petclinic - request duration [CI 0.99] : candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a
    dateFormat X
    axisFormat %s
section baseline
no_agent (37.477 ms) : 37179, 37775
.   : milestone, 37477,
appsec (47.035 ms) : 46611, 47459
.   : milestone, 47035,
code_origins (44.021 ms) : 43642, 44400
.   : milestone, 44021,
iast (45.879 ms) : 45482, 46275
.   : milestone, 45879,
profiling (48.329 ms) : 47867, 48792
.   : milestone, 48329,
tracing (43.579 ms) : 43212, 43946
.   : milestone, 43579,
section candidate
no_agent (38.295 ms) : 37989, 38602
.   : milestone, 38295,
appsec (48.156 ms) : 47713, 48598
.   : milestone, 48156,
code_origins (44.892 ms) : 44506, 45278
.   : milestone, 44892,
iast (45.723 ms) : 45320, 46126
.   : milestone, 45723,
profiling (50.21 ms) : 49682, 50738
.   : milestone, 50210,
tracing (45.931 ms) : 45552, 46310
.   : milestone, 45931,

baseline results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	37.477 ms [37.179 ms, 37.775 ms]	-
appsec	47.035 ms [46.611 ms, 47.459 ms]	9.558 ms (25.5%)
code_origins	44.021 ms [43.642 ms, 44.4 ms]	6.544 ms (17.5%)
iast	45.879 ms [45.482 ms, 46.275 ms]	8.402 ms (22.4%)
profiling	48.329 ms [47.867 ms, 48.792 ms]	10.852 ms (29.0%)
tracing	43.579 ms [43.212 ms, 43.946 ms]	6.102 ms (16.3%)

candidate results

Variant	Request duration [CI 0.99]	Δ no_agent
no_agent	38.295 ms [37.989 ms, 38.602 ms]	-
appsec	48.156 ms [47.713 ms, 48.598 ms]	9.861 ms (25.7%)
code_origins	44.892 ms [44.506 ms, 45.278 ms]	6.596 ms (17.2%)
iast	45.723 ms [45.32 ms, 46.126 ms]	7.427 ms (19.4%)
profiling	50.21 ms [49.682 ms, 50.738 ms]	11.915 ms (31.1%)
tracing	45.931 ms [45.552 ms, 46.31 ms]	7.636 ms (19.9%)

Dacapo

Parameters

	Baseline	Candidate
baseline_or_candidate	baseline	candidate
git_branch	master	nikita-tkachenko/coverage-reports-upload
git_commit_date	1756292489	1756298913
git_commit_sha	`8799a82`	`c0d713c`
release_version	1.53.0-SNAPSHOT~8799a82b0a	1.53.0-SNAPSHOT~c0d713c226

See matching parameters

	Baseline	Candidate
application	biojava	biojava
ci_job_date	1756300943	1756300943
ci_job_id	1101084368	1101084368
ci_pipeline_id	74846592	74846592
cpu_model	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz	Intel(R) Xeon(R) Platinum 8259CL CPU @ 2.50GHz
kernel_version	Linux runner-zfyrx7zua-project-304-concurrent-1-96645j4o 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux	Linux runner-zfyrx7zua-project-304-concurrent-1-96645j4o 6.8.0-1031-aws #33~22.04.1-Ubuntu SMP Thu Jun 26 14:22:30 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Summary

Found 0 performance improvements and 0 performance regressions! Performance is the same for 10 metrics, 2 unstable metrics.

Execution time for tomcat

gantt
    title tomcat - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a
    dateFormat X
    axisFormat %s
section baseline
no_agent (1.471 ms) : 1460, 1483
.   : milestone, 1471,
appsec (3.669 ms) : 3452, 3886
.   : milestone, 3669,
iast (2.201 ms) : 2138, 2265
.   : milestone, 2201,
iast_GLOBAL (2.23 ms) : 2168, 2293
.   : milestone, 2230,
profiling (2.47 ms) : 2307, 2632
.   : milestone, 2470,
tracing (2.017 ms) : 1968, 2065
.   : milestone, 2017,
section candidate
no_agent (1.481 ms) : 1469, 1492
.   : milestone, 1481,
appsec (3.668 ms) : 3450, 3887
.   : milestone, 3668,
iast (2.196 ms) : 2133, 2259
.   : milestone, 2196,
iast_GLOBAL (2.234 ms) : 2172, 2297
.   : milestone, 2234,
profiling (2.025 ms) : 1975, 2075
.   : milestone, 2025,
tracing (2.004 ms) : 1956, 2053
.   : milestone, 2004,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.471 ms [1.46 ms, 1.483 ms]	-
appsec	3.669 ms [3.452 ms, 3.886 ms]	2.198 ms (149.4%)
iast	2.201 ms [2.138 ms, 2.265 ms]	730.086 µs (49.6%)
iast_GLOBAL	2.23 ms [2.168 ms, 2.293 ms]	759.228 µs (51.6%)
profiling	2.47 ms [2.307 ms, 2.632 ms]	998.698 µs (67.9%)
tracing	2.017 ms [1.968 ms, 2.065 ms]	545.595 µs (37.1%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	1.481 ms [1.469 ms, 1.492 ms]	-
appsec	3.668 ms [3.45 ms, 3.887 ms]	2.188 ms (147.8%)
iast	2.196 ms [2.133 ms, 2.259 ms]	715.367 µs (48.3%)
iast_GLOBAL	2.234 ms [2.172 ms, 2.297 ms]	753.776 µs (50.9%)
profiling	2.025 ms [1.975 ms, 2.075 ms]	544.782 µs (36.8%)
tracing	2.004 ms [1.956 ms, 2.053 ms]	523.783 µs (35.4%)

Execution time for biojava

gantt
    title biojava - execution time [CI 0.99] : candidate=1.53.0-SNAPSHOT~c0d713c226, baseline=1.53.0-SNAPSHOT~8799a82b0a
    dateFormat X
    axisFormat %s
section baseline
no_agent (15.027 s) : 15027000, 15027000
.   : milestone, 15027000,
appsec (14.726 s) : 14726000, 14726000
.   : milestone, 14726000,
iast (18.687 s) : 18687000, 18687000
.   : milestone, 18687000,
iast_GLOBAL (18.034 s) : 18034000, 18034000
.   : milestone, 18034000,
profiling (15.405 s) : 15405000, 15405000
.   : milestone, 15405000,
tracing (14.882 s) : 14882000, 14882000
.   : milestone, 14882000,
section candidate
no_agent (15.693 s) : 15693000, 15693000
.   : milestone, 15693000,
appsec (14.893 s) : 14893000, 14893000
.   : milestone, 14893000,
iast (18.85 s) : 18850000, 18850000
.   : milestone, 18850000,
iast_GLOBAL (18.053 s) : 18053000, 18053000
.   : milestone, 18053000,
profiling (15.301 s) : 15301000, 15301000
.   : milestone, 15301000,
tracing (14.969 s) : 14969000, 14969000
.   : milestone, 14969000,

baseline results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.027 s [15.027 s, 15.027 s]	-
appsec	14.726 s [14.726 s, 14.726 s]	-301.0 ms (-2.0%)
iast	18.687 s [18.687 s, 18.687 s]	3.66 s (24.4%)
iast_GLOBAL	18.034 s [18.034 s, 18.034 s]	3.007 s (20.0%)
profiling	15.405 s [15.405 s, 15.405 s]	378.0 ms (2.5%)
tracing	14.882 s [14.882 s, 14.882 s]	-145.0 ms (-1.0%)

candidate results

Variant	Execution Time [CI 0.99]	Δ no_agent
no_agent	15.693 s [15.693 s, 15.693 s]	-
appsec	14.893 s [14.893 s, 14.893 s]	-800.0 ms (-5.1%)
iast	18.85 s [18.85 s, 18.85 s]	3.157 s (20.1%)
iast_GLOBAL	18.053 s [18.053 s, 18.053 s]	2.36 s (15.0%)
profiling	15.301 s [15.301 s, 15.301 s]	-392.0 ms (-2.5%)
tracing	14.969 s [14.969 s, 14.969 s]	-724.0 ms (-4.6%)

github-actions · 2025-08-28T13:10:56Z

Hi! 👋 Thanks for your pull request! 🎉

To help us review it, please make sure to:

Add at least one type, and one component or instrumentation label to the pull request

If you need help, please check our contributing guidelines.

| Package | Type | Package file | Manager | Update | Change | |---|---|---|---|---|---| | [com.datadoghq:dd-trace-api](https://github.com/datadog/dd-trace-java) | dependencies | misk/gradle/libs.versions.toml | gradle | minor | `1.52.1` -> `1.53.0` | --- ### Release Notes <details> <summary>datadog/dd-trace-java (com.datadoghq:dd-trace-api)</summary> ### [`v1.53.0`](https://github.com/DataDog/dd-trace-java/releases/tag/v1.53.0): 1.53.0 ### Components #### Application Security Management (WAF) - 🐛 Fix NullPointerException log in AppSec ([#9355](DataDog/dd-trace-java#9355) - [@jandro996](https://github.com/jandro996)) - ✨ Add Forwarded header as a source to client IP resolution ([#9310](DataDog/dd-trace-java#9310) - [@jandro996](https://github.com/jandro996)) #### Build & Tooling - 📖 Update CODEOWNERS ([#9294](DataDog/dd-trace-java#9294) - [@mhlidd](https://github.com/mhlidd)) - ✨ Add dependency on Maven Central deployment to OCI publish jobs ([#9204](DataDog/dd-trace-java#9204) - [@sarahchen6](https://github.com/sarahchen6)) #### Continuous Integration Visibility - ✨ Upload code coverage reports to Datadog ([#9425](DataDog/dd-trace-java#9425) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨⚡ Do not follow symlinks by default when building repository index ([#9318](DataDog/dd-trace-java#9318) - [@nikita-tkachenko-datadog](https://github.com/nikita-tkachenko-datadog)) - ✨ Improve tag propagation between test event levels ([#9278](DataDog/dd-trace-java#9278) - [@daniel-mohedano](https://github.com/daniel-mohedano)) #### Data Streams Monitoring - ✨ Create container hash tags propagation ([#9282](DataDog/dd-trace-java#9282) - [@ygree](https://github.com/ygree)) #### Database Monitoring - 🐛 Prevent crash in SQL Server's JDBC when tracing execute methods with generated keys ([#9321](DataDog/dd-trace-java#9321) - [@na-ji](https://github.com/na-ji)) - ✨ Create container hash tags propagation ([#9282](DataDog/dd-trace-java#9282) - [@ygree](https://github.com/ygree)) #### GraalVM native-image - 🐛 Update GraalVM config to reflect TempLocationManager's new package ([#9337](DataDog/dd-trace-java#9337) - [@luneo7](https://github.com/luneo7) - thanks for the contribution!) #### Metrics - 🐛 Add reflect config for CSS and graalvm ([#9449](DataDog/dd-trace-java#9449) - [@amarziali](https://github.com/amarziali)) - ✨ Align CSS health metrics ([#9444](DataDog/dd-trace-java#9444) - [@amarziali](https://github.com/amarziali)) - ✨ Aggregate peer tags for consumer span kind ([#9442](DataDog/dd-trace-java#9442) - [@amarziali](https://github.com/amarziali)) - ✨ Implement health metrics for client stats ([#9377](DataDog/dd-trace-java#9377) - [@amarziali](https://github.com/amarziali)) - ✨ Add peer tags, span kind and trace root flag to MetricKey bucket ([#9178](DataDog/dd-trace-java#9178) - [@bric3](https://github.com/bric3)) #### Profiling - ✨ Create a profile flare reporter (requires tracing or CI vis to be enabled) ([#9433](DataDog/dd-trace-java#9433) - [@MattAlp](https://github.com/MattAlp)) - 🐛 Properly fall-back on UDS profiling URL ([#9401](DataDog/dd-trace-java#9401) - [@jbachorik](https://github.com/jbachorik)) - 🐛 Properly handle trace agent IPv6 URL in profiling ([#9334](DataDog/dd-trace-java#9334) - [@jbachorik](https://github.com/jbachorik)) #### Realtime User Monitoring - 🐛 Avoid multiple injections on dispatch on jboss ([#9392](DataDog/dd-trace-java#9392) - [@amarziali](https://github.com/amarziali)) - ✨ Support async servlet for RUM injection ([#9333](DataDog/dd-trace-java#9333) - [@amarziali](https://github.com/amarziali)) - 🐛 Improve RUM injection matching and avoid truncating responses ([#9323](DataDog/dd-trace-java#9323) - [@amarziali](https://github.com/amarziali)) - 💡 Add telemetry for the RUM injector ([#9267](DataDog/dd-trace-java#9267) - [@sarahchen6](https://github.com/sarahchen6)) - ✨ Make rum injector stream/writer more resilient to errors ([#9184](DataDog/dd-trace-java#9184) - [@amarziali](https://github.com/amarziali)) #### Telemetry - 🐛 Remediate error logs ([#9459](DataDog/dd-trace-java#9459) - [@ygree](https://github.com/ygree)) - ✨ Report config\_id for Hands Off Config files ([#9299](DataDog/dd-trace-java#9299) - [@mtoffl01](https://github.com/mtoffl01)) - ✨ Adding Config Inversion Telemetry component ([#9244](DataDog/dd-trace-java#9244) - [@mhlidd](https://github.com/mhlidd)) #### Trace context propagation - 🧹 Migrate instrumentations to Context API ([#9384](DataDog/dd-trace-java#9384), [#9378](DataDog/dd-trace-java#9378), [#9358](DataDog/dd-trace-java#9358) - [@PerfectSlayer](https://github.com/PerfectSlayer)) - ✨🔍 Add baggage propagation telemetry ([#9289](DataDog/dd-trace-java#9289) - [@rachelyangdog](https://github.com/rachelyangdog)) - 🧹 Improve Instrumenter API to use Context instead of Span ([#9211](DataDog/dd-trace-java#9211) - [@PerfectSlayer](https://github.com/PerfectSlayer)) #### Tracer core - 💡:test\_tube: Enable process level tags for java 21 applications ([#9458](DataDog/dd-trace-java#9458) - [@amarziali](https://github.com/amarziali)) - ✨ Move base and peer service adder before span is published ([#9408](DataDog/dd-trace-java#9408) - [@amarziali](https://github.com/amarziali)) - ✨ Add ErrorPriorities level for manual instrumentation ([#9387](DataDog/dd-trace-java#9387) - [@cecile75](https://github.com/cecile75)) - ✨ Throw InvalidBooleanValueException in ConfigConverter.booleanValueOf ([#9370](DataDog/dd-trace-java#9370) - [@mtoffl01](https://github.com/mtoffl01)) - 🐛 Avoid NPE on featureDiscovery creation ([#9353](DataDog/dd-trace-java#9353) - [@amarziali](https://github.com/amarziali)) - ✨ Stable Config improvements ([#9259](DataDog/dd-trace-java#9259) - [@mtoffl01](https://github.com/mtoffl01)) ### Instrumentations #### gRPC instrumentation - 🐛 Changing GRPC instrumentation codes from `status.code` to `grpc.status.code` ([#9367](DataDog/dd-trace-java#9367) - [@mhlidd](https://github.com/mhlidd)) #### JDBC instrumentation - 🐛 Prevent crash in SQL Server's JDBC when tracing execute methods with generated keys ([#9321](DataDog/dd-trace-java#9321) - [@na-ji](https://github.com/na-ji)) #### Kotlin instrumentation - 🐛 Take defensive copy of original scope stack when restoring on different thread ([#9403](DataDog/dd-trace-java#9403) - [@mcculls](https://github.com/mcculls)) #### OpenTelemetry instrumentation - 🐛 Fix using SpanAttribute annotation on multiple parameters ([#9412](DataDog/dd-trace-java#9412) - [@cecile75](https://github.com/cecile75)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - "after 6pm every weekday,before 2am every weekday" in timezone Australia/Melbourne, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Never, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). GitOrigin-RevId: e23ec866cef91afc6b5226d5f0dc7da538d947e8

## Description This PR implements code coverage report upload functionality for pytest v3 plugin (`ddtrace/testing`), enabling automatic upload of LCOV coverage reports to Datadog's Code Coverage backend. This feature simplifies onboarding to Code Coverage for existing Test Optimization customers by automatically collecting and uploading coverage data during test execution. The implementation follows the approach established in the Java tracer (PR [#9425](DataDog/dd-trace-java#9425)) and includes: - **Coverage.py Integration**: Enhanced patching and wrapper system for coverage.py to capture coverage percentages and generate LCOV reports - **Report Generation**: Support for pytest-cov with LCOV report generation - **Upload Functionality**: Gzipped LCOV report upload to intake (`/api/v2/cicovreprt` endpoint), with proper event metadata - **Configuration Support**: Backend setting `coverage_report_upload_enabled` with environment variable `DD_CIVISIBILITY_CODE_COVERAGE_REPORT_UPLOAD_ENABLED` override - **Git/CI Metadata**: Automatic inclusion of repository, commit, branch, and CI provider tags in coverage reports ## Testing Comprehensive test suite covering multiple layers: ### Unit Tests - **Coverage.py Integration** (`tests/testing/internal/coverage/test_patch.py`): - Coverage lifecycle management (start/stop/erase) - LCOV report generation with percentage caching - Patching safety (double patch/unpatch scenarios) - Error handling for invalid paths and missing coverage data - Custom coverage parameters and instance management ### Integration Tests - **End-to-End Upload** (`tests/testing/internal/pytest/test_pytest_coverage_report_upload.py`): - Upload with pytest-cov and ModuleCodeCollector - LCOV format validation and gzip compression verification - Git metadata inclusion in event payloads - Upload failure resilience ### Configuration Edge Cases - **Settings Precedence**: Backend settings vs environment variable interactions - **Custom Coverage Configs**: Support for .coveragerc files with branch coverage - **Multiple Source Paths**: Coverage collection across different directories ### API Client Tests - **Upload Mechanism** (`tests/testing/internal/test_api_client.py`): - Multipart form-data structure validation - CI tag propagation and telemetry recording - HTTP error handling and retry logic ## Risks **Low Risk**: - Feature is controlled by backend setting (disabled by default) - Environment variable override provides user control - Upload failures don't affect test execution (fail-safe design) - Coverage collection uses existing ddtrace mechanisms, and can only be triggered if `coverage.py` is present. - Extensive error handling prevents crashes **Potential Considerations**: - Additional I/O during test runs for LCOV generation and upload - Network dependency for upload functionality (mitigated by error handling) - Coverage.py version compatibility (tested with standard versions) ## Additional Notes - **Backward Compatibility**: All coverage.py integration maintains existing behavior when upload is disabled - **Performance**: Minimal overhead during test execution; coverage generation happens at session end - **Format Support**: Currently supports LCOV format, extensible for future formats

nikita-tkachenko-datadog added 11 commits August 27, 2025 12:55

Upload code coverage reports from tracer

78dff5b

Add a backend setting for uploading coverage reports

b37c9cb

Automatically inject Jacoco if code coverage report upload is enabled

851f760

Add code_coverage.report_upload capability

b18173a

Add missing type field to cov report events

9431fe4

Add report upload capability to test fixtures

cb9fc88

Suppress SpotBugs warning

97b264b

Fix LCOV report generation to include uncovered executable lines

e79d6d6

Remove comment

2843e75

Update capability name

5dd3465

Fix unit test

8093ae4

nikita-tkachenko-datadog force-pushed the nikita-tkachenko/coverage-reports-upload branch from 2e80687 to 8093ae4 Compare August 27, 2025 10:55

nikita-tkachenko-datadog added 2 commits August 27, 2025 13:52

Fix codeNarc violations

2d91613

Codenarc

c0d713c

nikita-tkachenko-datadog marked this pull request as ready for review August 28, 2025 13:10

nikita-tkachenko-datadog requested review from a team as code owners August 28, 2025 13:10

nikita-tkachenko-datadog requested review from Mariovido and smola August 28, 2025 13:10

nikita-tkachenko-datadog added type: enhancement Enhancements and improvements comp: ci visibility Continuous Integration Visibility labels Aug 28, 2025

daniel-mohedano approved these changes Aug 28, 2025

View reviewed changes

nikita-tkachenko-datadog merged commit ab302c3 into master Aug 29, 2025
503 of 504 checks passed

nikita-tkachenko-datadog deleted the nikita-tkachenko/coverage-reports-upload branch August 29, 2025 13:54

github-actions bot added this to the 1.53.0 milestone Aug 29, 2025

nikita-tkachenko-datadog mentioned this pull request Sep 1, 2025

Update Maven smoke test fixtures with coverage report upload capability #9446

Merged

nikita-tkachenko-datadog mentioned this pull request Oct 3, 2025

Minor fixes for code coverage report upload #9659

Merged

juan-fernandez mentioned this pull request Jan 29, 2026

[test optimization][SDTEST-3004] Upload code coverage reports automatically DataDog/dd-trace-js#7335

Merged

gnufede mentioned this pull request Jan 29, 2026

chore(ci_visibility): code coverage report upload DataDog/dd-trace-py#16258

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upload code coverage reports to Datadog#9425

Upload code coverage reports to Datadog#9425
nikita-tkachenko-datadog merged 13 commits intomasterfrom
nikita-tkachenko/coverage-reports-upload

nikita-tkachenko-datadog commented Aug 26, 2025 •

edited by atlassian bot

Loading

Uh oh!

datadog-official bot commented Aug 26, 2025 •

edited

Loading

Uh oh!

pr-commenter bot commented Aug 26, 2025 •

edited

Loading

Uh oh!

github-actions bot commented Aug 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

nikita-tkachenko-datadog commented Aug 26, 2025 • edited by atlassian bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What Does This Do

Motivation

Contributor Checklist

Uh oh!

datadog-official bot commented Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pr-commenter bot commented Aug 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Startup

Parameters

Summary

Load

Parameters

Summary

Dacapo

Parameters

Summary

Uh oh!

github-actions bot commented Aug 28, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

nikita-tkachenko-datadog commented Aug 26, 2025 •

edited by atlassian bot

Loading

datadog-official bot commented Aug 26, 2025 •

edited

Loading

pr-commenter bot commented Aug 26, 2025 •

edited

Loading