Add OtlpWriter for OTLP traces export #11200
Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 19 commits intomasterfrom Apr 30, 2026
Merged
Add OtlpWriter for OTLP traces export #11200gh-worker-dd-mergequeue-cf854d[bot] merged 19 commits intomasterfrom
gh-worker-dd-mergequeue-cf854d[bot] merged 19 commits intomasterfrom
Conversation
BenchmarksStartupParameters
See matching parameters
SummaryFound 1 performance improvements and 0 performance regressions! Performance is the same for 62 metrics, 8 unstable metrics.
Startup time reports for insecure-bankgantt
title insecure-bank - global startup overhead: candidate=1.62.0-SNAPSHOT~96954d7d95, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.065 s) : 0, 1065416
Total [baseline] (8.869 s) : 0, 8868535
Agent [candidate] (1.063 s) : 0, 1062618
Total [candidate] (8.841 s) : 0, 8841085
section iast
Agent [baseline] (1.252 s) : 0, 1252425
Total [baseline] (9.572 s) : 0, 9572141
Agent [candidate] (1.244 s) : 0, 1243831
Total [candidate] (9.508 s) : 0, 9508034
gantt
title insecure-bank - break down per module: candidate=1.62.0-SNAPSHOT~96954d7d95, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.23 ms) : 0, 1230
crashtracking [candidate] (1.228 ms) : 0, 1228
BytebuddyAgent [baseline] (635.696 ms) : 0, 635696
BytebuddyAgent [candidate] (634.742 ms) : 0, 634742
AgentMeter [baseline] (29.506 ms) : 0, 29506
AgentMeter [candidate] (29.349 ms) : 0, 29349
GlobalTracer [baseline] (249.993 ms) : 0, 249993
GlobalTracer [candidate] (248.352 ms) : 0, 248352
AppSec [baseline] (33.154 ms) : 0, 33154
AppSec [candidate] (32.814 ms) : 0, 32814
Debugger [baseline] (60.175 ms) : 0, 60175
Debugger [candidate] (59.881 ms) : 0, 59881
Remote Config [baseline] (600.507 µs) : 0, 601
Remote Config [candidate] (599.823 µs) : 0, 600
Telemetry [baseline] (8.391 ms) : 0, 8391
Telemetry [candidate] (12.876 ms) : 0, 12876
Flare Poller [baseline] (10.586 ms) : 0, 10586
Flare Poller [candidate] (6.83 ms) : 0, 6830
section iast
crashtracking [baseline] (1.237 ms) : 0, 1237
crashtracking [candidate] (1.219 ms) : 0, 1219
BytebuddyAgent [baseline] (831.905 ms) : 0, 831905
BytebuddyAgent [candidate] (823.437 ms) : 0, 823437
AgentMeter [baseline] (11.383 ms) : 0, 11383
AgentMeter [candidate] (11.305 ms) : 0, 11305
GlobalTracer [baseline] (237.796 ms) : 0, 237796
GlobalTracer [candidate] (237.853 ms) : 0, 237853
IAST [baseline] (29.122 ms) : 0, 29122
IAST [candidate] (28.331 ms) : 0, 28331
AppSec [baseline] (30.542 ms) : 0, 30542
AppSec [candidate] (31.378 ms) : 0, 31378
Debugger [baseline] (62.432 ms) : 0, 62432
Debugger [candidate] (62.502 ms) : 0, 62502
Remote Config [baseline] (544.066 µs) : 0, 544
Remote Config [candidate] (519.849 µs) : 0, 520
Telemetry [baseline] (7.868 ms) : 0, 7868
Telemetry [candidate] (7.923 ms) : 0, 7923
Flare Poller [baseline] (3.365 ms) : 0, 3365
Flare Poller [candidate] (3.376 ms) : 0, 3376
Startup time reports for petclinicgantt
title petclinic - global startup overhead: candidate=1.62.0-SNAPSHOT~96954d7d95, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section tracing
Agent [baseline] (1.066 s) : 0, 1066074
Total [baseline] (11.121 s) : 0, 11121337
Agent [candidate] (1.065 s) : 0, 1065381
Total [candidate] (11.021 s) : 0, 11020853
section appsec
Agent [baseline] (1.267 s) : 0, 1266976
Total [baseline] (11.03 s) : 0, 11029960
Agent [candidate] (1.266 s) : 0, 1265815
Total [candidate] (11.092 s) : 0, 11091834
section iast
Agent [baseline] (1.246 s) : 0, 1246261
Total [baseline] (11.27 s) : 0, 11270261
Agent [candidate] (1.246 s) : 0, 1246115
Total [candidate] (11.304 s) : 0, 11303950
section profiling
Agent [baseline] (1.191 s) : 0, 1190647
Total [baseline] (11.032 s) : 0, 11031617
Agent [candidate] (1.195 s) : 0, 1194620
Total [candidate] (10.998 s) : 0, 10998036
gantt
title petclinic - break down per module: candidate=1.62.0-SNAPSHOT~96954d7d95, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section tracing
crashtracking [baseline] (1.232 ms) : 0, 1232
crashtracking [candidate] (1.229 ms) : 0, 1229
BytebuddyAgent [baseline] (637.078 ms) : 0, 637078
BytebuddyAgent [candidate] (636.015 ms) : 0, 636015
AgentMeter [baseline] (29.569 ms) : 0, 29569
AgentMeter [candidate] (29.53 ms) : 0, 29530
GlobalTracer [baseline] (249.512 ms) : 0, 249512
GlobalTracer [candidate] (248.607 ms) : 0, 248607
AppSec [baseline] (32.778 ms) : 0, 32778
AppSec [candidate] (32.823 ms) : 0, 32823
Debugger [baseline] (60.861 ms) : 0, 60861
Debugger [candidate] (60.781 ms) : 0, 60781
Remote Config [baseline] (607.716 µs) : 0, 608
Remote Config [candidate] (603.551 µs) : 0, 604
Telemetry [baseline] (8.459 ms) : 0, 8459
Telemetry [candidate] (8.395 ms) : 0, 8395
Flare Poller [baseline] (9.898 ms) : 0, 9898
Flare Poller [candidate] (11.402 ms) : 0, 11402
section appsec
crashtracking [baseline] (1.238 ms) : 0, 1238
crashtracking [candidate] (1.234 ms) : 0, 1234
BytebuddyAgent [baseline] (675.668 ms) : 0, 675668
BytebuddyAgent [candidate] (675.378 ms) : 0, 675378
AgentMeter [baseline] (12.305 ms) : 0, 12305
AgentMeter [candidate] (12.207 ms) : 0, 12207
GlobalTracer [baseline] (249.91 ms) : 0, 249910
GlobalTracer [candidate] (249.37 ms) : 0, 249370
IAST [baseline] (24.849 ms) : 0, 24849
IAST [candidate] (24.659 ms) : 0, 24659
AppSec [baseline] (186.048 ms) : 0, 186048
AppSec [candidate] (185.115 ms) : 0, 185115
Debugger [baseline] (65.17 ms) : 0, 65170
Debugger [candidate] (64.687 ms) : 0, 64687
Remote Config [baseline] (560.943 µs) : 0, 561
Remote Config [candidate] (556.645 µs) : 0, 557
Telemetry [baseline] (7.89 ms) : 0, 7890
Telemetry [candidate] (7.692 ms) : 0, 7692
Flare Poller [baseline] (6.758 ms) : 0, 6758
Flare Poller [candidate] (6.698 ms) : 0, 6698
section iast
crashtracking [baseline] (1.23 ms) : 0, 1230
crashtracking [candidate] (1.219 ms) : 0, 1219
BytebuddyAgent [baseline] (824.746 ms) : 0, 824746
BytebuddyAgent [candidate] (823.326 ms) : 0, 823326
AgentMeter [baseline] (11.331 ms) : 0, 11331
AgentMeter [candidate] (11.277 ms) : 0, 11277
GlobalTracer [baseline] (237.852 ms) : 0, 237852
GlobalTracer [candidate] (238.671 ms) : 0, 238671
IAST [baseline] (29.148 ms) : 0, 29148
IAST [candidate] (31.784 ms) : 0, 31784
AppSec [baseline] (30.448 ms) : 0, 30448
AppSec [candidate] (28.469 ms) : 0, 28469
Debugger [baseline] (63.265 ms) : 0, 63265
Debugger [candidate] (63.401 ms) : 0, 63401
Remote Config [baseline] (523.955 µs) : 0, 524
Remote Config [candidate] (535.83 µs) : 0, 536
Telemetry [baseline] (8.111 ms) : 0, 8111
Telemetry [candidate] (8.002 ms) : 0, 8002
Flare Poller [baseline] (3.476 ms) : 0, 3476
Flare Poller [candidate] (3.449 ms) : 0, 3449
section profiling
crashtracking [baseline] (1.19 ms) : 0, 1190
crashtracking [candidate] (1.184 ms) : 0, 1184
BytebuddyAgent [baseline] (694.309 ms) : 0, 694309
BytebuddyAgent [candidate] (697.518 ms) : 0, 697518
AgentMeter [baseline] (8.932 ms) : 0, 8932
AgentMeter [candidate] (9.019 ms) : 0, 9019
GlobalTracer [baseline] (208.868 ms) : 0, 208868
GlobalTracer [candidate] (209.668 ms) : 0, 209668
AppSec [baseline] (32.856 ms) : 0, 32856
AppSec [candidate] (33.183 ms) : 0, 33183
Debugger [baseline] (66.228 ms) : 0, 66228
Debugger [candidate] (65.984 ms) : 0, 65984
Remote Config [baseline] (585.5 µs) : 0, 585
Remote Config [candidate] (578.476 µs) : 0, 578
Telemetry [baseline] (8.139 ms) : 0, 8139
Telemetry [candidate] (8.259 ms) : 0, 8259
Flare Poller [baseline] (3.576 ms) : 0, 3576
Flare Poller [candidate] (3.537 ms) : 0, 3537
ProfilingAgent [baseline] (94.365 ms) : 0, 94365
ProfilingAgent [candidate] (94.093 ms) : 0, 94093
Profiling [baseline] (94.923 ms) : 0, 94923
Profiling [candidate] (94.649 ms) : 0, 94649
LoadParameters
See matching parameters
SummaryFound 2 performance improvements and 0 performance regressions! Performance is the same for 17 metrics, 17 unstable metrics.
Request duration reports for petclinicgantt
title petclinic - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~96954d7d95, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section baseline
no_agent (17.921 ms) : 17738, 18104
. : milestone, 17921,
appsec (18.75 ms) : 18560, 18939
. : milestone, 18750,
code_origins (18.009 ms) : 17832, 18186
. : milestone, 18009,
iast (17.91 ms) : 17732, 18088
. : milestone, 17910,
profiling (18.859 ms) : 18669, 19050
. : milestone, 18859,
tracing (18.307 ms) : 18126, 18489
. : milestone, 18307,
section candidate
no_agent (18.148 ms) : 17961, 18336
. : milestone, 18148,
appsec (18.54 ms) : 18355, 18726
. : milestone, 18540,
code_origins (17.949 ms) : 17775, 18124
. : milestone, 17949,
iast (18.141 ms) : 17962, 18320
. : milestone, 18141,
profiling (18.402 ms) : 18217, 18587
. : milestone, 18402,
tracing (18.372 ms) : 18187, 18557
. : milestone, 18372,
Request duration reports for insecure-bankgantt
title insecure-bank - request duration [CI 0.99] : candidate=1.62.0-SNAPSHOT~96954d7d95, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section baseline
no_agent (1.252 ms) : 1240, 1264
. : milestone, 1252,
iast (3.385 ms) : 3344, 3426
. : milestone, 3385,
iast_FULL (5.921 ms) : 5862, 5980
. : milestone, 5921,
iast_GLOBAL (3.838 ms) : 3772, 3903
. : milestone, 3838,
profiling (2.057 ms) : 2039, 2074
. : milestone, 2057,
tracing (1.962 ms) : 1945, 1978
. : milestone, 1962,
section candidate
no_agent (1.268 ms) : 1255, 1280
. : milestone, 1268,
iast (3.371 ms) : 3326, 3416
. : milestone, 3371,
iast_FULL (5.991 ms) : 5931, 6052
. : milestone, 5991,
iast_GLOBAL (3.637 ms) : 3580, 3694
. : milestone, 3637,
profiling (2.197 ms) : 2174, 2220
. : milestone, 2197,
tracing (1.871 ms) : 1856, 1887
. : milestone, 1871,
DacapoParameters
See matching parameters
SummaryFound 0 performance improvements and 0 performance regressions! Performance is the same for 11 metrics, 1 unstable metrics. Execution time for tomcatgantt
title tomcat - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~96954d7d95, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section baseline
no_agent (1.483 ms) : 1471, 1494
. : milestone, 1483,
appsec (3.823 ms) : 3602, 4045
. : milestone, 3823,
iast (2.269 ms) : 2199, 2339
. : milestone, 2269,
iast_GLOBAL (2.311 ms) : 2241, 2381
. : milestone, 2311,
profiling (2.093 ms) : 2038, 2148
. : milestone, 2093,
tracing (2.074 ms) : 2020, 2127
. : milestone, 2074,
section candidate
no_agent (1.487 ms) : 1475, 1498
. : milestone, 1487,
appsec (3.835 ms) : 3613, 4058
. : milestone, 3835,
iast (2.269 ms) : 2199, 2339
. : milestone, 2269,
iast_GLOBAL (2.314 ms) : 2244, 2384
. : milestone, 2314,
profiling (2.096 ms) : 2042, 2151
. : milestone, 2096,
tracing (2.07 ms) : 2016, 2123
. : milestone, 2070,
Execution time for biojavagantt
title biojava - execution time [CI 0.99] : candidate=1.62.0-SNAPSHOT~96954d7d95, baseline=1.62.0-SNAPSHOT~e6cac64dfd
dateFormat X
axisFormat %s
section baseline
no_agent (15.66 s) : 15660000, 15660000
. : milestone, 15660000,
appsec (14.607 s) : 14607000, 14607000
. : milestone, 14607000,
iast (18.429 s) : 18429000, 18429000
. : milestone, 18429000,
iast_GLOBAL (17.747 s) : 17747000, 17747000
. : milestone, 17747000,
profiling (15.633 s) : 15633000, 15633000
. : milestone, 15633000,
tracing (14.753 s) : 14753000, 14753000
. : milestone, 14753000,
section candidate
no_agent (14.928 s) : 14928000, 14928000
. : milestone, 14928000,
appsec (14.714 s) : 14714000, 14714000
. : milestone, 14714000,
iast (18.625 s) : 18625000, 18625000
. : milestone, 18625000,
iast_GLOBAL (17.868 s) : 17868000, 17868000
. : milestone, 17868000,
profiling (14.88 s) : 14880000, 14880000
. : milestone, 14880000,
tracing (14.71 s) : 14710000, 14710000
. : milestone, 14710000,
|
mcculls
reviewed
Apr 27, 2026
mtoffl01
commented
Apr 27, 2026
mtoffl01
commented
Apr 27, 2026
mtoffl01
commented
Apr 27, 2026
|
|
||
| @Override | ||
| public void onDroppedTrace(int spanCount) { | ||
| // TODO: surface drop counts via HealthMetrics |
Contributor
Author
There was a problem hiding this comment.
I've purposefully left this out as its not in the current spec
Contributor
|
Hi! 👋 Thanks for your pull request! 🎉 To help us review it, please make sure to:
If you need help, please check our contributing guidelines. |
mtoffl01
changed the title
mtoffl01
added
the
tag: ai generated
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4333c2d0ac
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
mtoffl01
added
type: enhancement
mtoffl01
changed the title
mcculls
reviewed
Apr 28, 2026
mcculls
reviewed
Apr 28, 2026
Contributor
mcculls
left a comment
mcculls
left a comment
There was a problem hiding this comment.
Just one Q about the extra sampling in OtlpPayloadDispatcher (if this is not needed then removing it may help with the E2E test) - and the GRPC signal path needs fixing.
…, plus related tests
mtoffl01
force-pushed
the
mtoff/otlp-writer
branch
from
f7b12c1 to
491533b
Compare
mcculls
reviewed
Apr 29, 2026
|
|
||
| private static void writeSpanTag( | ||
| StreamingBuffer buf, UTF8BytesString key, UTF8BytesString value) { | ||
| if (value == null) { |
Contributor
There was a problem hiding this comment.
This check should be moved to the appropriate caller of writeSpanTag - do you know which call is involved?
Contributor
There was a problem hiding this comment.
Contributor
Author
There was a problem hiding this comment.
I believe its on line 141, for span type:
writeSpanTag(buf, SPAN_TYPE, span.getSpanType());
Because spanType can be null. Should I change this to have an if span.getSpanType() != null prefix, and remove the null checks inside of the writeSpanTag functions?
Contributor
There was a problem hiding this comment.
Yes, thanks - it's the callers responsibility to do the check - that way it's clear in the code which tags are optional and which are mandatory - it also helps avoid masking issues such as when we expect a tag to always have a non-null value and for some reason it doesn't.
mcculls
reviewed
Apr 29, 2026
| } | ||
|
|
||
| private static void writeSpanTag(StreamingBuffer buf, UTF8BytesString key, CharSequence value) { | ||
| if (value == null) { |
Contributor
There was a problem hiding this comment.
This check should be moved to the appropriate caller of writeSpanTag - do you know which call is involved?
Contributor
|
The following files add Groovy tests to modules that are candidates for migration to Java / JUnit 5:
Consider writing these tests in Java / JUnit 5 instead to help with the ongoing migration effort. |
mcculls
approved these changes
Apr 29, 2026
Contributor
|
/merge |
|
View all feedbacks in Devflow UI.
The expected merge time in
|
github-merge-queue
Bot
removed this pull request from the merge queue due to failed status checks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What Does This Do
Adds a new
OtlpWriterthat sends Datadog traces via OTLP -- HTTP/protobuf (default) or gRPC -- to a collector endpoint. This new writer type is auto-selected whenOTEL_TRACES_EXPORTER=otlpis set and is otherwise off by default.New components:
OtlpWriter+OtlpPayloadDispatcherWriterFactorydriven byConfig.getOtlpTraces*()gettersConfigplumbing fortraceOtelExporter→OtlpWriterselectionMotivation
This PR builds on existing work that serializes Datadog spans in OTLP format by providing an "Otlp" writer mechanism for forwarding the data to a configurable endpoint.
Additional Notes
Much of the code was AI generated but based on a relatively detailed plan that I pre-approved before starting the code generation. This PR passes system tests: DataDog/system-tests#6819
Contributor Checklist
type:and (comp:orinst:) labels in addition to any other useful labelsclose,fix, or any linking keywords when referencing an issueUse
solvesinstead, and assign the PR milestone to the issueJira ticket: [PROJ-IDENT]
Note: Once your PR is ready to merge, add it to the merge queue by commenting
/merge./merge -ccancels the queue request./merge -f --reason "reason"skips all merge queue checks; please use this judiciously, as some checks do not run at the PR-level. For more information, see this doc.