feat(data-pipeline)!: Obfuscate v04 spans in agentless context - #2418
Open
paullegranddc wants to merge 16 commits into
Open
feat(data-pipeline)!: Obfuscate v04 spans in agentless context#2418paullegranddc wants to merge 16 commits into
paullegranddc wants to merge 16 commits into
Conversation
Contributor
Clippy Allow Annotation ReportTracked Clippy
By file and crateBy file
By crate
About This ReportThis report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. Panic-inducing macros in particular should be avoided. In the future, this report may become a PR-blocking quality gate. |
This comment has been minimized.
This comment has been minimized.
paullegranddc
marked this pull request as ready for review
August 25, 2026 23:47
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 637c473170
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
BenchmarksComparisonBenchmark execution time: 2026-08-27 16:03:18 Comparing candidate commit 3796c8b in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 153 metrics, 0 unstable metrics.
|
Collaborator
Author
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ad70e61397
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+644
to
+648
| handle.agentless_endpoint = match sanitize_string(url) { | ||
| Ok(s) => Some(s), | ||
| Err(e) => return Some(e), | ||
| }; | ||
| handle.agentless_api_key = match sanitize_string(api_key) { |
There was a problem hiding this comment.
When url is valid but api_key contains invalid UTF-8, this setter stores the new endpoint before returning an error for the key. Reusing the config can therefore enable agentless mode with an empty key, or, after reconfiguring an existing config, send the previous API key to the newly supplied endpoint. Sanitize both inputs into local values and only update the handle after both validations succeed.
Useful? React with 👍 / 👎.
Contributor
📚 Documentation Check Results📦
|
Contributor
🔒 Cargo Deny Results📦
|
ichinaski
reviewed
Aug 27, 2026
ichinaski
left a comment
ichinaski
left a comment
There was a problem hiding this comment.
It's somewhat unclear how much on parity this approach is with the Trace Agent obfuscation config. We should document this somewhere, and also test the schemas are the same.
I also don't fully understand why we want to keep 2 separate implementations here, one for protobuf, the other for raw spans (using JSON endpoint). These 2 paths look also inconsistent at first glance (pb implementation being more complex and more tested).
|
|
||
| /// Enables agentless APM trace export and sets the intake URL and API key. | ||
| /// | ||
| /// When set, APM trace spans are sent directly to the Datadog HTTP intake in JSON format |
There was a problem hiding this comment.
Why do we use the JSON endpoint when sending data to the Intake? Is this the recommended approach by the Intake team (instead of using the protobuf serialization)?
Collaborator
Author
There was a problem hiding this comment.
This is the the intake used by among others RUM to send traces from browser. It is already accepting trafic that doesn't go through the agent, which is why it was picked.
| #[cfg(feature = "agentless")] | ||
| { | ||
| // For agentless we want to tag top level spans, but not perform | ||
| // stats aggregation or span drops |
There was a problem hiding this comment.
Why doesn't agentless aggregate stats and drop spans? Is this intended to be implemented further down the line?
Collaborator
Author
There was a problem hiding this comment.
Yes, this is implemented in this PR (which does stats quantization and obfuscation unconditionaly since there is no /info) #2309
| "libdd-trace-stats/stats-obfuscation" | ||
| ] | ||
|
|
||
| agentless = ["stats-obfuscation", "libdd-trace-obfuscation", "https"] |
There was a problem hiding this comment.
This dependency is odd, since agentless can never activate CSS (it never reads /info endpoint from an agent). Why do we have stats-obfuscation here?
Collaborator
Author
There was a problem hiding this comment.
That's a leftover from a previous attempt at feature gating things. We want stats-obfuscation for this PR #2309 , but I'll add it there
| #[derive(Debug, Default, Deserialize)] | ||
| /// Mirrors the Datadog Agent defaults | ||
| /// see `pkg/config/schema/yaml/apm_config.yaml` | ||
| impl Default for MemcachedConfig { |
There was a problem hiding this comment.
Commenting here but applicable to all Configs below: It seems we are flipping the defaults to actually enable many of the obfuscators? It seems like a breaking change? Is this intended?
Collaborator
Author
There was a problem hiding this comment.
I have confirmed that the default impl was not used anywhere.
ObfuscationConfig is only passed to obfuscate_span which is used
- not used in libdatadog
- used in the serverless mini agent, but the config is read from env variables and does not depend on the new Default impl https://github.com/DataDog/serverless-components/blob/main/crates/datadog-trace-agent/src/config.rs#L216-L220
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
Collaborator
Author
The protobuf span struct is obsolete and used only by serverless for their extension. I cannot remove the If we ever want to send protobuf spans to the intake APM clients using libdatadog, we will do it by adding a protobuf serializer to the v1 span struct. |
bwoebi
approved these changes
Aug 27, 2026
bwoebi
left a comment
bwoebi
left a comment
Contributor
There was a problem hiding this comment.
I'm not too happy that it double obfuscates common operations, which have already been obfuscated once for stats computation.
But that's a future optimization, not a blocker. The implementation looks right to me: trivially obfuscating every span, and avoiding reallocating/copying for stuff which clearly doesn't need to be obfuscated.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?