feat(data-pipeline): emit native trace export telemetry by mabdinur · Pull Request #2338 · DataDog/libdatadog

mabdinur · 2026-08-10T19:07:42Z

Summary

Follow-up to DataDog/dd-trace-py#18967. When dd-trace-py uses NativeWriter, libdatadog owns trace serialization and export, so it must emit the same writer telemetry.

This covers every native trace export path:

Agentless JSON
Agent v0.4, v0.5, and v1 msgpack
OTLP JSON and protobuf

Metrics

spans_enqueued_for_serialization when spans enter the native export pipeline
spans_dropped with reason:serialization_error when serialization definitively fails
spans_dropped with reason:api_error when transport definitively fails
Existing trace_api.* and trace-chunk telemetry extended to agentless and OTLP export

Pre-send serialization failures count both affected trace chunks and spans. Transport failures retain the existing chunk-drop reason while recording the corresponding span drop as api_error.

Retry and byte accounting

Transport helpers invoke a crate-internal observer once after the retry loop with the final SendWithRetryResult:

trace_api.requests reflects the total number of HTTP attempts.
Response, error, chunk, and span outcomes are recorded once per payload.
Retries never multiply chunk or span drop counts.

Existing send_agentless_traces_http, send_otlp_traces_http, and send_with_retry signatures and error behavior remain unchanged. The additive send_with_retry_and_size helper also returns the final post-compression payload size for agentless byte telemetry, matching the bytes placed on the wire without recompressing the payload.

Test plan

cargo check -p libdd-data-pipeline
cargo check -p libdd-data-pipeline --no-default-features
cargo +stable clippy -p libdd-data-pipeline --all-targets -- -D warnings
cargo +stable clippy -p libdd-data-pipeline --all-targets --features compression -- -D warnings
cargo +stable clippy -p libdd-trace-utils --all-targets --features compression -- -D warnings
cargo nextest run -p libdd-data-pipeline (169 passed)
cargo nextest run -p libdd-data-pipeline --features compression (169 passed)
cargo nextest run -p libdd-trace-utils --features compression (336 passed)
cargo test -p libdd-data-pipeline --doc (2 passed)
cargo test -p libdd-trace-utils --features compression --doc (30 passed, 1 ignored)
cargo +nightly-2026-07-26 fmt --all -- --check
git diff --check

github-actions · 2026-08-10T19:09:32Z

📚 Documentation Check Results

⚠️ 2048 documentation warning(s) found

📦 `libdd-data-pipeline` - 1283 warning(s)

📦 `libdd-trace-utils` - 765 warning(s)

Updated: 2026-08-25 15:31:21 UTC | Commit: 315dad4 | missing-docs job results

github-actions · 2026-08-10T19:12:04Z

🔒 Cargo Deny Results

⚠️ 7 issue(s) found, showing only errors (advisories, bans, sources)

📦 `libdd-data-pipeline` - 4 error(s)

Show output

error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:77:1
   │
77 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.2.0
                     │   ├── libdd-capabilities-impl v4.0.0
                     │   │   ├── libdd-data-pipeline v8.0.0
                     │   │   ├── libdd-shared-runtime v3.0.0
                     │   │   │   ├── libdd-data-pipeline v8.0.0 (*)
                     │   │   │   ├── libdd-dogstatsd-client v5.0.0
                     │   │   │   │   ├── libdd-data-pipeline v8.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v7.0.0
                     │   │   │   │       └── libdd-data-pipeline v8.0.0 (*)
                     │   │   │   ├── libdd-telemetry v7.0.0
                     │   │   │   │   ├── libdd-data-pipeline v8.0.0 (*)
                     │   │   │   │   └── libdd-trace-stats v7.0.0 (*)
                     │   │   │   └── libdd-trace-stats v7.0.0 (*)
                     │   │   ├── (dev) libdd-telemetry v7.0.0 (*)
                     │   │   ├── libdd-trace-stats v7.0.0 (*)
                     │   │   └── libdd-trace-utils v10.1.0
                     │   │       ├── libdd-data-pipeline v8.0.0 (*)
                     │   │       ├── libdd-trace-obfuscation v6.0.0
                     │   │       │   └── libdd-trace-stats v7.0.0 (*)
                     │   │       ├── libdd-trace-stats v7.0.0 (*)
                     │   │       └── (dev) libdd-trace-utils v10.1.0 (*)
                     │   ├── libdd-data-pipeline v8.0.0 (*)
                     │   ├── libdd-dogstatsd-client v5.0.0 (*)
                     │   ├── libdd-shared-runtime v3.0.0 (*)
                     │   ├── libdd-telemetry v7.0.0 (*)
                     │   ├── libdd-trace-obfuscation v6.0.0 (*)
                     │   ├── libdd-trace-stats v7.0.0 (*)
                     │   └── libdd-trace-utils v10.1.0 (*)
                     ├── (dev) libdd-data-pipeline v8.0.0 (*)
                     ├── (dev) libdd-ddsketch v1.1.1
                     │   ├── libdd-data-pipeline v8.0.0 (*)
                     │   ├── libdd-telemetry v7.0.0 (*)
                     │   └── libdd-trace-stats v7.0.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.1
                     │   ├── libdd-data-pipeline v8.0.0 (*)
                     │   └── libdd-trace-utils v10.1.0 (*)
                     ├── (dev) libdd-trace-obfuscation v6.0.0 (*)
                     ├── (dev) libdd-trace-stats v7.0.0 (*)
                     └── (dev) libdd-trace-utils v10.1.0 (*)

error[vulnerability]: h2 unbounded empty DATA frames
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:117:1
    │
117 │ h2 0.4.6 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0258
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0258
    ├ The h2 crate, used internally by hyper, had a flaw that would accept and queue empty DATA frames without limit.
      If streams were not actively drained, this could lead to unbounded memory usage, or a panic if the length overflows.
      
      Low severity.
      
      Patched in v0.4.16.
    ├ Announcement: https://github.com/hyperium/hyper/security/advisories/GHSA-q83h-524g-xf6h
    ├ Solution: Upgrade to >=0.4.16 (try `cargo update -p h2`)
    ├ h2 v0.4.6
      └── hyper v1.6.0
          ├── httpmock v0.8.0-alpha.1
          │   ├── (dev) libdd-data-pipeline v8.0.0
          │   ├── (dev) libdd-telemetry v7.0.0
          │   │   ├── libdd-data-pipeline v8.0.0 (*)
          │   │   └── libdd-trace-stats v7.0.0
          │   │       └── libdd-data-pipeline v8.0.0 (*)
          │   ├── (dev) libdd-trace-stats v7.0.0 (*)
          │   └── libdd-trace-utils v10.1.0
          │       ├── libdd-data-pipeline v8.0.0 (*)
          │       ├── libdd-trace-obfuscation v6.0.0
          │       │   └── libdd-trace-stats v7.0.0 (*)
          │       ├── libdd-trace-stats v7.0.0 (*)
          │       └── (dev) libdd-trace-utils v10.1.0 (*)
          ├── hyper-rustls v0.27.7
          │   └── libdd-common v5.2.0
          │       ├── libdd-capabilities-impl v4.0.0
          │       │   ├── libdd-data-pipeline v8.0.0 (*)
          │       │   ├── libdd-shared-runtime v3.0.0
          │       │   │   ├── libdd-data-pipeline v8.0.0 (*)
          │       │   │   ├── libdd-dogstatsd-client v5.0.0
          │       │   │   │   ├── libdd-data-pipeline v8.0.0 (*)
          │       │   │   │   └── libdd-trace-stats v7.0.0 (*)
          │       │   │   ├── libdd-telemetry v7.0.0 (*)
          │       │   │   └── libdd-trace-stats v7.0.0 (*)
          │       │   ├── (dev) libdd-telemetry v7.0.0 (*)
          │       │   ├── libdd-trace-stats v7.0.0 (*)
          │       │   └── libdd-trace-utils v10.1.0 (*)
          │       ├── libdd-data-pipeline v8.0.0 (*)
          │       ├── libdd-dogstatsd-client v5.0.0 (*)
          │       ├── libdd-shared-runtime v3.0.0 (*)
          │       ├── libdd-telemetry v7.0.0 (*)
          │       ├── libdd-trace-obfuscation v6.0.0 (*)
          │       ├── libdd-trace-stats v7.0.0 (*)
          │       └── libdd-trace-utils v10.1.0 (*)
          ├── hyper-util v0.1.17
          │   ├── httpmock v0.8.0-alpha.1 (*)
          │   ├── hyper-rustls v0.27.7 (*)
          │   └── libdd-common v5.2.0 (*)
          ├── libdd-common v5.2.0 (*)
          └── libdd-trace-utils v10.1.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:223:1
    │
223 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── libdd-common v5.2.0
      │   ├── libdd-capabilities-impl v4.0.0
      │   │   ├── libdd-data-pipeline v8.0.0
      │   │   ├── libdd-shared-runtime v3.0.0
      │   │   │   ├── libdd-data-pipeline v8.0.0 (*)
      │   │   │   ├── libdd-dogstatsd-client v5.0.0
      │   │   │   │   ├── libdd-data-pipeline v8.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v7.0.0
      │   │   │   │       └── libdd-data-pipeline v8.0.0 (*)
      │   │   │   ├── libdd-telemetry v7.0.0
      │   │   │   │   ├── libdd-data-pipeline v8.0.0 (*)
      │   │   │   │   └── libdd-trace-stats v7.0.0 (*)
      │   │   │   └── libdd-trace-stats v7.0.0 (*)
      │   │   ├── (dev) libdd-telemetry v7.0.0 (*)
      │   │   ├── libdd-trace-stats v7.0.0 (*)
      │   │   └── libdd-trace-utils v10.1.0
      │   │       ├── libdd-data-pipeline v8.0.0 (*)
      │   │       ├── libdd-trace-obfuscation v6.0.0
      │   │       │   └── libdd-trace-stats v7.0.0 (*)
      │   │       ├── libdd-trace-stats v7.0.0 (*)
      │   │       └── (dev) libdd-trace-utils v10.1.0 (*)
      │   ├── libdd-data-pipeline v8.0.0 (*)
      │   ├── libdd-dogstatsd-client v5.0.0 (*)
      │   ├── libdd-shared-runtime v3.0.0 (*)
      │   ├── libdd-telemetry v7.0.0 (*)
      │   ├── libdd-trace-obfuscation v6.0.0 (*)
      │   ├── libdd-trace-stats v7.0.0 (*)
      │   └── libdd-trace-utils v10.1.0 (*)
      ├── (dev) libdd-data-pipeline v8.0.0 (*)
      ├── (dev) libdd-ddsketch v1.1.1
      │   ├── libdd-data-pipeline v8.0.0 (*)
      │   ├── libdd-telemetry v7.0.0 (*)
      │   └── libdd-trace-stats v7.0.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.1
      │   ├── libdd-data-pipeline v8.0.0 (*)
      │   └── libdd-trace-utils v10.1.0 (*)
      ├── (dev) libdd-trace-stats v7.0.0 (*)
      ├── libdd-trace-utils v10.1.0 (*)
      └── proptest v1.5.0
          ├── (dev) libdd-common v5.2.0 (*)
          └── (dev) libdd-tinybytes v1.1.2
              ├── libdd-data-pipeline v8.0.0 (*)
              ├── (dev) libdd-tinybytes v1.1.2 (*)
              └── libdd-trace-utils v10.1.0 (*)

error[vulnerability]: Denial of Service via Stack Exhaustion
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:295:1
    │
295 │ time 0.3.41 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
    │
    ├ ID: RUSTSEC-2026-0009
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0009
    ├ ## Impact
      
      When user-provided input is provided to any type that parses with the RFC 2822 format, a denial of
      service attack via stack exhaustion is possible. The attack relies on formally deprecated and
      rarely-used features that are part of the RFC 2822 format used in a malicious manner. Ordinary,
      non-malicious input will never encounter this scenario.
      
      ## Patches
      
      A limit to the depth of recursion was added in v0.3.47. From this version, an error will be returned
      rather than exhausting the stack.
      
      ## Workarounds
      
      Limiting the length of user input is the simplest way to avoid stack exhaustion, as the amount of
      the stack consumed would be at most a factor of the length of the input.
    ├ Announcement: https://github.com/time-rs/time/blob/main/CHANGELOG.md#0347-2026-02-05
    ├ Solution: Upgrade to >=0.3.47 (try `cargo update -p time`)
    ├ time v0.3.41
      └── tracing-appender v0.2.3
          └── libdd-log v1.0.0
              └── (dev) libdd-data-pipeline v8.0.0

advisories FAILED, bans ok, sources ok

📦 `libdd-trace-utils` - 3 error(s)

Show output

error[vulnerability]: Invalid pointer dereference in `fmt::Pointer` impl for `Atomic` and `Shared` when the underlying pointer is invalid
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:63:1
   │
63 │ crossbeam-epoch 0.9.18 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0204
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0204
   ├ Affected versions of `fmt::Display` dereference the underlying pointer. This causes a invalid pointer dereference e.g., when a pointer created with `Atomic::null` or `Shared::null`. `fmt::Debug` impls and pre-0.9 `fmt::Display` impls, which do not dereference pointers, are not affected by this issue.
   ├ Announcement: https://github.com/crossbeam-rs/crossbeam/pull/1276
   ├ Solution: Upgrade to >=0.9.20 (try `cargo update -p crossbeam-epoch`)
   ├ crossbeam-epoch v0.9.18
     └── crossbeam-deque v0.8.5
         └── rayon-core v1.12.1
             └── rayon v1.10.0
                 └── criterion v0.5.1
                     ├── libdd-common v5.2.0
                     │   ├── libdd-capabilities-impl v4.0.0
                     │   │   └── libdd-trace-utils v10.1.0
                     │   │       └── (dev) libdd-trace-utils v10.1.0 (*)
                     │   └── libdd-trace-utils v10.1.0 (*)
                     ├── (dev) libdd-trace-normalization v3.0.1
                     │   └── libdd-trace-utils v10.1.0 (*)
                     └── (dev) libdd-trace-utils v10.1.0 (*)

error[vulnerability]: h2 unbounded empty DATA frames
   ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:97:1
   │
97 │ h2 0.4.6 registry+https://github.com/rust-lang/crates.io-index
   │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ security vulnerability detected
   │
   ├ ID: RUSTSEC-2026-0258
   ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0258
   ├ The h2 crate, used internally by hyper, had a flaw that would accept and queue empty DATA frames without limit.
     If streams were not actively drained, this could lead to unbounded memory usage, or a panic if the length overflows.
     
     Low severity.
     
     Patched in v0.4.16.
   ├ Announcement: https://github.com/hyperium/hyper/security/advisories/GHSA-q83h-524g-xf6h
   ├ Solution: Upgrade to >=0.4.16 (try `cargo update -p h2`)
   ├ h2 v0.4.6
     └── hyper v1.6.0
         ├── httpmock v0.8.0-alpha.1
         │   └── libdd-trace-utils v10.1.0
         │       └── (dev) libdd-trace-utils v10.1.0 (*)
         ├── hyper-rustls v0.27.7
         │   └── libdd-common v5.2.0
         │       ├── libdd-capabilities-impl v4.0.0
         │       │   └── libdd-trace-utils v10.1.0 (*)
         │       └── libdd-trace-utils v10.1.0 (*)
         ├── hyper-util v0.1.17
         │   ├── httpmock v0.8.0-alpha.1 (*)
         │   ├── hyper-rustls v0.27.7 (*)
         │   └── libdd-common v5.2.0 (*)
         ├── libdd-common v5.2.0 (*)
         └── libdd-trace-utils v10.1.0 (*)

error[unsound]: Rand is unsound with a custom logger using `rand::rng()`
    ┌─ /home/runner/work/libdatadog/libdatadog/Cargo.lock:184:1
    │
184 │ rand 0.8.5 registry+https://github.com/rust-lang/crates.io-index
    │ ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ unsound advisory detected
    │
    ├ ID: RUSTSEC-2026-0097
    ├ Advisory: https://rustsec.org/advisories/RUSTSEC-2026-0097
    ├ It has been reported (by [@lopopolo](https://github.com/lopopolo)) that the `rand` library is [unsound](https://rust-lang.github.io/unsafe-code-guidelines/glossary.html#soundness-of-code--of-a-library) (i.e. that safe code using the public API can cause Undefined Behaviour) when all the following conditions are met:
      
      - The `log` and `thread_rng` features are enabled
      - A [custom logger](https://docs.rs/log/latest/log/#implementing-a-logger) is defined
      - The custom logger accesses `rand::rng()` (previously `rand::thread_rng()`) and calls any `TryRng` (previously `RngCore`) methods on `ThreadRng`
      - The `ThreadRng` (attempts to) reseed while called from the custom logger (this happens every 64 kB of generated data)
      - Trace-level logging is enabled or warn-level logging is enabled and the random source (the `getrandom` crate) is unable to provide a new seed
      
      `TryRng` (previously `RngCore`) methods for `ThreadRng` use `unsafe` code to cast `*mut BlockRng<ReseedingCore>` to `&mut BlockRng<ReseedingCore>`. When all the above conditions are met this results in an aliased mutable reference, violating the Stacked Borrows rules. Miri is able to detect this violation in sample code. Since construction of [aliased mutable references is Undefined Behaviour](https://doc.rust-lang.org/stable/nomicon/references.html), the behaviour of optimized builds is hard to predict.
    ├ Announcement: https://github.com/rust-random/rand/pull/1763
    ├ Solution: Upgrade to >=0.10.1 OR <0.10.0, >=0.9.3 OR <0.9.0, >=0.8.6 (try `cargo update -p rand`)
    ├ rand v0.8.5
      ├── (dev) libdd-common v5.2.0
      │   ├── libdd-capabilities-impl v4.0.0
      │   │   └── libdd-trace-utils v10.1.0
      │   │       └── (dev) libdd-trace-utils v10.1.0 (*)
      │   └── libdd-trace-utils v10.1.0 (*)
      ├── (dev) libdd-trace-normalization v3.0.1
      │   └── libdd-trace-utils v10.1.0 (*)
      ├── libdd-trace-utils v10.1.0 (*)
      └── proptest v1.5.0
          ├── (dev) libdd-common v5.2.0 (*)
          └── (dev) libdd-tinybytes v1.1.2
              ├── (dev) libdd-tinybytes v1.1.2 (*)
              └── libdd-trace-utils v10.1.0 (*)

advisories FAILED, bans ok, sources ok

Updated: 2026-08-25 15:32:37 UTC | Commit: 315dad4 | dependency-check job results

pr-commenter · 2026-08-10T19:12:43Z

Benchmarks

Comparison

Benchmark execution time: 2026-08-25 15:57:44

Comparing candidate commit 8c9d3fc in PR branch codex/trace-export-telemetry-metrics with baseline commit 69df7ed in branch main.

Found 7 performance improvements and 1 performance regressions! Performance is the same for 131 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:redis/obfuscate_redis_string

🟩 execution_time [-3.726µs; -3.232µs] or [-10.240%; -8.882%]

scenario:vec_map/as_deduped_map/already_deduped/16

🟩 execution_time [-1.028ns; -1.006ns] or [-4.188%; -4.100%]

scenario:vec_map/as_deduped_map/already_deduped/8

🟩 execution_time [-1.024ns; -0.994ns] or [-6.872%; -6.670%]

scenario:vec_map/get_hit/16

🟩 execution_time [-35.473ns; -35.211ns] or [-15.040%; -14.928%]
🟩 throughput [+11914402.877op/s; +11996142.277op/s] or [+17.564%; +17.684%]

scenario:vec_map/get_hit/8

🟩 execution_time [-10.931ns; -10.894ns] or [-16.865%; -16.808%]
🟩 throughput [+24944963.422op/s; +25031047.565op/s] or [+20.210%; +20.280%]

scenario:vec_map/get_miss/16

🟥 execution_time [+0.545ns; +0.633ns] or [+5.682%; +6.597%]

Benchmark execution time: 2026-08-25 15:57:12

Comparing candidate commit 8c9d3fc in PR branch codex/trace-export-telemetry-metrics with baseline commit 69df7ed in branch main.

Found 2 performance improvements and 3 performance regressions! Performance is the same for 68 metrics, 10 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

🟩 = significantly better candidate vs. baseline
🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:datadog_sample_span/parent_not_sampled_short_circuit/wall_time

🟩 execution_time [-0.404ns; -0.399ns] or [-4.363%; -4.313%]

scenario:datadog_sample_span/parent_sampled_short_circuit/wall_time

🟩 execution_time [-0.406ns; -0.401ns] or [-4.391%; -4.340%]

scenario:datadog_sample_span/resource_pattern_rule_matching/wall_time

🟥 execution_time [+13.596ns; +13.762ns] or [+5.286%; +5.351%]

scenario:trace_buffer/4_senders/no_delay

🟥 execution_time [+217.711µs; +245.729µs] or [+9.538%; +10.766%]
🟥 throughput [-155807.882op/s; -137349.597op/s] or [-9.866%; -8.697%]

Candidate

Omitted due to size.

Baseline

Omitted due to size.

datadog-datadog-prod-us1 · 2026-08-10T19:18:28Z

✅ All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

🎯 Code Coverage (details)
• Patch Coverage: 78.05%
• Overall Coverage: 76.64% (+0.08%)

_{This comment will be updated automatically if new data arrives.

🔗 Commit SHA: 8c9d3fc | Docs | View more details | Give us feedback!}

dd-octo-sts · 2026-08-10T19:45:17Z

Artifact Size Benchmark Report

aarch64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a	89.19 MB	89.26 MB	+.07% (+70.57 KB) 🔍
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so	8.20 MB	8.20 MB	0% (0 B) 👌

aarch64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so	11.06 MB	11.06 MB	+.01% (+1.76 KB) 🔍
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a	100.36 MB	100.43 MB	+.06% (+71.71 KB) 🔍

libdatadog-x64-windows

Artifact	Baseline	Commit	Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll	26.63 MB	26.62 MB	--.04% (-12.00 KB) 💪
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib	94.56 KB	94.56 KB	0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb	191.42 MB	180.59 MB	--5.65% (-10.82 MB) 💪
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib	1001.43 MB	769.82 MB	--23.12% (-231.61 MB) 💪
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll	8.71 MB	8.74 MB	+.34% (+31.00 KB) 🔍
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib	94.56 KB	94.56 KB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb	25.62 MB	25.62 MB	+.03% (+8.00 KB) 🔍
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib	51.01 MB	51.07 MB	+.12% (+67.16 KB) 🔍

libdatadog-x86-windows

Artifact	Baseline	Commit	Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll	23.22 MB	23.21 MB	--.02% (-5.00 KB) 💪
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib	96.04 KB	96.04 KB	0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb	196.33 MB	185.57 MB	--5.47% (-10.75 MB) 💪
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib	990.81 MB	757.87 MB	--23.50% (-232.93 MB) 💪
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll	6.73 MB	6.75 MB	+.27% (+19.00 KB) 🔍
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib	96.04 KB	96.04 KB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb	27.55 MB	27.56 MB	+.02% (+8.00 KB) 🔍
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib	48.55 MB	48.60 MB	+.10% (+52.34 KB) 🔍

x86_64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a	79.46 MB	79.52 MB	+.07% (+63.99 KB) 🔍
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so	9.12 MB	9.13 MB	+.12% (+12.00 KB) 🔍

x86_64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a	95.12 MB	95.18 MB	+.06% (+64.19 KB) 🔍
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so	11.14 MB	11.15 MB	+.11% (+13.53 KB) 🔍

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a07af01a22

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

mabdinur · 2026-08-25T14:43:16Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 45359c6565

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

mabdinur · 2026-08-25T15:29:18Z

@codex review

chatgpt-codex-connector · 2026-08-25T15:32:27Z

Codex Review: Didn't find any major issues. You're on a roll.

Reviewed commit: 8c9d3fc310

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

feat(data-pipeline): emit trace export telemetry metrics

befd1d0

github-actions Bot added the data-pipeline label Aug 10, 2026

mabdinur added 2 commits August 10, 2026 16:55

refactor(data-pipeline): preserve exporter visibility

a4070e1

refactor(data-pipeline): preserve mapper signature

a07af01

mabdinur marked this pull request as ready for review August 19, 2026 14:08

mabdinur requested review from a team as code owners August 19, 2026 14:08

mabdinur requested review from vpellan and removed request for a team August 19, 2026 14:08

chatgpt-codex-connector Bot reviewed Aug 19, 2026

View reviewed changes

Comment thread libdd-data-pipeline/src/agentless/exporter.rs Outdated

Comment thread libdd-data-pipeline/src/trace_exporter/mod.rs Outdated

Comment thread libdd-data-pipeline/src/trace_exporter/mod.rs Outdated

mabdinur requested a review from a team as a code owner August 20, 2026 13:42

github-actions Bot added the mini-agent label Aug 20, 2026

fix(data-pipeline): preserve trace transport behavior

3b00525

mabdinur force-pushed the codex/trace-export-telemetry-metrics branch from b827abc to 3b00525 Compare August 20, 2026 14:17

Aaalibaba42 requested changes Aug 25, 2026

View reviewed changes

Comment thread libdd-data-pipeline/src/trace_exporter/mod.rs Outdated

fix(data-pipeline): count spans after export filtering

6c0dae8

mabdinur requested a review from Aaalibaba42 August 25, 2026 14:04

refactor(data-pipeline): centralize payload telemetry counts

45359c6

mabdinur requested review from a team, Aaalibaba42 and vpellan and removed request for a team, Aaalibaba42 and vpellan August 25, 2026 14:43

chatgpt-codex-connector Bot reviewed Aug 25, 2026

View reviewed changes

Comment thread libdd-data-pipeline/src/trace_exporter/mod.rs Outdated

refactor(data-pipeline): reduce telemetry accounting diff

8c9d3fc

Uh oh!

Conversation

mabdinur commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Metrics

Retry and byte accounting

Test plan

Uh oh!

github-actions Bot commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

📚 Documentation Check Results

📦 libdd-data-pipeline - 1283 warning(s)

📦 libdd-trace-utils - 765 warning(s)

Uh oh!

github-actions Bot commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔒 Cargo Deny Results

📦 libdd-data-pipeline - 4 error(s)

📦 libdd-trace-utils - 3 error(s)

Uh oh!

pr-commenter Bot commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Benchmarks

Comparison

Explanation

More details about the CI and significant changes

scenario:redis/obfuscate_redis_string

scenario:vec_map/as_deduped_map/already_deduped/16

scenario:vec_map/as_deduped_map/already_deduped/8

scenario:vec_map/get_hit/16

scenario:vec_map/get_hit/8

scenario:vec_map/get_miss/16

Explanation

More details about the CI and significant changes

scenario:datadog_sample_span/parent_not_sampled_short_circuit/wall_time

scenario:datadog_sample_span/parent_sampled_short_circuit/wall_time

scenario:datadog_sample_span/resource_pattern_rule_matching/wall_time

scenario:trace_buffer/4_senders/no_delay

Candidate

Baseline

Uh oh!

datadog-datadog-prod-us1 Bot commented Aug 10, 2026 • edited by datadog-prod-us1-3 Bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

dd-octo-sts Bot commented Aug 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Artifact Size Benchmark Report

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mabdinur commented Aug 25, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

mabdinur commented Aug 25, 2026

Uh oh!

chatgpt-codex-connector Bot commented Aug 25, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

mabdinur commented Aug 10, 2026 •

edited

Loading

github-actions Bot commented Aug 10, 2026 •

edited

Loading

📦 `libdd-data-pipeline` - 1283 warning(s)

📦 `libdd-trace-utils` - 765 warning(s)

github-actions Bot commented Aug 10, 2026 •

edited

Loading

📦 `libdd-data-pipeline` - 4 error(s)

📦 `libdd-trace-utils` - 3 error(s)

pr-commenter Bot commented Aug 10, 2026 •

edited

Loading

datadog-datadog-prod-us1 Bot commented Aug 10, 2026 •

edited by datadog-prod-us1-3 Bot

Loading

dd-octo-sts Bot commented Aug 10, 2026 •

edited

Loading