fix(ipc)!: gracefully handle SHM errors instead of unwrapping - #2408
Open
bwoebi wants to merge 2 commits into
Open
fix(ipc)!: gracefully handle SHM errors instead of unwrapping#2408bwoebi wants to merge 2 commits into
bwoebi wants to merge 2 commits into
Conversation
Contributor
Clippy Allow Annotation ReportTracked Clippy
By file and crateBy file
By crate
About This ReportThis report tracks Clippy allow annotations for specific rules, showing how they've changed in this PR. Decreasing the number of these annotations generally improves code quality. Panic-inducing macros in particular should be avoided. In the future, this report may become a PR-blocking quality gate. |
Contributor
📚 Documentation Check Results📦
|
|
Contributor
🔒 Cargo Deny Results📦
|
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7bb94b4645
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Comment on lines
+180
to
+182
| handle | ||
| .set_mapping_size(expected_size) | ||
| .map_err(io::Error::other)?; |
There was a problem hiding this comment.
When the metadata-page mmap fails after set_mapping_size succeeds, this function returns the error but remaps self using the enlarged local size. A later write of the same size therefore skips ensure_space, reports success, and writes data even though the shared size word was never advanced by fetch_max, so readers continue using the old size and cannot observe the payload. Restore the previous local size on this error path so a subsequent write retries publishing the new size.
AGENTS.md reference: AGENTS.md:L70-L73
Useful? React with 👍 / 👎.
Comment on lines
+189
to
+190
| limiter: if path.product() == RemoteConfigProduct::LiveDebugging { | ||
| Some(SHM_LIMITER.lock_or_panic().alloc()) | ||
| crate::tracer::with_shm_limiter(|mem| mem.alloc()).and_then(|result| { |
There was a problem hiding this comment.
If allocation fails here during transient shared-memory pressure, the immutable StoredShmFile.limiter is permanently set to None. Contrary to the warning, subsequent remote-config fetches reuse this stored file, and ConfigFileStorage::update only replaces its handle, so even after SHM recovers the config keeps serializing limiter index 0 until it is removed and stored again. Retrying allocation for existing live-debug files would restore rate limiting after recovery.
AGENTS.md reference: AGENTS.md:L70-L73
Useful? React with 👍 / 👎.
Comment on lines
+132
to
+134
| if let Err(e) = | ||
| writer.write(&serde_json::to_vec(&status.info).unwrap()) | ||
| { |
There was a problem hiding this comment.
When growing this segment fails transiently, the fetcher has already saved status.state_hash at line 119, so subsequent /info polls normally return SameState and never execute this write again. The in-process future is updated while SHM readers remain on the previous agent info indefinitely, until the agent's state hash changes for an unrelated reason. Preserve the pending payload or avoid advancing state until the SHM publication succeeds.
AGENTS.md reference: AGENTS.md:L70-L73
Useful? React with 👍 / 👎.
BenchmarksComparisonBenchmark execution time: 2026-08-26 16:28:54 Comparing candidate commit e177ecf in PR branch Found 0 performance improvements and 0 performance regressions! Performance is the same for 2 metrics, 0 unstable metrics.
|
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz | e177ecf | 1787761240 | bob/shm-error |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| write only interface | execution_time | 2.065µs | 2.087µs ± 0.007µs | 2.087µs ± 0.003µs | 2.090µs | 2.101µs | 2.108µs | 2.110µs | 1.12% | 0.543 | 1.700 | 0.33% | 0.000µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| write only interface | execution_time | [2.086µs; 2.088µs] or [-0.046%; +0.046%] | None | None | None |
Group 2
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz | e177ecf | 1787761240 | bob/shm-error |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| two way interface | execution_time | 21.187µs | 21.626µs ± 0.288µs | 21.575µs ± 0.123µs | 21.721µs | 22.073µs | 22.474µs | 23.981µs | 11.15% | 3.357 | 21.795 | 1.33% | 0.020µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| two way interface | execution_time | [21.587µs; 21.666µs] or [-0.185%; +0.185%] | None | None | None |
Baseline
Baseline benchmark details
Group 1
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz | 3131362 | 1787756561 | main |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| write only interface | execution_time | 2.062µs | 2.082µs ± 0.008µs | 2.081µs ± 0.003µs | 2.084µs | 2.097µs | 2.104µs | 2.106µs | 1.21% | 0.944 | 1.578 | 0.36% | 0.001µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| write only interface | execution_time | [2.081µs; 2.083µs] or [-0.050%; +0.050%] | None | None | None |
Group 2
| cpu_model | git_commit_sha | git_commit_date | git_branch |
|---|---|---|---|
| Intel(R) Xeon(R) Platinum 8175M CPU @ 2.50GHz | 3131362 | 1787756561 | main |
| scenario | metric | min | mean ± sd | median ± mad | p75 | p95 | p99 | max | peak_to_median_ratio | skewness | kurtosis | cv | sem | runs | sample_size |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| two way interface | execution_time | 21.241µs | 21.632µs ± 0.313µs | 21.585µs ± 0.147µs | 21.744µs | 22.029µs | 22.316µs | 24.654µs | 14.22% | 4.812 | 41.879 | 1.44% | 0.022µs | 1 | 200 |
| scenario | metric | 95% CI mean | Shapiro-Wilk pvalue | Ljung-Box pvalue (lag=1) | Dip test pvalue |
|---|---|---|---|---|---|
| two way interface | execution_time | [21.589µs; 21.676µs] or [-0.201%; +0.201%] | None | None | None |
Contributor
Artifact Size Benchmark Reportaarch64-alpine-linux-musl
aarch64-unknown-linux-gnu
libdatadog-x64-windows
libdatadog-x86-windows
x86_64-alpine-linux-musl
x86_64-unknown-linux-gnu
|
bwoebi
changed the title
- Restore fmt on telemetry.rs and one_way_shared_memory.rs warn! calls - Handle the now-fallible OneWayShmWriter::write in ddog_agent_remote_config_write instead of discarding the Result - Restore the previous local mapping size on macOS when publishing the new size fails, so a subsequent write retries instead of silently assuming success - Retry allocating the live-debugging rate limiter slot on later updates of an existing config file, not just on first store - Do not advance the agent-info state hash until the new payload is actually published to shared memory, so a failed write is retried instead of being silently skipped forever
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This prevents errors especially around ENOSPC on customers environments from stopping the sidecar alltogether.
BREAKING CHANGE:
OneWayShmWriter::write,MappedMem::ensure_space,ShmLimiterMemory::alloc, andShmLimiterMemory::alloc_with_granularityinlibdd-ipcnow returnio::Resultinstead of panicking on failure.