fix(APMSP-3164): prevent untrusted code from being executed

[hoolioh]

What does this PR do?

1. Adds a a guard in order to prevent untrusted sources to execute code. This guard is added to the following workflows:

• benchmarks
• fuzz
• impacted crates

2. Removes $CI_JOB_TOKEN from being persisted on disk.
3. Pins bothe the benchmark's image and script so unwanted changes don't get executed.

Motivation

Fix AMPSP-3164.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: af050f723f

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[datadog-datadog-prod-us1-2]

🎯 Code Coverage (details)
• Patch Coverage: 100.00%
• Overall Coverage: 76.72% (+0.01%)

_{This comment will be updated automatically if new data arrives.
🔗 Commit SHA: b61206a | Docs | View more details | Give us feedback!}

[dd-octo-sts]

Artifact Size Benchmark Report

aarch64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.so	8.26 MB	8.26 MB	0% (0 B) 👌
/aarch64-alpine-linux-musl/lib/libdatadog_profiling.a	89.73 MB	89.73 MB	0% (0 B) 👌

aarch64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.so	11.13 MB	11.13 MB	0% (0 B) 👌
/aarch64-unknown-linux-gnu/lib/libdatadog_profiling.a	100.95 MB	100.95 MB	0% (0 B) 👌

libdatadog-x64-windows

Artifact	Baseline	Commit	Change
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.dll	26.68 MB	26.68 MB	0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.lib	94.96 KB	94.96 KB	0% (0 B) 👌
/libdatadog-x64-windows/debug/dynamic/datadog_profiling_ffi.pdb	181.12 MB	181.12 MB	+0% (+8.00 KB) 👌
/libdatadog-x64-windows/debug/static/datadog_profiling_ffi.lib	773.75 MB	773.75 MB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.dll	8.75 MB	8.75 MB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.lib	94.96 KB	94.96 KB	0% (0 B) 👌
/libdatadog-x64-windows/release/dynamic/datadog_profiling_ffi.pdb	25.69 MB	25.69 MB	0% (0 B) 👌
/libdatadog-x64-windows/release/static/datadog_profiling_ffi.lib	51.17 MB	51.17 MB	0% (0 B) 👌

libdatadog-x86-windows

Artifact	Baseline	Commit	Change
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.dll	23.25 MB	23.25 MB	0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.lib	96.45 KB	96.45 KB	0% (0 B) 👌
/libdatadog-x86-windows/debug/dynamic/datadog_profiling_ffi.pdb	186.05 MB	186.06 MB	+0% (+16.00 KB) 👌
/libdatadog-x86-windows/debug/static/datadog_profiling_ffi.lib	760.72 MB	760.72 MB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.dll	6.76 MB	6.76 MB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.lib	96.45 KB	96.45 KB	0% (0 B) 👌
/libdatadog-x86-windows/release/dynamic/datadog_profiling_ffi.pdb	27.62 MB	27.62 MB	0% (0 B) 👌
/libdatadog-x86-windows/release/static/datadog_profiling_ffi.lib	48.69 MB	48.69 MB	0% (0 B) 👌

x86_64-alpine-linux-musl

Artifact	Baseline	Commit	Change
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.a	79.99 MB	79.99 MB	0% (0 B) 👌
/x86_64-alpine-linux-musl/lib/libdatadog_profiling.so	9.18 MB	9.18 MB	0% (0 B) 👌

x86_64-unknown-linux-gnu

Artifact	Baseline	Commit	Change
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.a	95.71 MB	95.71 MB	0% (0 B) 👌
/x86_64-unknown-linux-gnu/lib/libdatadog_profiling.so	11.19 MB	11.19 MB	0% (0 B) 👌

[pr-commenter]

Benchmarks

Comparison

Benchmark execution time: 2026-08-27 10:28:35

Comparing candidate commit b61206a in PR branch julio/APMSP-3164 with baseline commit 4cfd390 in branch main.

Found 12 performance improvements and 10 performance regressions! Performance is the same for 131 metrics, 0 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

• 🟩 = significantly better candidate vs. baseline
• 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:msgpack_decoder::v05/high_sharing/10000

• 🟥 execution_time [+633.671µs; +636.254µs] or [+8.386%; +8.420%]
• 🟥 throughput [-102787.394op/s; -102374.500op/s] or [-7.767%; -7.736%]

scenario:normalization/normalize_name/normalize_name/Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Long-.Too-Lo...

• 🟩 execution_time [-18.943µs; -18.785µs] or [-9.235%; -9.158%]
• 🟩 throughput [+491737.859op/s; +495873.206op/s] or [+10.086%; +10.171%]

scenario:normalization/normalize_name/normalize_name/bad-name

• 🟩 execution_time [-2.029µs; -1.989µs] or [-10.544%; -10.338%]
• 🟩 throughput [+5997460.412op/s; +6125198.564op/s] or [+11.540%; +11.786%]

scenario:normalization/normalize_name/normalize_name/good

• 🟩 execution_time [-1.725µs; -1.701µs] or [-15.116%; -14.897%]
• 🟩 throughput [+15354147.716op/s; +15584216.531op/s] or [+17.526%; +17.789%]

scenario:normalization/normalize_service/normalize_service/A0000000000000000000000000000000000000000000000000...

• 🟩 execution_time [-36.599µs; -36.338µs] or [-6.848%; -6.799%]
• 🟩 throughput [+136506.558op/s; +137512.292op/s] or [+7.296%; +7.350%]

scenario:normalization/normalize_service/normalize_service/Test Conversion 0f Weird !@#$%^&**() Characters

• 🟩 execution_time [-21.384µs; -21.264µs] or [-11.212%; -11.149%]
• 🟩 throughput [+658251.905op/s; +661793.679op/s] or [+12.554%; +12.622%]

scenario:vec_map/as_deduped_map/already_deduped/8

• 🟥 execution_time [+0.599ns; +0.606ns] or [+4.316%; +4.371%]

scenario:vec_map/as_deduped_map/needs_dedup_1_in_10/8

• 🟩 execution_time [-16.399ns; -16.222ns] or [-4.440%; -4.392%]

scenario:vec_map/get_miss/16

• 🟩 execution_time [-1.123ns; -1.077ns] or [-10.747%; -10.300%]

scenario:vec_map/get_miss/8

• 🟥 execution_time [+1.491ns; +1.566ns] or [+24.943%; +26.186%]

scenario:vec_map/get_mut/128

• 🟥 execution_time [+1.686µs; +1.794µs] or [+11.900%; +12.662%]
• 🟥 throughput [-1019347.566op/s; -957808.288op/s] or [-11.279%; -10.599%]

scenario:vec_map/get_mut/64

• 🟥 execution_time [+416.730ns; +455.773ns] or [+10.712%; +11.715%]
• 🟥 throughput [-1735946.048op/s; -1585463.043op/s] or [-10.546%; -9.632%]

scenario:vec_map/get_mut/8

• 🟥 execution_time [+6.828ns; +11.630ns] or [+8.152%; +13.885%]
• 🟥 throughput [-12148342.878op/s; -6937056.972op/s] or [-12.482%; -7.128%]

Benchmark execution time: 2026-08-27 10:36:32

Comparing candidate commit b61206a in PR branch julio/APMSP-3164 with baseline commit 4cfd390 in branch main.

Found 10 performance improvements and 4 performance regressions! Performance is the same for 154 metrics, 10 unstable metrics.

Explanation

This is an A/B test comparing a candidate commit's performance against that of a baseline commit. Performance changes are noted in the tables below as:

• 🟩 = significantly better candidate vs. baseline
• 🟥 = significantly worse candidate vs. baseline

We compute a confidence interval (CI) over the relative difference of means between metrics from the candidate and baseline commits, considering the baseline as the reference.

If the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD), the change is considered significant.

Feel free to reach out to #apm-benchmarking-platform on Slack if you have any questions.

More details about the CI and significant changes

You can imagine this CI as a range of values that is likely to contain the true difference of means between the candidate and baseline commits.

CIs of the difference of means are often centered around 0%, because often changes are not that big:

---------------------------------(------|---^--------)-------------------------------->
                              -0.6%    0%  0.3%     +1.2%
                                 |          |        |
         lower bound of the CI --'          |        |
sample mean (center of the CI) -------------'        |
         upper bound of the CI ----------------------'

As described above, a change is considered significant if the CI is entirely outside the configured SIGNIFICANT_IMPACT_THRESHOLD (or the deprecated UNCONFIDENCE_THRESHOLD).

For instance, for an execution time metric, this confidence interval indicates a significantly worse performance:

----------------------------------------|---------|---(---------^---------)---------->
                                       0%        1%  1.3%      2.2%      3.1%
                                                  |   |         |         |
       significant impact threshold --------------'   |         |         |
                      lower bound of CI --------------'         |         |
       sample mean (center of the CI) --------------------------'         |
                      upper bound of CI ----------------------------------'

scenario:alloc_free/system/4096

• 🟥 execution_time [+11.525ns; +11.705ns] or [+14.104%; +14.323%]

scenario:datadog_sample_span/multiple_rules_first_match/wall_time

• 🟩 execution_time [-8.620ns; -8.513ns] or [-5.156%; -5.092%]

scenario:datadog_sample_span/multiple_rules_last_match/wall_time

• 🟩 execution_time [-10.893ns; -10.764ns] or [-5.254%; -5.191%]

scenario:datadog_sample_span/name_pattern_rule_matching/wall_time

• 🟩 execution_time [-11.802ns; -11.700ns] or [-4.409%; -4.371%]

scenario:datadog_sample_span/resource_pattern_rule_matching/wall_time

• 🟩 execution_time [-11.086ns; -10.931ns] or [-4.152%; -4.094%]

scenario:datadog_sample_span/resource_pattern_rule_not_matching/wall_time

• 🟩 execution_time [-8.161ns; -8.076ns] or [-4.871%; -4.820%]

scenario:datadog_sample_span/service_rule_matching/wall_time

• 🟩 execution_time [-11.784ns; -11.661ns] or [-4.538%; -4.491%]

scenario:glob_matcher/ascii_pattern_unicode_subject/wall_time

• 🟩 execution_time [-7.730ns; -7.606ns] or [-5.355%; -5.269%]

scenario:glob_matcher/ascii_wildcard_question_match/wall_time

• 🟩 execution_time [-21.395ns; -21.373ns] or [-36.764%; -36.727%]

scenario:glob_matcher/ascii_wildcard_star_match/wall_time

• 🟩 execution_time [-21.484ns; -21.453ns] or [-36.879%; -36.827%]

scenario:no_profiler/short_circuit/4096

• 🟥 execution_time [+7.868ns; +8.004ns] or [+8.217%; +8.359%]

scenario:profiler_attached/fast_path_system/4096

• 🟩 execution_time [-13.816ns; -13.673ns] or [-13.188%; -13.052%]

scenario:trace_buffer/4_senders/no_delay

• 🟥 execution_time [+264.103µs; +289.086µs] or [+11.346%; +12.419%]
• 🟥 throughput [-173813.214op/s; -157788.828op/s] or [-11.225%; -10.190%]

Candidate

Omitted due to size.

Baseline

Omitted due to size.