fix(security): detect declared-marker obfuscation

[mohgupta-ship-it]

Powered by Codex.

Summary

This stacked PR extends #408 with deterministic detection for instructions that explicitly tell a reader to remove a literal marker before interpreting a command. It closes cases such as a declared xyz marker hiding rm -rf *, while preserving exact source locations and avoiding any execution or LLM dependency.

Deterministic flow

flowchart TD
    A["Original instruction text"] --> B["Find explicit remove or ignore marker directive"]
    B --> C{"Directive safe to reconstruct?"}
    C -- "No or ambiguous" --> D["Fail closed: incomplete analysis"]
    C -- "Yes" --> E["Remove literal marker once within bounded scope"]
    E --> F["Preserve derived-to-source offset map"]
    F --> G["Run existing static analyzers on original and derived views"]
    G --> H{"Dangerous command or policy match?"}
    H -- "Yes" --> I["Emit mapped finding"]
    H -- "No" --> J["Continue normal static analysis"]

Loading

Security behavior

• Supports quoted markers and tag-like markers with bounded, one-pass literal removal.
• Rejects ambiguous, encoded, overlapping, multi-marker, and resource-exhausted forms instead of guessing SAFE.
• Carries exact source offsets and source lines from reconstructed text into findings.
• Extends rm root-glob detection across split or combined flags, --, redirections, line continuations, quoting, escaped or fragmented command words, command/process substitution, and operand reordering.
• Avoids executing, evaluating, recursively decoding, or invoking an LLM.
• Keeps prose, quoted globs, suffix globs, negated directives, and safe marker examples from becoming command findings.

Bounds

The reconstruction path is deliberately bounded: marker length 16, declaration scope 768, lookahead 8192, payload 700, at most 8 active directives, and at most 64 removals. Unsupported forms become an incomplete-analysis ledger entry.

Validation

• uv run make lint
• uv run make format-check
• uv run make test-ci: 3116 passed, 13 skipped, 38 deselected, 4 xfailed
• Focused reconstruction suite: 118 passed
• Relevant static analyzer suites: 335 passed, 4 xfailed
• Security end-to-end suite: 64 passed
• Remediation and artifact suites: 210 passed
• uv run python -m build --no-isolation
• Docker image build and repository smoke test passed
• Real CLI corpus exercised explicitly with --no-llm: malicious reconstructed commands are detected, unsupported marker forms fail closed, and safe marker text remains SAFE

The exact CI run used Python 3.12, matching the repository workflow. The same initial coverage-only failures observed under Python 3.14 were reproduced on the untouched #408 base and are not caused by this change.

Adversarial review

The final diff was challenged across code standards, design/trust boundaries, false positives, false negatives, source mapping, truncation/window ownership, and no-LLM end-to-end behavior. No unresolved P0, P1, or P2 finding remains.

Deliberate limit

This is not arbitrary undeclared gapped-string or recursive deobfuscation. Reconstruction requires an explicit literal marker-removal instruction. Unsupported syntax fails closed so a future broader DP or scoring layer can be introduced without silently treating uncertain input as safe.

Stack: built directly on #408 (codex/security-text-normalization).