diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index d90f594..7c0a763 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -79,14 +79,28 @@ jobs: env: APP_BIND: 127.0.0.1:8082 DB_URL: sqlite:///./loadtest.db + # DEBUG=false so the load test exercises the production settings path, + # which means these must clear the 32-character minimum in + # Settings._validate_secret. Throwaway CI values, not secrets. DEBUG: "false" - KEY_HMAC_SECRET: ci-load-hmac-secret - PII_TOKEN_SALT: ci-load-salt - WEBHOOK_SECRET: ci-load-webhook-secret + KEY_HMAC_SECRET: ci-load-hmac-secret-padded-to-32-chars-min + PII_TOKEN_SALT: ci-load-token-salt-padded-to-32-chars-min + WEBHOOK_SECRET: ci-load-webhook-secret-padded-to-32-chars-min PRECHECK_DLQ: /tmp/precheck-load.dlq.jsonl LOAD_TEST_API_KEY: GAI_ci_load_test_key PRECHECK_BASE_URL: http://127.0.0.1:8082 LOAD_USER_POOL_SIZE: "120" + # The k6 profile is 100 iters/s for 30s (3000 requests) and every one of + # them carries the same API key — LOAD_USER_POOL_SIZE only varies the + # user_id inside the payload, while the limiter keys on + # hash_api_key(raw_key). Against the 100/min per-key default that lets + # roughly 113 requests through and 429s the rest, which is what the job + # has been measuring. Raised so the run exercises the precheck hot path + # rather than its own rate limiter. + RATE_LIMIT_REQUESTS_PER_MINUTE: "20000" + RATE_LIMIT_ORG_REQUESTS_PER_MINUTE: "20000" + RATE_LIMIT_TOKENS_PER_MINUTE: "10000000" + RATE_LIMIT_ORG_TOKENS_PER_MINUTE: "10000000" steps: - uses: actions/checkout@v4 diff --git a/PROJECT_SPECS.md b/PROJECT_SPECS.md index 1966084..8f08b6d 100644 --- a/PROJECT_SPECS.md +++ b/PROJECT_SPECS.md @@ -24,8 +24,21 @@ GovernsAI Precheck is a policy evaluation and PII redaction service that provide ### 3. PII Detection & Redaction - **Presidio integration**: Advanced NLP-based PII detection - **Fallback detection**: Regex-based detection when Presidio unavailable -- **Multiple PII types**: Email, SSN, phone numbers, credit cards, API keys, JWT tokens -- **False positive filtering**: Context-aware filtering to reduce false positives +- **Multiple PII types**: names, addresses, email, SSN, phone numbers, credit cards, IBAN, IP addresses, HIPAA PHI identifiers, PCI fields, API keys, JWT tokens +- **Multilingual**: analyzer built per configured language (`PRESIDIO_LANGUAGES`); en, es, fr, de, zh models ship in the image +- **False positive filtering**: context-aware filtering over the matched span +- **Measured**: `bench/` reports leakage, over-redaction and latency per language, tier and entity type — see `bench/README.md` + +**Entity coverage.** `DETECT_ENTITIES` in `app/policies.py` is the list the +analyzer is asked for. `ORGANIZATION` and `DATE_TIME` are deliberately excluded: +both are recognised by the spaCy pipeline but neither identifies a person, and +including them redacted 61.5% of PII-free control text. Utility loss on that +scale makes operators disable redaction outright, which is worse than leaving +those entities in place. + +**Known residual gaps** (measured, unfixed): obfuscated surface forms 75% +leakage, partial disclosure 83.3%, Chinese 32.1%, US_SSN 14.3%. Closing the +first two needs a different class of detector, not a better pattern. ### 4. Event Emission & Monitoring - **Webhook events**: Real-time policy decision events with HMAC authentication @@ -591,10 +604,16 @@ The policy evaluation system follows a strict precedence hierarchy (highest to l ### 1. **DENY_TOOLS** (Highest Priority) - **Purpose**: Hard deny for dangerous tools -- **Tools**: `python.exec`, `bash.exec`, `code.exec`, `shell.exec` +- **Tools**: `python.exec`, `bash.exec`, `code.exec`, `shell.exec`, `subprocess.run`, `subprocess.popen`, `os.system`, `child_process.exec`, `child_process.spawn`, `runtime.exec` - **Decision**: Always `deny` - **Policy ID**: `deny-exec` - **Reason**: `blocked tool: code/exec` +- **Matching**: `is_denied_tool()` normalises case, whitespace and separators + (`_`, `-`, `/`, `:` all fold to `.`), then matches on exact name, component + set (so `exec.python` matches `python.exec`), containment within a namespaced + path (`agent.python.exec.v2`), and explicit `namespace.*` wildcards. Denial is + a security boundary, so matching is deliberately generous: a false deny is a + support ticket, a false allow is an incident. ### 2. **TOOL_SPECIFIC** (High Priority) - **Purpose**: Tool-specific rules in `policy.tool_access.yaml` @@ -931,7 +950,9 @@ Budget limits can be configured per user: | `ON_ERROR` | Error handling behavior | `block` | | `POLICY_FILE` | Policy file path | `policy.tool_access.yaml` | | `USE_PRESIDIO` | Enable Presidio PII detection | `true` | -| `PRESIDIO_MODEL` | spaCy model for Presidio | `en_core_web_sm` | +| `PRESIDIO_MODEL` | English spaCy model for Presidio | `en_core_web_sm` | +| `PRESIDIO_LANGUAGES` | Comma-separated analyzer languages; each needs its spaCy model installed | `en` | +| `PRESIDIO_LANGUAGE_MODE` | `hint` (one pass, caller-supplied language) or `union` (all configured languages, recall-safe, ~N× NLP cost and higher false positives) | `hint` | ### API Configuration @@ -1171,6 +1192,40 @@ curl -X POST http://localhost:8080/api/v1/postcheck \ ``` ## Recent Changes Log +- **2026-08-11**: **PII detection benchmark + four detection fixes.** Added `bench/`, a + 438-item span-annotated corpus across five languages measuring the deployed + redaction path rather than the detector in isolation. Overall leakage fell + **51.3% → 8.2%**; high-sensitivity entity leakage **89.1% → 10.6%**; PERSON and + LOCATION **100% → ~11.6%**; over-redaction stayed at 0% and p50 latency was + unchanged (2.46 → 2.48 ms). + - **Entity allowlist**: the analyzer was asked only for the 16 keys of + `ANONYMIZE_OPERATORS`, which contain no `PERSON`, `LOCATION` or `NRP`. The + spaCy model detected names at score 0.85 and the pipeline then discarded + them — names and addresses were never redacted, in any language. Replaced + with an explicit `DETECT_ENTITIES` list; `ORGANIZATION` and `DATE_TIME` + stay excluded to protect utility. + - **Language pin**: `ANALYZER.analyze(..., language="en")` was hardcoded at all + three call sites and `AnalyzerEngine` was built with `supported_languages=["en"]`, + so the es/fr/de/zh models the Dockerfile installs were unreachable. Added + `PRESIDIO_LANGUAGES`, `PRESIDIO_LANGUAGE_MODE`, per-request language hints via + `tool_config.metadata.language`, and a shared `analyze_text()` so the three + sites cannot drift. + - **Per-language recognizer gaps**: Presidio registers `CreditCardRecognizer` + for en/es only, and its default phone regions exclude ES/FR/CN. A hinted + fr/de/zh request leaked 61.5% of card numbers. Both now registered per + configured language. + - **False-positive filter**: `is_false_positive()` was receiving the whole input + text instead of the matched span, so its suppression rules almost never + fired. The `US_SSN` recognizer also passed `deny_list=["password", "key", ...]`, + but Presidio's `deny_list` is a list of terms to *detect* — the literal word + "key" matched as an SSN. Removed; suppression lives in `is_false_positive()`. + - **Denylist matching**: `if tool in deny_tools` was an exact string test. + `Python.Exec`, `python_exec`, `exec.python` and `agent.python.exec.v2` all + walked past it. Replaced with `is_denied_tool()` (normalisation, component-set, + containment, `namespace.*` wildcards) and extended the defaults with + `subprocess.run`, `os.system` and other direct execution primitives. + - **Tests**: +69 (`test_deny_tools_matching.py`, `test_multilingual_pii.py`); + suite 258 → 327 passing, coverage 75.1% → 76.8%. - **2026-04-23**: Added the Mode 2 sidecar / proxy gateway design document at `docs/design/sidecar-mode.md` - **Language Decision**: Recommends Go for the proxy hot path over Node.js and Python - **Interception Model**: Defines `POST /v1/chat/completions` interception with `precheck` before upstream forwarding diff --git a/app/api.py b/app/api.py index f1a5a06..71dacb5 100644 --- a/app/api.py +++ b/app/api.py @@ -8,7 +8,14 @@ from datetime import datetime from typing import List, Optional, Tuple -from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Request, Response +from fastapi import ( + APIRouter, + BackgroundTasks, + Depends, + HTTPException, + Request, + Response, +) from sqlalchemy.orm import Session from .auth import AuthContext, require_api_key diff --git a/app/policies.py b/app/policies.py index 73093c2..7ad5e44 100644 --- a/app/policies.py +++ b/app/policies.py @@ -1,4 +1,5 @@ import hashlib +import importlib.util import os import re import time @@ -13,6 +14,10 @@ RecognizerRegistry, ) from presidio_analyzer.nlp_engine import SpacyNlpEngine +from presidio_analyzer.predefined_recognizers import ( + CreditCardRecognizer, + PhoneRecognizer, +) from presidio_anonymizer import AnonymizerEngine from presidio_anonymizer.entities import OperatorConfig @@ -114,158 +119,286 @@ def _replace_regex(s: str, pattern: re.Pattern, placeholder: str) -> str: USE_PRESIDIO = settings.use_presidio if hasattr(settings, "use_presidio") else True -def build_presidio(): - """Initialize Presidio analyzer and anonymizer with custom recognizers""" - try: - # Initialize spaCy NLP engine with configured model and load it - model_name = getattr(settings, "presidio_model", "en_core_web_sm") - # Presidio 2.x expects a list of {lang_code, model_name} - nlp_engine = SpacyNlpEngine( - models=[{"lang_code": "en", "model_name": model_name}] - ) - nlp_engine.load() - registry = RecognizerRegistry() - registry.load_predefined_recognizers(nlp_engine=nlp_engine) - - # Custom API key recognizer - api_key_pattern = Pattern( - name="API_KEY", - regex=r"(?:sk|pk|AKIA|ghp|gho|ghu|ghs|ghr)_[A-Za-z0-9]{16,40}", - score=0.6, - ) - api_key_recognizer = PatternRecognizer( +# spaCy model used for each supported language. The English entry is +# overridable via PRESIDIO_MODEL (sm/md/lg); the rest track the models the +# Dockerfile installs. +DEFAULT_SPACY_MODELS = { + "en": "en_core_web_sm", + "es": "es_core_news_sm", + "fr": "fr_core_news_sm", + "de": "de_core_news_sm", + "zh": "zh_core_web_sm", +} + +# phonenumbers regions to try per configured language. Presidio's default region +# set is US/UK/DE/FE/IL/IN/CA/BR, which silently misses ES, FR and CN numbers +# even when the matching spaCy model is loaded. +PHONE_REGIONS_BY_LANGUAGE = { + "en": ["US", "UK", "CA", "IN", "AU"], + "es": ["ES", "MX", "AR"], + "fr": ["FR", "BE", "CA"], + "de": ["DE", "AT", "CH"], + "zh": ["CN", "HK", "TW"], +} + +# Languages the analyzer was actually built for. Populated by build_presidio; +# a configured language whose spaCy model is missing is dropped from this list +# rather than taking the whole service down. +SUPPORTED_LANGUAGES: List[str] = ["en"] + + +def _model_available(model_name: str) -> bool: + return importlib.util.find_spec(model_name) is not None + + +def _resolve_language_models() -> List[Dict[str, str]]: + """Build the {lang_code, model_name} list for the configured languages.""" + configured = settings.presidio_language_list() + models: List[Dict[str, str]] = [] + for lang in configured: + if lang == "en": + model_name = getattr(settings, "presidio_model", "en_core_web_sm") + else: + model_name = DEFAULT_SPACY_MODELS.get(lang) + if not model_name: + print(f"Presidio: no spaCy model mapped for language {lang!r}; skipping") + continue + if not _model_available(model_name): + print( + f"Presidio: spaCy model {model_name!r} for language {lang!r} is not " + "installed; skipping that language" + ) + continue + models.append({"lang_code": lang, "model_name": model_name}) + + if not models: + models = [{"lang_code": "en", "model_name": "en_core_web_sm"}] + return models + + +def _custom_recognizers(language: str) -> List[PatternRecognizer]: + """Pattern recognizers registered for every supported language. + + Presidio scopes each recognizer to one language, so the same set is built + per language rather than once globally — otherwise a Spanish request gets + the spaCy NER model but none of the SSN/PHI/PCI patterns. + """ + recognizers: List[PatternRecognizer] = [] + + # API keys. Underscore is optional so bare AWS-style keys (AKIA...) match, + # and vendor-prefixed keys (sk_live_..., pk_test_...) match through their + # internal separators. + recognizers.append( + PatternRecognizer( supported_entity="API_KEY", - patterns=[api_key_pattern], + supported_language=language, + patterns=[ + Pattern( + name="API_KEY_VENDOR_PREFIXED", + regex=r"\b(?:sk|pk|rk)_(?:live|test)_[A-Za-z0-9]{10,40}\b", + score=0.75, + ), + Pattern( + name="API_KEY", + regex=r"\b(?:sk|pk|AKIA|ghp|gho|ghu|ghs|ghr)_?[A-Za-z0-9]{16,40}\b", + score=0.6, + ), + ], context=["secret", "token", "apikey", "api_key", "bearer", "key"], ) - registry.add_recognizer(api_key_recognizer) + ) - # JWT token recognizer - jwt_pattern = Pattern( - name="JWT_TOKEN", - regex=r"eyJ[A-Za-z0-9_-]*\.eyJ[A-Za-z0-9_-]*\.[A-Za-z0-9_-]*", - score=0.8, - ) - jwt_recognizer = PatternRecognizer( + recognizers.append( + PatternRecognizer( supported_entity="JWT_TOKEN", - patterns=[jwt_pattern], + supported_language=language, + patterns=[ + Pattern( + name="JWT_TOKEN", + regex=r"eyJ[A-Za-z0-9_-]*\.eyJ[A-Za-z0-9_-]*\.[A-Za-z0-9_-]*", + score=0.8, + ) + ], context=["token", "jwt", "bearer", "authorization"], ) - registry.add_recognizer(jwt_recognizer) + ) - # Override SSN recognizer to be more context-aware and exclude passwords - ssn_pattern = Pattern( - name="US_SSN", - regex=r"\b(?!000|666|9\d{2})\d{3}[-]?(?!00)\d{2}[-]?(?!0000)\d{4}\b", - score=0.8, - ) - ssn_recognizer = PatternRecognizer( + # Override SSN recognizer to be more context-aware and exclude passwords + recognizers.append( + PatternRecognizer( supported_entity="US_SSN", - patterns=[ssn_pattern], + supported_language=language, + patterns=[ + Pattern( + name="US_SSN", + regex=r"\b(?!000|666|9\d{2})\d{3}[-]?(?!00)\d{2}[-]?(?!0000)\d{4}\b", + score=0.8, + ) + ], context=["ssn", "social", "security", "tax", "id", "number"], - deny_list=["password", "pass", "pwd", "secret", "key", "token"], + # No deny_list here. Presidio's deny_list is a list of terms to + # *detect*, not to suppress — passing ["password", "key", ...] made + # the literal word "key" match as US_SSN. Suppression belongs in + # is_false_positive(), which runs over the matched span below. ) - registry.add_recognizer(ssn_recognizer) - - # HIPAA PHI recognizers - registry.add_recognizer( - PatternRecognizer( - supported_entity="US_MEDICAL_RECORD_NUMBER", - patterns=[ - Pattern( - name="US_MEDICAL_RECORD_NUMBER", - regex=r"\b(?:MRN|Medical\s*Record(?:\s*Number)?|Patient\s*ID)\s*[:#]?\s*[A-Z0-9\-]{6,20}\b", - score=0.82, - ) - ], - context=["mrn", "medical record", "patient id", "chart"], - ) + ) + + # HIPAA PHI recognizers + recognizers.append( + PatternRecognizer( + supported_entity="US_MEDICAL_RECORD_NUMBER", + supported_language=language, + patterns=[ + Pattern( + name="US_MEDICAL_RECORD_NUMBER", + regex=r"\b(?:MRN|Medical\s*Record(?:\s*Number)?|Patient\s*ID)\s*[:#]?\s*[A-Z0-9\-]{6,20}\b", + score=0.82, + ) + ], + context=["mrn", "medical record", "patient id", "chart"], ) - registry.add_recognizer( - PatternRecognizer( - supported_entity="US_HEALTH_MEMBER_ID", - patterns=[ - Pattern( - name="US_HEALTH_MEMBER_ID", - regex=r"\b(?:Member|Policy|Insurance)\s*(?:ID|Number)\s*[:#]?\s*[A-Z0-9\-]{6,24}\b", - score=0.8, - ) - ], - context=["member id", "policy", "insurance", "payer"], - ) + ) + recognizers.append( + PatternRecognizer( + supported_entity="US_HEALTH_MEMBER_ID", + supported_language=language, + patterns=[ + Pattern( + name="US_HEALTH_MEMBER_ID", + regex=r"\b(?:Member|Policy|Insurance)\s*(?:ID|Number)\s*[:#]?\s*[A-Z0-9\-]{6,24}\b", + score=0.8, + ) + ], + context=["member id", "policy", "insurance", "payer"], ) - registry.add_recognizer( - PatternRecognizer( - supported_entity="US_NPI", - patterns=[ - Pattern( - name="US_NPI", - regex=r"\b(?:NPI|National\s*Provider\s*Identifier)\s*[:#]?\s*\d{10}\b", - score=0.85, - ) - ], - context=["npi", "provider", "national provider identifier"], - ) + ) + recognizers.append( + PatternRecognizer( + supported_entity="US_NPI", + supported_language=language, + patterns=[ + Pattern( + name="US_NPI", + regex=r"\b(?:NPI|National\s*Provider\s*Identifier)\s*[:#]?\s*\d{10}\b", + score=0.85, + ) + ], + context=["npi", "provider", "national provider identifier"], ) - registry.add_recognizer( - PatternRecognizer( - supported_entity="US_DEA", - patterns=[ - Pattern( - name="US_DEA", - regex=r"\b(?:DEA|DEA\s*Number)\s*[:#]?\s*[A-Z]{2}\d{7}\b", - score=0.85, - ) - ], - context=["dea", "prescriber", "controlled substance"], - ) + ) + recognizers.append( + PatternRecognizer( + supported_entity="US_DEA", + supported_language=language, + patterns=[ + Pattern( + name="US_DEA", + regex=r"\b(?:DEA|DEA\s*Number)\s*[:#]?\s*[A-Z]{2}\d{7}\b", + score=0.85, + ) + ], + context=["dea", "prescriber", "controlled substance"], ) - registry.add_recognizer( - PatternRecognizer( - supported_entity="US_DATE_OF_BIRTH", - patterns=[ - Pattern( - name="US_DATE_OF_BIRTH", - regex=r"\b(?:DOB|Date\s*of\s*Birth)\s*[:#]?\s*(?:0?[1-9]|1[0-2])[/-](?:0?[1-9]|[12][0-9]|3[01])[/-](?:19|20)?\d{2}\b", - score=0.78, - ) - ], - context=["dob", "date of birth", "patient"], - ) + ) + recognizers.append( + PatternRecognizer( + supported_entity="US_DATE_OF_BIRTH", + supported_language=language, + patterns=[ + Pattern( + name="US_DATE_OF_BIRTH", + regex=r"\b(?:DOB|Date\s*of\s*Birth)\s*[:#]?\s*(?:0?[1-9]|1[0-2])[/-](?:0?[1-9]|[12][0-9]|3[01])[/-](?:19|20)?\d{2}\b", + score=0.78, + ) + ], + context=["dob", "date of birth", "patient"], ) + ) - # PCI-DSS related recognizers - registry.add_recognizer( - PatternRecognizer( - supported_entity="PCI_CVV", - patterns=[ - Pattern( - name="PCI_CVV", - regex=r"\b(?:cvv|cvc|cvn|security\s*code)\s*[:#]?\s*\d{3,4}\b", - score=0.88, - ) - ], - context=["cvv", "cvc", "security code", "payment"], - ) + # PCI-DSS related recognizers + recognizers.append( + PatternRecognizer( + supported_entity="PCI_CVV", + supported_language=language, + patterns=[ + Pattern( + name="PCI_CVV", + regex=r"\b(?:cvv|cvc|cvn|security\s*code)\s*[:#]?\s*\d{3,4}\b", + score=0.88, + ) + ], + context=["cvv", "cvc", "security code", "payment"], ) - registry.add_recognizer( - PatternRecognizer( - supported_entity="PCI_EXPIRY", - patterns=[ - Pattern( - name="PCI_EXPIRY", - regex=r"\b(?:exp(?:iry|iration)?|valid\s*thru)\s*[:#]?\s*(?:0[1-9]|1[0-2])[/-](?:\d{2}|\d{4})\b", - score=0.84, - ) - ], - context=["expiry", "expiration", "valid thru", "payment"], - ) + ) + recognizers.append( + PatternRecognizer( + supported_entity="PCI_EXPIRY", + supported_language=language, + patterns=[ + Pattern( + name="PCI_EXPIRY", + regex=r"\b(?:exp(?:iry|iration)?|valid\s*thru)\s*[:#]?\s*(?:0[1-9]|1[0-2])[/-](?:\d{2}|\d{4})\b", + score=0.84, + ) + ], + context=["expiry", "expiration", "valid thru", "payment"], ) + ) + + return recognizers + + +def build_presidio(): + """Initialize Presidio analyzer and anonymizer with custom recognizers. + + Builds one NLP pipeline per configured language (PRESIDIO_LANGUAGES) and + registers both the predefined and the custom recognizers for each of them. + """ + global SUPPORTED_LANGUAGES + try: + models = _resolve_language_models() + languages = [m["lang_code"] for m in models] + + # Presidio 2.x expects a list of {lang_code, model_name} + nlp_engine = SpacyNlpEngine(models=models) + nlp_engine.load() + + # The registry carries its own language list and Presidio rejects an + # analyzer whose languages differ from it; constructing the registry + # without this is what silently drops everything back to English. + registry = RecognizerRegistry(supported_languages=languages) + registry.load_predefined_recognizers(languages=languages, nlp_engine=nlp_engine) + + for language in languages: + for recognizer in _custom_recognizers(language): + registry.add_recognizer(recognizer) + + regions = PHONE_REGIONS_BY_LANGUAGE.get(language) + if regions: + registry.add_recognizer( + PhoneRecognizer( + supported_language=language, + supported_regions=tuple(regions), + ) + ) + + # Presidio ships CreditCardRecognizer for en/es only. A card number + # is language-independent — Luhn does not care what language the + # sentence around it is in — so register it everywhere. Without + # this, a hinted fr/de/zh request leaks 61.5% of card numbers + # (measured in bench/). + if "CREDIT_CARD" not in registry.get_supported_entities([language]): + registry.add_recognizer( + CreditCardRecognizer(supported_language=language) + ) analyzer = AnalyzerEngine( - registry=registry, nlp_engine=nlp_engine, supported_languages=["en"] + registry=registry, + nlp_engine=nlp_engine, + supported_languages=languages, ) anonymizer = AnonymizerEngine() + SUPPORTED_LANGUAGES = languages return analyzer, anonymizer except Exception as e: print(f"Failed to initialize Presidio: {e}") @@ -315,8 +448,42 @@ def init_presidio(): "PCI_EXPIRY": OperatorConfig("replace", {"new_value": ""}), "API_KEY": OperatorConfig("replace", {"new_value": "[REDACTED_API_KEY]"}), "JWT_TOKEN": OperatorConfig("replace", {"new_value": "[REDACTED_JWT]"}), + "PERSON": OperatorConfig("replace", {"new_value": ""}), + "LOCATION": OperatorConfig("replace", {"new_value": ""}), + "NRP": OperatorConfig("replace", {"new_value": ""}), } +# Entity types the analyzer is asked for. Kept separate from +# ANONYMIZE_OPERATORS because that mapping carries a "DEFAULT" key which is not +# an entity type, and because the two lists have different reasons to change. +# +# ORGANIZATION and DATE_TIME are deliberately excluded. Both are recognised by +# the spaCy pipeline but neither identifies a person on its own, and including +# them redacted 61.5% of PII-free control text in bench/ — company names, +# "quarterly", and "UTC" all match. Utility loss on that scale makes operators +# turn redaction off entirely, which is a worse outcome than the entities being +# left in place. +DETECT_ENTITIES = [ + "PERSON", + "LOCATION", + "NRP", + "EMAIL_ADDRESS", + "PHONE_NUMBER", + "CREDIT_CARD", + "IBAN_CODE", + "IP_ADDRESS", + "US_SSN", + "US_MEDICAL_RECORD_NUMBER", + "US_HEALTH_MEMBER_ID", + "US_NPI", + "US_DEA", + "US_DATE_OF_BIRTH", + "PCI_CVV", + "PCI_EXPIRY", + "API_KEY", + "JWT_TOKEN", +] + def entity_type_to_placeholder(entity_type: str) -> str: """Convert Presidio entity type to descriptive placeholder""" @@ -336,26 +503,119 @@ def entity_type_to_placeholder(entity_type: str) -> str: "IBAN_CODE": "", "API_KEY": "", "JWT_TOKEN": "", + "PERSON": "", + "LOCATION": "", + "NRP": "", } return entity_mapping.get(entity_type, f"") +def _languages_to_analyze(language: Optional[str]) -> List[str]: + """Which language pipelines to run for one request. + + A caller-supplied language always wins when it is supported. Otherwise the + behaviour depends on PRESIDIO_LANGUAGE_MODE: "hint" analyses in the first + configured language only, "union" analyses in all of them. + """ + supported = SUPPORTED_LANGUAGES or ["en"] + if language: + code = language.strip().lower() + if code in supported: + return [code] + + mode = getattr(settings, "presidio_language_mode", "hint") + if mode == "union": + return list(supported) + return supported[:1] + + +def _merge_results(results: List[Any]) -> List[Any]: + """Drop lower-confidence findings that overlap a higher-confidence one. + + Union mode analyses the same text several times, so the same span comes + back once per language. Presidio's anonymizer tolerates duplicates but + reason codes and counts would double, so collapse them here. + """ + ordered = sorted(results, key=lambda r: (-r.score, r.start, r.end)) + kept: List[Any] = [] + for candidate in ordered: + overlaps = any( + candidate.start < existing.end and existing.start < candidate.end + for existing in kept + ) + if not overlaps: + kept.append(candidate) + return sorted(kept, key=lambda r: r.start) + + +def language_hint( + tool_config: Optional[Dict] = None, policy_config: Optional[Dict] = None +) -> Optional[str]: + """Extract the caller's language hint, if any. + + SDK callers pass it as `tool_config.metadata.language`; a policy may also + pin a language for every request in an org. Neither is required — without a + hint, PRESIDIO_LANGUAGE_MODE decides. + """ + if isinstance(tool_config, dict): + metadata = tool_config.get("metadata") + if isinstance(metadata, dict): + hinted = metadata.get("language") + if isinstance(hinted, str) and hinted.strip(): + return hinted + if isinstance(policy_config, dict): + hinted = policy_config.get("language") + if isinstance(hinted, str) and hinted.strip(): + return hinted + return None + + +def analyze_text( + text: str, + language: Optional[str] = None, + entities: Optional[List[str]] = None, +) -> List[Any]: + """Run the analyzer across the resolved languages and merge the findings. + + Shared by the anonymization path and the two detection-only call sites so + that language handling cannot drift between them. + """ + if not USE_PRESIDIO or ANALYZER is None: + return [] + + ents = entities or DETECT_ENTITIES + collected: List[Any] = [] + for lang in _languages_to_analyze(language): + try: + collected.extend(ANALYZER.analyze(text=text, entities=ents, language=lang)) + except Exception as exc: + print(f"Presidio analyze failed for language {lang!r}: {exc}") + return _merge_results(collected) + + def anonymize_text_presidio( - text: str, field_name: str = "", entities: Optional[List[str]] = None + text: str, + field_name: str = "", + entities: Optional[List[str]] = None, + language: Optional[str] = None, ) -> Tuple[str, List[str]]: - """Anonymize text using Presidio""" + """Anonymize text using Presidio. + + `language` is the caller's hint (e.g. from tool_config.metadata). When it is + absent, PRESIDIO_LANGUAGE_MODE decides whether to analyse in the default + language only or in every configured language. + """ if not USE_PRESIDIO or ANALYZER is None: return text, [] - ents = entities or list(ANONYMIZE_OPERATORS.keys()) - results = ANALYZER.analyze(text=text, entities=ents, language="en") + results = analyze_text(text, language=language, entities=entities) if not results: return text, [] # Filter out false positives filtered_results = [] for r in results: - if not is_false_positive(r.entity_type, field_name, text): + if not is_false_positive(r.entity_type, field_name, text[r.start : r.end]): filtered_results.append(r) if not filtered_results: @@ -531,6 +791,79 @@ def is_false_positive(entity_type: str, field_name: str, value: str) -> bool: return False +# Tools denied unless an org's policy overrides the list. Names that no +# legitimate integration carries: each one is a direct code-execution primitive. +DEFAULT_DENY_TOOLS = [ + "python.exec", + "bash.exec", + "code.exec", + "shell.exec", + "subprocess.run", + "subprocess.popen", + "os.system", + "child_process.exec", + "child_process.spawn", + "runtime.exec", +] + +_TOOL_SEPARATORS = re.compile(r"[\s_\-/:]+") +_TOOL_DOT_RUNS = re.compile(r"\.+") + + +def normalize_tool_name(tool: str) -> str: + """Fold a tool name to a canonical dotted form. + + Case, surrounding whitespace, and the choice of separator are presentation + details, not identity: `Python.Exec`, `python_exec`, `python-exec` and + ` python.exec ` all name the same primitive. + """ + folded = _TOOL_SEPARATORS.sub(".", (tool or "").strip().lower()) + return _TOOL_DOT_RUNS.sub(".", folded).strip(".") + + +def is_denied_tool(tool: str, deny_tools: Optional[List[str]] = None) -> bool: + """Whether `tool` matches the denylist. + + Replaces an exact `tool in deny_tools` membership test, which any of the + following walked straight past: `Python.Exec` (case), `python_exec` + (separator), `exec.python` (component order), `agent.python.exec.v2` + (namespacing). Denial is a security boundary, so matching is deliberately + generous — a false deny is a support ticket, a false allow is an incident. + """ + normalized = normalize_tool_name(tool) + if not normalized: + return False + + entries = DEFAULT_DENY_TOOLS if deny_tools is None else deny_tools + components = set(normalized.split(".")) + padded = f".{normalized}." + + for raw_entry in entries: + entry = normalize_tool_name(raw_entry if isinstance(raw_entry, str) else "") + if not entry: + continue + + # Explicit wildcard: "shell.*" denies the whole namespace. + if entry.endswith(".*"): + prefix = entry[:-2] + if normalized == prefix or normalized.startswith(f"{prefix}."): + return True + continue + + if normalized == entry: + return True + + # Same components in a different order — exec.python vs python.exec. + if set(entry.split(".")) == components: + return True + + # Denied name appearing inside a longer namespaced path. + if f".{entry}." in padded: + return True + + return False + + def redact_obj( obj: Any, reasons: Optional[Set[str]] = None, field_name: str = "" ) -> Tuple[Any, Set[str]]: @@ -970,10 +1303,8 @@ def _evaluate_dynamic_policy( try: # PRECEDENCE LEVEL 1: Hard deny for dangerous tools - deny_tools = policy_config.get( - "deny_tools", ["python.exec", "bash.exec", "code.exec", "shell.exec"] - ) - if tool in deny_tools: + deny_tools = policy_config.get("deny_tools", DEFAULT_DENY_TOOLS) + if is_denied_tool(tool, deny_tools): return { "decision": "deny", "raw_text_out": raw_text, @@ -1103,11 +1434,11 @@ def _apply_tool_specific_policy_dynamic( # Run PII detection on raw text findings = [] if USE_PRESIDIO and ANALYZER is not None: - results = ANALYZER.analyze( - text=raw_text, entities=list(ANONYMIZE_OPERATORS.keys()), language="en" + results = analyze_text( + raw_text, language=language_hint(tool_config, policy_config) ) for r in results: - if not is_false_positive(r.entity_type, "", raw_text): + if not is_false_positive(r.entity_type, "", raw_text[r.start : r.end]): findings.append( { "type": f"PII:{r.entity_type.lower()}", @@ -1416,11 +1747,11 @@ def _apply_strict_fallback( # Detect standard PII types using Presidio or regex if USE_PRESIDIO and ANALYZER is not None: # Use Presidio to detect all standard PII types - results = ANALYZER.analyze( - text=raw_text, entities=list(ANONYMIZE_OPERATORS.keys()), language="en" + results = analyze_text( + raw_text, language=language_hint(tool_config, policy_config) ) for r in results: - if not is_false_positive(r.entity_type, "", raw_text): + if not is_false_positive(r.entity_type, "", raw_text[r.start : r.end]): all_findings.append( { "type": f"PII:{r.entity_type.lower()}", diff --git a/app/policy_source.py b/app/policy_source.py index f3a0952..fc761a2 100644 --- a/app/policy_source.py +++ b/app/policy_source.py @@ -11,6 +11,7 @@ already understands (precheck/app/policies.py), so the evaluator does not need to learn a new format — we just translate at the edge. """ + from __future__ import annotations import logging @@ -18,7 +19,7 @@ import threading import time from dataclasses import dataclass -from typing import Optional +from typing import Any, Dict, Optional from sqlalchemy.orm import Session @@ -55,13 +56,13 @@ class _CacheEntry: # shape `defaults[direction]["action"]` with action ∈ # {"deny", "redact", "tokenize", "pass_through"}. _PII_ACTION_MAP = { - "redact": "redact", - "block": "deny", - "deny": "deny", - "tokenize": "tokenize", - "pass": "pass_through", + "redact": "redact", + "block": "deny", + "deny": "deny", + "tokenize": "tokenize", + "pass": "pass_through", "pass_through": "pass_through", - "allow": "pass_through", + "allow": "pass_through", } @@ -72,7 +73,10 @@ def _map_row_to_policy_config(row: DashboardPolicy) -> dict: (precheck/app/policies.py). Unknown keys in `row.defaults` are preserved verbatim so future extensions don't need to touch this function. """ - raw_defaults = row.defaults or {} + # Annotated Any rather than Dict[str, Any]: `row.defaults` is a SQLAlchemy + # Column[Any] at type-check time, so a concrete dict annotation conflicts + # with the assignment even though the runtime value is a dict. + raw_defaults: Any = row.defaults or {} # Translate the v1 convention; default to "redact" if the field is absent. pii_action_in = str(raw_defaults.get("pii", "redact")).lower() @@ -82,20 +86,20 @@ def _map_row_to_policy_config(row: DashboardPolicy) -> dict: "version": row.version or "v1", "defaults": { "ingress": {"action": pii_action}, - "egress": {"action": pii_action}, + "egress": {"action": pii_action}, # Forward any non-pii defaults verbatim for forward-compat. **{k: v for k, v in raw_defaults.items() if k != "pii"}, }, - "tool_access": row.tool_access or {}, - "deny_tools": row.deny_tools or [], - "allow_tools": row.allow_tools or [], + "tool_access": row.tool_access or {}, + "deny_tools": row.deny_tools or [], + "allow_tools": row.allow_tools or [], "network_scopes": row.network_scopes or [], - "network_tools": row.network_tools or [], - "on_error": row.on_error or "block", + "network_tools": row.network_tools or [], + "on_error": row.on_error or "block", # Provenance — useful in logs and audit, ignored by the evaluator. - "_policy_id": row.id, - "_policy_name": row.name, - "_priority": row.priority, + "_policy_id": row.id, + "_policy_name": row.name, + "_priority": row.priority, } @@ -111,7 +115,9 @@ def _fetch_from_db(org_id: str) -> Optional[dict]: DashboardPolicy.org_id == org_id, DashboardPolicy.is_active.is_(True), ) - .order_by(DashboardPolicy.priority.desc(), DashboardPolicy.updated_at.desc()) + .order_by( + DashboardPolicy.priority.desc(), DashboardPolicy.updated_at.desc() + ) .first() ) if row is None: @@ -120,9 +126,7 @@ def _fetch_from_db(org_id: str) -> Optional[dict]: except Exception as exc: # Don't blow up the request path on a DB hiccup — let the caller fall # back to the YAML policy. Logged loudly so it's visible in audits. - logger.warning( - "policy_source: db fetch failed for org=%s err=%s", org_id, exc - ) + logger.warning("policy_source: db fetch failed for org=%s err=%s", org_id, exc) return None finally: db.close() diff --git a/app/settings.py b/app/settings.py index 73131a8..84392c6 100644 --- a/app/settings.py +++ b/app/settings.py @@ -18,6 +18,9 @@ # - "local": per-replica in-memory fallback. Intended for single-replica dev. _RATE_LIMIT_FAIL_MODES = {"closed", "open", "local"} +# Allowed values for PRESIDIO_LANGUAGE_MODE. See Settings.presidio_language_mode. +_PRESIDIO_LANGUAGE_MODES = {"hint", "union"} + class Settings(BaseSettings): """Application settings loaded from environment variables""" @@ -57,7 +60,23 @@ class Settings(BaseSettings): # Presidio configuration use_presidio: bool = True - presidio_model: str = "en_core_web_sm" # sm, md, lg + presidio_model: str = "en_core_web_sm" # English model override: sm, md, lg + + # Languages the analyzer is built for, comma-separated. Each entry needs its + # spaCy model present in the image (see Dockerfile). Adding a language here + # without its model installed degrades to the remaining languages rather + # than failing the service. + presidio_languages: str = "en" + + # How the language for a given request is chosen when more than one is + # configured: + # - "hint": use the caller-supplied language, else the first configured one. + # Cheapest; wrong hint means that request is analysed in the + # wrong language and its PII is missed. + # - "union": analyse in every configured language and union the findings. + # Recall-safe default for governance; costs roughly N x the NLP + # time, so it is opt-in rather than automatic. + presidio_language_mode: str = "hint" # API configuration — demo_api_key intentionally removed; all keys must live in DB api_key_header: str = "X-Governs-Key" @@ -88,6 +107,15 @@ class Settings(BaseSettings): # Policy file configuration policy_file: str = "policy.tool_access.yaml" + def presidio_language_list(self) -> list[str]: + """Configured analyzer languages, normalised and de-duplicated.""" + seen: list[str] = [] + for raw in self.presidio_languages.split(","): + code = raw.strip().lower() + if code and code not in seen: + seen.append(code) + return seen or ["en"] + @model_validator(mode="after") def _reject_default_secrets(self) -> "Settings": if self.rate_limit_fail_mode not in _RATE_LIMIT_FAIL_MODES: @@ -95,6 +123,11 @@ def _reject_default_secrets(self) -> "Settings": f"RATE_LIMIT_FAIL_MODE must be one of {sorted(_RATE_LIMIT_FAIL_MODES)}; " f"got {self.rate_limit_fail_mode!r}." ) + if self.presidio_language_mode not in _PRESIDIO_LANGUAGE_MODES: + raise ValueError( + f"PRESIDIO_LANGUAGE_MODE must be one of {sorted(_PRESIDIO_LANGUAGE_MODES)}; " + f"got {self.presidio_language_mode!r}." + ) if not self.debug: self._validate_secret( name="PII_TOKEN_SALT", diff --git a/app/storage.py b/app/storage.py index f34dc1a..0abb3da 100644 --- a/app/storage.py +++ b/app/storage.py @@ -149,9 +149,7 @@ def create_tables(): `DashboardPolicy` is intentionally excluded — that table is owned and migrated by the dashboard (Prisma). Precheck only reads it. See ADR-005. """ - owned_tables = [ - t for t in Base.metadata.sorted_tables if t.name != "Policy" - ] + owned_tables = [t for t in Base.metadata.sorted_tables if t.name != "Policy"] Base.metadata.create_all(bind=engine, tables=owned_tables) diff --git a/bench/README.md b/bench/README.md new file mode 100644 index 0000000..ae248f7 --- /dev/null +++ b/bench/README.md @@ -0,0 +1,92 @@ +# PII detection benchmark + +Measures what precheck's redaction path actually catches, as opposed to what +the detector it wraps scores in isolation. Published detector benchmarks +evaluate Presidio as a standalone model on text records; this one runs the +pipeline as deployed — entity allowlist, language configuration, false-positive +filter and all — because that is what decides whether a customer's SSN leaves +the building. + +## Running it + +```bash +# from precheck/ +venv/bin/python -m bench.run # all arms, English only +PRESIDIO_LANGUAGES=en,es,fr,de,zh venv/bin/python -m bench.run --arms regex legacy fixed +venv/bin/python -m bench.run --json results.json # machine-readable +venv/bin/python -m bench.corpus # write corpus.jsonl and print counts +``` + +Non-English arms need the matching spaCy models: + +```bash +venv/bin/python -m spacy download es_core_news_sm # and fr_, de_, zh_ +``` + +## Corpus + +438 items, 630 span-annotated entities, five languages, generated +deterministically from `SEED = 20260811`. Ground truth is recorded at +generation time — templates carry `{SLOT}` markers and the renderer stores each +filled value's exact offsets — so there is no annotation step and no ambiguity. +Every credit card is Luhn-valid, so a detector that gates on the checksum is not +penalised for rejecting a number that was never a card. + +Tiers: `plain`, `high_sensitivity` (names, addresses, medical and national +identifiers), `partial` (last-4, initials, year-only DOB), `obfuscated` (spaced, +spelled-out), `structured` (PII nested in JSON tool arguments at depth 1–4), and +`clean` (no PII at all — the over-redaction control). + +## Arms + +| Arm | What it is | +|---|---| +| `regex` | `anonymize_text_regex` — the no-Presidio fallback | +| `legacy` | The pre-fix shipped path: English-only, 16-entity allowlist, whole-text false-positive filter. Entity list frozen in `run.py` so this keeps measuring what shipped | +| `presidio` | Current entry point, no caller hint | +| `fixed` | Current entry point with the corpus language as hint — what an SDK caller does via `tool_config.metadata.language` | +| `fixed_nohint` | Current entry point, no hint — exercises `PRESIDIO_LANGUAGE_MODE` | +| `presidio_unrestricted` | Diagnostic: same model, no entity allowlist. The gap to `legacy` is what the allowlist alone was discarding | + +## Metrics + +`leakage` is the headline: the fraction of ground-truth PII values that survive +verbatim in the output. Per value, not per span-overlap — an operator cares +whether the SSN left, not how many characters matched. `over_redact` is the +utility counterweight, measured on the clean tier: a layer that redacts +everything has zero leakage and zero value, and operators switch it off. + +## Results + +Five languages, `PRESIDIO_LANGUAGE_MODE=hint`, 438 items: + +| Arm | leakage | high-sensitivity | over-redaction | p50 | +|---|---|---|---|---| +| `regex` | 52.2% | 89.1% | 0.0% | 0.01 ms | +| `legacy` | 51.3% | 89.1% | 0.0% | 2.46 ms | +| `fixed` | **8.2%** | **10.6%** | 0.0% | 2.48 ms | + +PERSON and LOCATION went from 100% leakage to 11.6% and 11.7%. The +high-sensitivity figure is the one to watch: it covers the entity families that +regulated customers are buying redaction for, and 89.1% of them used to pass +through untouched. + +## Known residual gaps + +These are measured, not hypothetical, and none is fixed: + +- **Obfuscated forms — 75% leakage.** `j dot smith at acme dot com`, digits + spelled out, characters spaced. Pattern recognizers see nothing. +- **Partial disclosure — 83.3%.** "the card ending 0366", "initials M.G., born + 1982". Arguably identifying in combination; no detector in the pipeline + models combinations. +- **Chinese — 32.1%**, well above the 1.8–4.5% of the European languages. + `zh_core_web_sm` is the weakest NER model of the five. +- **US_SSN — 14.3%**, unchanged across every arm: the recognizer requires + context words that some templates do not supply. +- **IBAN — 8.3%.** + +The obfuscated and partial numbers match the published failure profile for +pattern-plus-NER detectors (REDACT, arXiv:2606.19881, reports 0.07 and 0.02 +recall respectively for Presidio). Closing them needs a different class of +detector — an LLM pass or a purpose-built encoder — not a better regex. diff --git a/bench/__init__.py b/bench/__init__.py new file mode 100644 index 0000000..e69de29 diff --git a/bench/corpus.py b/bench/corpus.py new file mode 100644 index 0000000..63523e4 --- /dev/null +++ b/bench/corpus.py @@ -0,0 +1,558 @@ +# SPDX-License-Identifier: MIT +# Copyright (c) 2024 GovernsAI. All rights reserved. +"""Span-annotated PII corpus generator for the precheck detection benchmark. + +Ground truth is produced at generation time: templates carry `{slot}` markers, +and the renderer records the exact character offsets each slot occupies in the +rendered string. No post-hoc annotation, no human labelling, no ambiguity. + +Design notes +------------ +* Deterministic — a fixed seed produces byte-identical corpora, so before/after + runs are directly comparable. +* No new runtime dependencies. Value pools are hand-built per locale rather than + pulled from Faker so that the generator stays inside the existing dependency + set and locale realism is explicit and reviewable. +* Difficulty tiers mirror the axes on which off-the-shelf detectors are known to + degrade (REDACT, arXiv:2606.19881): high-sensitivity entity families, partial + disclosure, and obfuscated surface forms. +""" + +from __future__ import annotations + +import json +import random +import re +from dataclasses import asdict, dataclass, field +from typing import Any, Dict, Iterable, List, Optional, Tuple + +SEED = 20260811 + +LANGUAGES = ["en", "es", "fr", "de", "zh"] + +TIERS = [ + "plain", # standard prose, well-formed entities + "high_sensitivity", # names, addresses, medical + national identifiers + "partial", # partial disclosure (last-4, initials, year-only DOB) + "obfuscated", # spaced / spelled-out / separator-mangled surface forms + "structured", # PII inside nested JSON tool arguments + "clean", # no PII at all — over-redaction control +] + + +@dataclass +class Span: + start: int + end: int + type: str + value: str + + +@dataclass +class Item: + id: str + lang: str + tier: str + text: str + spans: List[Span] = field(default_factory=list) + tool: str = "chat" + + def to_json(self) -> Dict[str, Any]: + d = asdict(self) + return d + + +# --------------------------------------------------------------------------- +# Value pools +# --------------------------------------------------------------------------- + +PERSONS = { + "en": [ + "John Smith", + "Maria Garcia", + "Robert Johnson", + "Sarah Chen", + "David Miller", + ], + "es": [ + "Juan Martínez", + "Lucía Fernández", + "Carlos Ruiz", + "Ana Torres", + "Miguel Ortega", + ], + "fr": [ + "Jean Dupont", + "Marie Lefèvre", + "Pierre Moreau", + "Claire Rousseau", + "Luc Bernard", + ], + "de": [ + "Hans Müller", + "Anna Schmidt", + "Peter Wagner", + "Julia Becker", + "Thomas Fischer", + ], + "zh": ["张伟", "王芳", "李娜", "刘强", "陈静"], +} + +LOCATIONS = { + "en": [ + "1600 Pennsylvania Avenue, Washington DC", + "42 Baker Street, London", + "500 Market St, San Francisco", + ], + "es": [ + "Calle Gran Vía 28, Madrid", + "Avenida Diagonal 405, Barcelona", + "Plaza Mayor 7, Salamanca", + ], + "fr": [ + "12 Rue de Rivoli, Paris", + "3 Avenue Jean Médecin, Nice", + "8 Quai Saint-Vincent, Lyon", + ], + "de": [ + "Unter den Linden 5, Berlin", + "Maximilianstraße 13, München", + "Reeperbahn 22, Hamburg", + ], + "zh": [ + "北京市朝阳区建国路88号", + "上海市浦东新区世纪大道100号", + "广州市天河区天河路299号", + ], +} + +ORG_DOMAINS = ["acme.com", "globex.io", "initech.co", "umbrella-health.org"] + +EMAIL_LOCALS = ["j.smith", "m.garcia", "contact", "a.torres", "h.mueller"] + +PHONES = { + "en": ["415-555-0132", "(202) 555-0177", "+1 415 555 0199"], + "es": ["+34 612 34 56 78", "912 345 678"], + "fr": ["+33 6 12 34 56 78", "01 42 68 53 00"], + "de": ["+49 151 23456789", "030 123456"], + "zh": ["+86 138 0013 8000", "010-12345678"], +} + +# All Luhn-valid: a detector that gates on the checksum (as Presidio does) must +# not be penalised for rejecting a number that was never a card in the first place. +CARDS = ["4532015112830366", "5425233430109903", "4111 1111 1111 1111"] + +IBANS = [ + "DE89370400440532013000", + "FR1420041010050500013M02606", + "ES9121000418450200051332", +] + +SSNS = ["123-45-6789", "078-05-1120"] + +IPS = ["192.168.14.203", "10.0.42.7", "203.0.113.51"] + +# Synthetic, and deliberately test-prefixed rather than live-prefixed: a +# live-prefixed literal trips GitHub push protection even when the value is +# fabricated. The detection pattern covers both prefixes, so the fixture still +# exercises the vendor-prefixed-with-internal-underscore shape. +API_KEYS = [ + "sk_test_BENCHFIXTURE0000000000000", + "AKIABENCHFIXTUREEXAMPLE0", +] + +JWTS = [ + "eyJhbGciOiJIUzI1NiJ9.eyJzdWIiOiIxMjM0NTY3ODkwIn0.dozjgNryP4J3jVmNHl0w5N_XgL0n3I9PlFUP0THsR8U" +] + +MRNS = ["MRN: 4457812", "Medical Record Number: A-9928134"] +NPIS = ["NPI: 1234567893"] +DEAS = ["DEA: BM1234563"] +MEMBER_IDS = ["Member ID: HX-88213904"] +DOBS = ["DOB: 04/17/1982", "Date of Birth: 11/02/1975"] +CVVS = ["CVV: 831"] +EXPIRIES = ["exp: 04/28"] + +# Templates: {slot} markers are replaced and their offsets recorded. +TEMPLATES: Dict[str, Dict[str, List[str]]] = { + "en": { + "plain": [ + "Please email {EMAIL_ADDRESS} about the invoice.", + "Call me back on {PHONE_NUMBER} after 3pm.", + "Charge the card {CREDIT_CARD} for the annual plan.", + "The server at {IP_ADDRESS} is refusing connections.", + "Wire the retainer to {IBAN_CODE} by Friday.", + "Our staging key is {API_KEY} — rotate it tonight.", + "Auth header carries {JWT_TOKEN} on every call.", + "His social is {US_SSN}, needed for the I-9.", + ], + "high_sensitivity": [ + "Patient {PERSON} lives at {LOCATION} and is due for a follow-up.", + "Dr. {PERSON} prescribed medication for patient {PERSON}.", + "Ship the settlement cheque to {PERSON} at {LOCATION}.", + "{PERSON} filed the complaint; contact them at {EMAIL_ADDRESS}.", + "Chart for {PERSON} — {US_MEDICAL_RECORD_NUMBER}, {US_DATE_OF_BIRTH}.", + "Prescriber {PERSON}, {US_NPI}, {US_DEA}.", + "Claim submitted by {PERSON} under {US_HEALTH_MEMBER_ID}.", + ], + "partial": [ + "The card ending in 0366 belongs to J. S. — confirm before charging.", + "Patient initials M.G., born 1982, seen last Tuesday.", + "Reach the account holder on the number ending 0132.", + "SSN last four is 6789 for the applicant from Ohio.", + ], + "obfuscated": [ + "Reach him at j dot smith at acme dot com when you can.", + "Contact: j . s m i t h @ a c m e . c o m", + "Card number four five three two 0151 1283 0366.", + "Phone: four one five, five five five, zero one three two.", + ], + "clean": [ + "The deployment finished at 14:32 UTC with no errors.", + "Version 4.2.1 ships the new retry policy for webhooks.", + "Acme Corporation reported quarterly revenue above forecast.", + "Ticket 123-45 was closed as a duplicate of 678-90.", + "The build agent uses 8 vCPUs and 32 GB of memory.", + ], + }, + "es": { + "plain": [ + "Por favor escribe a {EMAIL_ADDRESS} sobre la factura.", + "Llámame al {PHONE_NUMBER} después de las tres.", + "Cobra la tarjeta {CREDIT_CARD} del plan anual.", + "Transfiere el anticipo a {IBAN_CODE} antes del viernes.", + ], + "high_sensitivity": [ + "El paciente {PERSON} vive en {LOCATION} y necesita revisión.", + "La doctora {PERSON} atendió al paciente {PERSON} esta mañana.", + "Envíen el cheque a {PERSON} en {LOCATION}.", + "{PERSON} presentó la reclamación; su correo es {EMAIL_ADDRESS}.", + ], + "partial": [ + "La tarjeta terminada en 0366 pertenece a J. M. — confirmar antes de cobrar.", + "Paciente con iniciales L. F., nacida en 1982.", + ], + "obfuscated": [ + "Su correo es j punto smith arroba acme punto com.", + "Teléfono: seis uno dos, tres cuatro, cinco seis.", + ], + "clean": [ + "El despliegue terminó a las 14:32 UTC sin errores.", + "La versión 4.2.1 incluye la nueva política de reintentos.", + ], + }, + "fr": { + "plain": [ + "Merci d'écrire à {EMAIL_ADDRESS} au sujet de la facture.", + "Rappelez-moi au {PHONE_NUMBER} après 15h.", + "Débitez la carte {CREDIT_CARD} pour l'abonnement annuel.", + "Virez les honoraires sur {IBAN_CODE} avant vendredi.", + ], + "high_sensitivity": [ + "Le patient {PERSON} habite au {LOCATION} et doit être revu.", + "Le docteur {PERSON} a soigné le patient {PERSON} ce matin.", + "Envoyez le chèque à {PERSON} au {LOCATION}.", + "{PERSON} a déposé la plainte ; son courriel est {EMAIL_ADDRESS}.", + ], + "partial": [ + "La carte se terminant par 0366 appartient à J. D. — à confirmer.", + "Patiente aux initiales M. L., née en 1982.", + ], + "obfuscated": [ + "Son adresse est j point smith arobase acme point com.", + "Téléphone : zéro six, douze, trente-quatre.", + ], + "clean": [ + "Le déploiement s'est terminé à 14h32 UTC sans erreur.", + "La version 4.2.1 ajoute la politique de relance.", + ], + }, + "de": { + "plain": [ + "Bitte schreiben Sie an {EMAIL_ADDRESS} wegen der Rechnung.", + "Rufen Sie mich unter {PHONE_NUMBER} nach 15 Uhr zurück.", + "Belasten Sie die Karte {CREDIT_CARD} für den Jahresplan.", + "Überweisen Sie das Honorar auf {IBAN_CODE} bis Freitag.", + ], + "high_sensitivity": [ + "Der Patient {PERSON} wohnt in {LOCATION} und braucht eine Nachuntersuchung.", + "Doktor {PERSON} behandelte heute den Patienten {PERSON}.", + "Senden Sie den Scheck an {PERSON} in {LOCATION}.", + "{PERSON} hat die Beschwerde eingereicht; E-Mail: {EMAIL_ADDRESS}.", + ], + "partial": [ + "Die Karte endend auf 0366 gehört H. M. — bitte bestätigen.", + "Patientin mit den Initialen A. S., geboren 1982.", + ], + "obfuscated": [ + "Seine Adresse ist j punkt smith at acme punkt com.", + "Telefon: null eins fünf eins, zwei drei vier.", + ], + "clean": [ + "Das Deployment endete um 14:32 UTC ohne Fehler.", + "Version 4.2.1 bringt die neue Wiederholungsrichtlinie.", + ], + }, + "zh": { + "plain": [ + "请发邮件到 {EMAIL_ADDRESS} 询问发票事宜。", + "下午三点后请拨打 {PHONE_NUMBER} 联系我。", + "请用银行卡 {CREDIT_CARD} 支付年费。", + ], + "high_sensitivity": [ + "患者 {PERSON} 住在 {LOCATION}，需要复诊。", + "{PERSON} 医生今天为患者 {PERSON} 看诊。", + "请把支票寄给 {PERSON}，地址是 {LOCATION}。", + "{PERSON} 提交了投诉，邮箱是 {EMAIL_ADDRESS}。", + ], + "partial": [ + "尾号 0366 的卡属于张先生，扣款前请确认。", + "患者姓名缩写 W.F.，1982 年出生。", + ], + "obfuscated": [ + "他的邮箱是 j 点 smith 艾特 acme 点 com。", + "电话：一三八 零零一三 八零零零。", + ], + "clean": [ + "部署于 14:32 UTC 完成，没有报错。", + "4.2.1 版本加入了新的重试策略。", + ], + }, +} + +SLOT_POOLS = { + "PERSON": lambda rng, lang: rng.choice(PERSONS[lang]), + "LOCATION": lambda rng, lang: rng.choice(LOCATIONS[lang]), + "EMAIL_ADDRESS": lambda rng, lang: f"{rng.choice(EMAIL_LOCALS)}@{rng.choice(ORG_DOMAINS)}", + "PHONE_NUMBER": lambda rng, lang: rng.choice(PHONES.get(lang, PHONES["en"])), + "CREDIT_CARD": lambda rng, lang: rng.choice(CARDS), + "IBAN_CODE": lambda rng, lang: rng.choice(IBANS), + "US_SSN": lambda rng, lang: rng.choice(SSNS), + "IP_ADDRESS": lambda rng, lang: rng.choice(IPS), + "API_KEY": lambda rng, lang: rng.choice(API_KEYS), + "JWT_TOKEN": lambda rng, lang: rng.choice(JWTS), + "US_MEDICAL_RECORD_NUMBER": lambda rng, lang: rng.choice(MRNS), + "US_NPI": lambda rng, lang: rng.choice(NPIS), + "US_DEA": lambda rng, lang: rng.choice(DEAS), + "US_HEALTH_MEMBER_ID": lambda rng, lang: rng.choice(MEMBER_IDS), + "US_DATE_OF_BIRTH": lambda rng, lang: rng.choice(DOBS), + "PCI_CVV": lambda rng, lang: rng.choice(CVVS), + "PCI_EXPIRY": lambda rng, lang: rng.choice(EXPIRIES), +} + +SLOT_RE = re.compile(r"\{([A-Z_]+)\}") + +# Entity families, used to report high-sensitivity recall separately. +HIGH_SENSITIVITY = { + "PERSON", + "LOCATION", + "US_SSN", + "US_MEDICAL_RECORD_NUMBER", + "US_NPI", + "US_DEA", + "US_HEALTH_MEMBER_ID", + "US_DATE_OF_BIRTH", +} + + +def render(template: str, rng: random.Random, lang: str) -> Tuple[str, List[Span]]: + """Fill {SLOT} markers, recording exact offsets of each filled value.""" + out: List[str] = [] + spans: List[Span] = [] + pos = 0 + cursor = 0 + for m in SLOT_RE.finditer(template): + literal = template[cursor : m.start()] + out.append(literal) + pos += len(literal) + + entity = m.group(1) + value = SLOT_POOLS[entity](rng, lang) + spans.append(Span(start=pos, end=pos + len(value), type=entity, value=value)) + out.append(value) + pos += len(value) + cursor = m.end() + + tail = template[cursor:] + out.append(tail) + return "".join(out), spans + + +def _structured_items( + rng: random.Random, lang: str, n: int, start_idx: int +) -> List[Item]: + """PII nested inside JSON tool arguments at varying depth.""" + items: List[Item] = [] + for i in range(n): + person = rng.choice(PERSONS[lang]) + email = f"{rng.choice(EMAIL_LOCALS)}@{rng.choice(ORG_DOMAINS)}" + card = rng.choice(CARDS) + depth = (i % 3) + 1 + + leaf: Dict[str, Any] = { + "customer_name": person, + "contact_email": email, + "payment": {"card_number": card}, + } + payload: Dict[str, Any] = leaf + for d in range(depth): + payload = {f"level_{d}": payload} + + text = json.dumps(payload, ensure_ascii=False) + spans: List[Span] = [] + for value, etype in ( + (person, "PERSON"), + (email, "EMAIL_ADDRESS"), + (card, "CREDIT_CARD"), + ): + idx = text.find(value) + if idx >= 0: + spans.append( + Span(start=idx, end=idx + len(value), type=etype, value=value) + ) + + items.append( + Item( + id=f"{lang}-structured-{start_idx + i:04d}", + lang=lang, + tier="structured", + text=text, + spans=spans, + tool="db.query", + ) + ) + return items + + +# Partial and obfuscated templates carry their PII inline rather than in slots; +# these tables give the ground-truth values the detector is expected to catch. +INLINE_TRUTH: Dict[str, List[Tuple[str, str]]] = { + "The card ending in 0366 belongs to J. S. — confirm before charging.": [ + ("0366", "CREDIT_CARD") + ], + "Patient initials M.G., born 1982, seen last Tuesday.": [("M.G.", "PERSON")], + "Reach the account holder on the number ending 0132.": [("0132", "PHONE_NUMBER")], + "SSN last four is 6789 for the applicant from Ohio.": [("6789", "US_SSN")], + "Reach him at j dot smith at acme dot com when you can.": [ + ("j dot smith at acme dot com", "EMAIL_ADDRESS") + ], + "Contact: j . s m i t h @ a c m e . c o m": [ + ("j . s m i t h @ a c m e . c o m", "EMAIL_ADDRESS") + ], + "Card number four five three two 0151 1283 0366.": [ + ("four five three two 0151 1283 0366", "CREDIT_CARD") + ], + "Phone: four one five, five five five, zero one three two.": [ + ("four one five, five five five, zero one three two", "PHONE_NUMBER") + ], + "La tarjeta terminada en 0366 pertenece a J. M. — confirmar antes de cobrar.": [ + ("0366", "CREDIT_CARD") + ], + "Paciente con iniciales L. F., nacida en 1982.": [("L. F.", "PERSON")], + "Su correo es j punto smith arroba acme punto com.": [ + ("j punto smith arroba acme punto com", "EMAIL_ADDRESS") + ], + "Teléfono: seis uno dos, tres cuatro, cinco seis.": [ + ("seis uno dos, tres cuatro, cinco seis", "PHONE_NUMBER") + ], + "La carte se terminant par 0366 appartient à J. D. — à confirmer.": [ + ("0366", "CREDIT_CARD") + ], + "Patiente aux initiales M. L., née en 1982.": [("M. L.", "PERSON")], + "Son adresse est j point smith arobase acme point com.": [ + ("j point smith arobase acme point com", "EMAIL_ADDRESS") + ], + "Téléphone : zéro six, douze, trente-quatre.": [ + ("zéro six, douze, trente-quatre", "PHONE_NUMBER") + ], + "Die Karte endend auf 0366 gehört H. M. — bitte bestätigen.": [ + ("0366", "CREDIT_CARD") + ], + "Patientin mit den Initialen A. S., geboren 1982.": [("A. S.", "PERSON")], + "Seine Adresse ist j punkt smith at acme punkt com.": [ + ("j punkt smith at acme punkt com", "EMAIL_ADDRESS") + ], + "Telefon: null eins fünf eins, zwei drei vier.": [ + ("null eins fünf eins, zwei drei vier", "PHONE_NUMBER") + ], + "尾号 0366 的卡属于张先生，扣款前请确认。": [("0366", "CREDIT_CARD")], + "患者姓名缩写 W.F.，1982 年出生。": [("W.F.", "PERSON")], + "他的邮箱是 j 点 smith 艾特 acme 点 com。": [ + ("j 点 smith 艾特 acme 点 com", "EMAIL_ADDRESS") + ], + "电话：一三八 零零一三 八零零零。": [("一三八 零零一三 八零零零", "PHONE_NUMBER")], +} + + +def build( + per_template: int = 6, languages: Optional[Iterable[str]] = None +) -> List[Item]: + """Generate the corpus. Deterministic for a fixed SEED.""" + rng = random.Random(SEED) + langs = list(languages or LANGUAGES) + items: List[Item] = [] + counter = 0 + + for lang in langs: + tmpl_by_tier = TEMPLATES[lang] + for tier in ["plain", "high_sensitivity", "clean"]: + for template in tmpl_by_tier.get(tier, []): + for _ in range(per_template): + text, spans = render(template, rng, lang) + items.append( + Item( + id=f"{lang}-{tier}-{counter:04d}", + lang=lang, + tier=tier, + text=text, + spans=spans, + ) + ) + counter += 1 + + for tier in ["partial", "obfuscated"]: + for template in tmpl_by_tier.get(tier, []): + truth = INLINE_TRUTH.get(template, []) + spans = [] + for value, etype in truth: + idx = template.find(value) + if idx >= 0: + spans.append( + Span( + start=idx, end=idx + len(value), type=etype, value=value + ) + ) + items.append( + Item( + id=f"{lang}-{tier}-{counter:04d}", + lang=lang, + tier=tier, + text=template, + spans=spans, + ) + ) + counter += 1 + + items.extend(_structured_items(rng, lang, per_template * 2, counter)) + counter += per_template * 2 + + return items + + +def write_jsonl(items: List[Item], path: str) -> None: + with open(path, "w", encoding="utf-8") as fh: + for it in items: + fh.write(json.dumps(it.to_json(), ensure_ascii=False) + "\n") + + +if __name__ == "__main__": # pragma: no cover - manual invocation + corpus = build() + write_jsonl(corpus, "bench/corpus.jsonl") + by_tier: Dict[str, int] = {} + by_lang: Dict[str, int] = {} + for it in corpus: + by_tier[it.tier] = by_tier.get(it.tier, 0) + 1 + by_lang[it.lang] = by_lang.get(it.lang, 0) + 1 + print(f"{len(corpus)} items") + print("by tier:", by_tier) + print("by lang:", by_lang) diff --git a/bench/run.py b/bench/run.py new file mode 100644 index 0000000..c058c01 --- /dev/null +++ b/bench/run.py @@ -0,0 +1,358 @@ +# SPDX-License-Identifier: MIT +# Copyright (c) 2024 GovernsAI. All rights reserved. +"""Run the PII detection benchmark against precheck's redaction paths. + +Usage (from precheck/): + venv/bin/python -m bench.run # all arms, all languages + venv/bin/python -m bench.run --arms regex # single arm + venv/bin/python -m bench.run --json out.json # machine-readable report + +Metrics +------- +leakage fraction of ground-truth PII values that survive verbatim in the + output. This is the safety metric: one surviving value is a + disclosure regardless of how many others were caught. +item_leakage fraction of items where at least one value survived. +over_redact fraction of clean-tier items (no PII) that were modified anyway. +latency_ms per-call wall clock, p50/p95. + +Leakage is deliberately the headline rather than span F1: an operator cares +whether the SSN left the building, not how many characters overlapped. +""" + +from __future__ import annotations + +import argparse +import json +import statistics +import sys +import time +from collections import defaultdict +from typing import Callable, Dict, List, Optional, Tuple + +from bench.corpus import HIGH_SENSITIVITY, Item, build + +Arm = Callable[[str, str], Tuple[str, List[str]]] + +# The entity allowlist the pre-fix code passed to Presidio: the keys of +# ANONYMIZE_OPERATORS as they stood before PERSON/LOCATION/NRP were added. +# Frozen here so the `legacy` arm keeps measuring what shipped even as the +# application constant moves on. +LEGACY_ENTITIES = [ + "DEFAULT", + "CREDIT_CARD", + "PHONE_NUMBER", + "EMAIL_ADDRESS", + "IP_ADDRESS", + "IBAN_CODE", + "US_SSN", + "US_MEDICAL_RECORD_NUMBER", + "US_HEALTH_MEMBER_ID", + "US_NPI", + "US_DEA", + "US_DATE_OF_BIRTH", + "PCI_CVV", + "PCI_EXPIRY", + "API_KEY", + "JWT_TOKEN", +] + + +# --------------------------------------------------------------------------- +# Arms +# --------------------------------------------------------------------------- + + +def arm_regex(text: str, lang: str) -> Tuple[str, List[str]]: + from app.policies import anonymize_text_regex + + return anonymize_text_regex(text) + + +def arm_legacy(text: str, lang: str) -> Tuple[str, List[str]]: + """Reproduces the pre-fix shipped path: English-only, allowlist-constrained. + + Kept so before/after numbers come out of a single run rather than being + compared across two trees. + """ + from presidio_anonymizer.entities import OperatorConfig + + from app.policies import ( + ANALYZER, + ANONYMIZER, + entity_type_to_placeholder, + is_false_positive, + ) + + if ANALYZER is None: + return text, [] + try: + results = ANALYZER.analyze(text=text, entities=LEGACY_ENTITIES, language="en") + except Exception: + return text, [] + # The pre-fix code passed the whole text to the false-positive filter + # rather than the matched span, so the filter almost never fired. Preserved + # here deliberately: this arm documents what shipped, not what should have. + results = [r for r in results if not is_false_positive(r.entity_type, "", text)] + if not results: + return text, [] + ops = { + r.entity_type: OperatorConfig( + "replace", {"new_value": entity_type_to_placeholder(r.entity_type)} + ) + for r in results + } + out = ANONYMIZER.anonymize(text=text, analyzer_results=results, operators=ops).text + return out, sorted({f"pii.redacted:{r.entity_type.lower()}" for r in results}) + + +def arm_presidio(text: str, lang: str) -> Tuple[str, List[str]]: + """Current shipped entry point with no caller hint.""" + from app.policies import anonymize_text_presidio + + return anonymize_text_presidio(text) + + +def arm_presidio_unrestricted(text: str, lang: str) -> Tuple[str, List[str]]: + """Diagnostic: same model, no entity allowlist, still English-pinned. + + The delta between this and `presidio` is the cost of the entity allowlist + alone — i.e. detection the loaded model already performs and the pipeline + then discards. + """ + from presidio_anonymizer.entities import OperatorConfig + + from app.policies import ANALYZER, ANONYMIZER, entity_type_to_placeholder + + if ANALYZER is None: + return text, [] + results = ANALYZER.analyze(text=text, language="en") + if not results: + return text, [] + ops = { + r.entity_type: OperatorConfig( + "replace", {"new_value": entity_type_to_placeholder(r.entity_type)} + ) + for r in results + } + out = ANONYMIZER.anonymize(text=text, analyzer_results=results, operators=ops).text + return out, sorted({f"pii.redacted:{r.entity_type.lower()}" for r in results}) + + +def arm_precheck_fixed(text: str, lang: str) -> Tuple[str, List[str]]: + """Post-fix path: multi-language analyzer + widened entity set. + + Passes the corpus language as the caller hint, which is what an SDK caller + does via tool_config.metadata.language. + """ + from app.policies import anonymize_text_presidio + + return anonymize_text_presidio(text, language=lang) + + +def arm_precheck_fixed_nohint(text: str, lang: str) -> Tuple[str, List[str]]: + """Post-fix path with no language hint — exercises PRESIDIO_LANGUAGE_MODE.""" + from app.policies import anonymize_text_presidio + + return anonymize_text_presidio(text) + + +ARMS: Dict[str, Arm] = { + "regex": arm_regex, + "legacy": arm_legacy, + "presidio": arm_presidio, + "presidio_unrestricted": arm_presidio_unrestricted, + "fixed": arm_precheck_fixed, + "fixed_nohint": arm_precheck_fixed_nohint, +} + + +# --------------------------------------------------------------------------- +# Scoring +# --------------------------------------------------------------------------- + + +class Accumulator: + def __init__(self) -> None: + self.spans_total = 0 + self.spans_leaked = 0 + self.items_total = 0 + self.items_leaked = 0 + self.clean_total = 0 + self.clean_modified = 0 + self.latencies: List[float] = [] + self.by_entity: Dict[str, List[int]] = defaultdict( + lambda: [0, 0] + ) # [leaked, total] + + def rate(self, num: int, den: int) -> Optional[float]: + return round(num / den, 4) if den else None + + def summary(self) -> Dict[str, object]: + lat = sorted(self.latencies) + p50 = round(statistics.median(lat), 2) if lat else None + p95 = round(lat[int(len(lat) * 0.95)], 2) if len(lat) >= 20 else None + return { + "spans": self.spans_total, + "leakage": self.rate(self.spans_leaked, self.spans_total), + "item_leakage": self.rate(self.items_leaked, self.items_total), + "over_redact": self.rate(self.clean_modified, self.clean_total), + "latency_p50_ms": p50, + "latency_p95_ms": p95, + } + + +def score_item(item: Item, out: str) -> Tuple[int, int, List[str]]: + """Return (leaked_spans, total_spans, leaked_entity_types).""" + leaked = 0 + types: List[str] = [] + for span in item.spans: + if span.value and span.value in out: + leaked += 1 + types.append(span.type) + return leaked, len(item.spans), types + + +def run_arm(name: str, arm: Arm, items: List[Item]) -> Dict[str, object]: + overall = Accumulator() + by_lang: Dict[str, Accumulator] = defaultdict(Accumulator) + by_tier: Dict[str, Accumulator] = defaultdict(Accumulator) + high_sens = Accumulator() + + for item in items: + t0 = time.perf_counter() + try: + out, _reasons = arm(item.text, item.lang) + except Exception as exc: # an arm that crashes leaks everything + print( + f" ! {name} raised on {item.id}: {type(exc).__name__}: {exc}", + file=sys.stderr, + ) + out = item.text + elapsed_ms = (time.perf_counter() - t0) * 1000 + + leaked, total, leaked_types = score_item(item, out) + + for acc in (overall, by_lang[item.lang], by_tier[item.tier]): + acc.latencies.append(elapsed_ms) + acc.spans_total += total + acc.spans_leaked += leaked + acc.items_total += 1 + if leaked: + acc.items_leaked += 1 + if item.tier == "clean": + acc.clean_total += 1 + if out != item.text: + acc.clean_modified += 1 + + for span in item.spans: + etype = span.type + overall.by_entity[etype][1] += 1 + if span.value in out: + overall.by_entity[etype][0] += 1 + if etype in HIGH_SENSITIVITY: + high_sens.spans_total += 1 + if span.value in out: + high_sens.spans_leaked += 1 + + return { + "overall": overall.summary(), + "high_sensitivity_leakage": overall.rate( + high_sens.spans_leaked, high_sens.spans_total + ), + "by_lang": {k: v.summary() for k, v in sorted(by_lang.items())}, + "by_tier": {k: v.summary() for k, v in sorted(by_tier.items())}, + "by_entity": { + k: { + "leaked": v[0], + "total": v[1], + "leakage": round(v[0] / v[1], 4) if v[1] else None, + } + for k, v in sorted(overall.by_entity.items()) + }, + } + + +def fmt_pct(v: Optional[float]) -> str: + return " n/a" if v is None else f"{v * 100:5.1f}%" + + +def print_report(results: Dict[str, Dict[str, object]]) -> None: + arms = list(results.keys()) + + print("\n=== Overall (leakage = ground-truth PII values surviving verbatim) ===\n") + print( + f"{'arm':<24}{'leakage':>9}{'item_leak':>11}{'high_sens':>11}{'over_redact':>13}{'p50 ms':>9}{'p95 ms':>9}" + ) + for arm in arms: + o = results[arm]["overall"] + print( + f"{arm:<24}{fmt_pct(o['leakage']):>9}{fmt_pct(o['item_leakage']):>11}" + f"{fmt_pct(results[arm]['high_sensitivity_leakage']):>11}" + f"{fmt_pct(o['over_redact']):>13}" + f"{str(o['latency_p50_ms']):>9}{str(o['latency_p95_ms']):>9}" + ) + + print("\n=== Leakage by language ===\n") + langs = sorted({lang for arm in arms for lang in results[arm]["by_lang"]}) + print(f"{'arm':<24}" + "".join(f"{lang:>9}" for lang in langs)) + for arm in arms: + row = "".join( + fmt_pct(results[arm]["by_lang"].get(lang, {}).get("leakage")) + for lang in langs + ) + print(f"{arm:<24}{row}") + + print("\n=== Leakage by tier ===\n") + tiers = sorted({t for arm in arms for t in results[arm]["by_tier"]}) + print(f"{'arm':<24}" + "".join(f"{t[:9]:>18}" for t in tiers)) + for arm in arms: + row = "".join( + f"{fmt_pct(results[arm]['by_tier'].get(t, {}).get('leakage')):>18}" + for t in tiers + ) + print(f"{arm:<24}{row}") + + print("\n=== Leakage by entity type ===\n") + ents = sorted({e for arm in arms for e in results[arm]["by_entity"]}) + print(f"{'entity':<28}" + "".join(f"{a[:16]:>22}" for a in arms)) + for ent in ents: + row = "" + for arm in arms: + cell = results[arm]["by_entity"].get(ent) + row += f"{fmt_pct(cell['leakage']) if cell else ' n/a':>22}" + print(f"{ent:<28}{row}") + + +def main() -> int: + ap = argparse.ArgumentParser() + ap.add_argument("--arms", nargs="*", default=list(ARMS.keys())) + ap.add_argument("--languages", nargs="*", default=None) + ap.add_argument("--per-template", type=int, default=6) + ap.add_argument("--json", dest="json_out", default=None) + args = ap.parse_args() + + items = build(per_template=args.per_template, languages=args.languages) + print( + f"corpus: {len(items)} items, {sum(len(i.spans) for i in items)} annotated spans" + ) + + results: Dict[str, Dict[str, object]] = {} + for name in args.arms: + if name not in ARMS: + print(f"unknown arm: {name}", file=sys.stderr) + return 2 + print(f"running arm: {name} ...", flush=True) + results[name] = run_arm(name, ARMS[name], items) + + print_report(results) + + if args.json_out: + with open(args.json_out, "w", encoding="utf-8") as fh: + json.dump(results, fh, indent=2) + print(f"\nwrote {args.json_out}") + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/env.example b/env.example index dd5752f..999f41e 100644 --- a/env.example +++ b/env.example @@ -18,8 +18,26 @@ PRECHECK_ALLOW_CACHE_TTL_SECONDS=60 # Presidio Configuration USE_PRESIDIO=true +# English spaCy model (sm | md | lg). Larger models raise PERSON/LOCATION +# recall at the cost of memory and load time. PRESIDIO_MODEL=en_core_web_sm +# Languages the analyzer is built for, comma-separated. Each needs its spaCy +# model present in the image (the Dockerfile installs en, es, fr, de, zh). +# A language listed here without its model is skipped with a log line rather +# than failing startup. +PRESIDIO_LANGUAGES=en + +# How the language is chosen per request when several are configured: +# hint - use the caller's tool_config.metadata.language, else the first +# configured language. One NLP pass; a wrong hint misses that +# request's PII. +# union - analyse in every configured language and merge findings. Recall-safe +# but roughly N x the NLP time, and non-native NER models produce +# false positives on clean text (53.8% of PII-free control text was +# modified in bench/ at five languages). Opt in deliberately. +PRESIDIO_LANGUAGE_MODE=hint + # API Configuration API_KEY_HEADER=X-Governs-Key # API_KEY=your-api-key-here # Optional: API key from .env (fallback if not in header) diff --git a/tests/test_deny_tools_matching.py b/tests/test_deny_tools_matching.py new file mode 100644 index 0000000..76d1fc0 --- /dev/null +++ b/tests/test_deny_tools_matching.py @@ -0,0 +1,119 @@ +# SPDX-License-Identifier: MIT +# Copyright (c) 2024 GovernsAI. All rights reserved. +"""TEST — denylist matching for dangerous tools. + +The previous implementation was `if tool in deny_tools`, an exact string +membership test. Every case in TestBypassVariants walked past it while naming +the same code-execution primitive. Denial is a security boundary: these tests +pin the matcher's generosity so a later refactor cannot quietly narrow it. +""" + +import pytest + +from app.policies import DEFAULT_DENY_TOOLS, is_denied_tool, normalize_tool_name + + +class TestNormalizeToolName: + @pytest.mark.parametrize( + "raw,expected", + [ + ("python.exec", "python.exec"), + ("Python.Exec", "python.exec"), + ("PYTHON.EXEC", "python.exec"), + (" python.exec ", "python.exec"), + ("python_exec", "python.exec"), + ("python-exec", "python.exec"), + ("python/exec", "python.exec"), + ("python:exec", "python.exec"), + ("python..exec", "python.exec"), + (".python.exec.", "python.exec"), + ("", ""), + ], + ) + def test_folds_to_canonical_form(self, raw, expected): + assert normalize_tool_name(raw) == expected + + def test_handles_none(self): + assert normalize_tool_name(None) == "" + + +class TestBypassVariants: + """Each of these was allowed by the exact-match implementation.""" + + @pytest.mark.parametrize( + "tool", + [ + "Python.Exec", + "PYTHON.EXEC", + " python.exec ", + "python_exec", + "python-exec", + "python/exec", + "shell/exec", + "Bash.Exec", + "exec.python", # component order + "exec.shell", + "agent.python.exec.v2", # namespaced + "tools.bash.exec", + ], + ) + def test_denied(self, tool): + assert is_denied_tool(tool) is True + + @pytest.mark.parametrize( + "tool", + [ + "subprocess.run", + "subprocess.popen", + "os.system", + "child_process.spawn", + "runtime.exec", + ], + ) + def test_semantically_equivalent_primitives_denied(self, tool): + """Names that are code execution by any other spelling.""" + assert is_denied_tool(tool) is True + + +class TestLegitimateToolsStillAllowed: + @pytest.mark.parametrize( + "tool", + [ + "chat", + "db.query", + "web.search", + "http.get", + "python.format", # not exec + "exec_summary", # substring of "exec", not the primitive + "executive.report", + "search.python.docs", + "", + ], + ) + def test_allowed(self, tool): + assert is_denied_tool(tool) is False + + def test_empty_denylist_allows_everything(self): + assert is_denied_tool("python.exec", []) is False + + +class TestCustomDenylists: + def test_operator_supplied_list_is_normalized_too(self): + assert is_denied_tool("Danger_Tool", ["danger.tool"]) is True + + def test_wildcard_denies_namespace(self): + assert is_denied_tool("shell.anything", ["shell.*"]) is True + assert is_denied_tool("shell", ["shell.*"]) is True + + def test_wildcard_does_not_leak_across_prefixes(self): + assert is_denied_tool("shelly.thing", ["shell.*"]) is False + + def test_non_string_entries_are_skipped(self): + assert is_denied_tool("python.exec", [None, 42, "python.exec"]) is True + assert is_denied_tool("chat", [None, 42]) is False + + +class TestDefaults: + def test_legacy_defaults_retained(self): + for tool in ["python.exec", "bash.exec", "code.exec", "shell.exec"]: + assert tool in DEFAULT_DENY_TOOLS diff --git a/tests/test_multilingual_pii.py b/tests/test_multilingual_pii.py new file mode 100644 index 0000000..1418a49 --- /dev/null +++ b/tests/test_multilingual_pii.py @@ -0,0 +1,176 @@ +# SPDX-License-Identifier: MIT +# Copyright (c) 2024 GovernsAI. All rights reserved. +"""TEST — language handling and entity coverage in the Presidio path. + +Two defects these tests pin down: + +1. `ANALYZER.analyze(..., language="en")` was hardcoded at every call site, so + the es/fr/de/zh spaCy models the image installs were never reachable. +2. The entity allowlist passed to the analyzer omitted PERSON, LOCATION and + NRP, so names and addresses were detected by the NER model and then dropped + before redaction. `bench/` measured 100% leakage on both, in English too. + +Tests that need non-English models are marked `multilingual` and skip when the +models are absent; run them inside the precheck image. +""" + +import importlib.util + +import pytest + +from app import policies +from app.policies import ( + DETECT_ENTITIES, + anonymize_text_presidio, + language_hint, +) +from app.settings import Settings + + +def _has_model(name: str) -> bool: + return importlib.util.find_spec(name) is not None + + +requires_presidio = pytest.mark.skipif( + policies.ANALYZER is None, + reason="Presidio analyzer unavailable in this environment", +) + + +class TestEntityCoverage: + """The allowlist must carry the entity types a redaction policy implies.""" + + @pytest.mark.parametrize("entity", ["PERSON", "LOCATION", "NRP"]) + def test_identity_entities_are_requested(self, entity): + assert entity in DETECT_ENTITIES + + @pytest.mark.parametrize("entity", ["ORGANIZATION", "DATE_TIME"]) + def test_low_precision_entities_excluded(self, entity): + """Both wreck PII-free text: 'Acme Corporation', 'quarterly', 'UTC'.""" + assert entity not in DETECT_ENTITIES + + def test_default_is_not_an_entity_type(self): + """The old code passed ANONYMIZE_OPERATORS keys, which include DEFAULT.""" + assert "DEFAULT" not in DETECT_ENTITIES + + +@requires_presidio +class TestPersonAndLocationRedaction: + def test_person_name_redacted(self): + out, reasons = anonymize_text_presidio( + "Patient Maria Garcia is due for a follow-up.", language="en" + ) + assert "Maria Garcia" not in out + assert any("person" in r for r in reasons) + + def test_location_redacted(self): + out, _ = anonymize_text_presidio( + "Ship the cheque to 1600 Pennsylvania Avenue, Washington DC.", language="en" + ) + assert "Washington DC" not in out + + def test_clean_text_survives(self): + """Utility control: no PII in, nothing redacted out.""" + text = "Acme Corporation reported quarterly revenue above forecast." + out, reasons = anonymize_text_presidio(text, language="en") + assert out == text + assert reasons == [] + + +@requires_presidio +class TestLanguageResolution: + def test_unknown_hint_falls_back_to_supported_language(self): + out, _ = anonymize_text_presidio( + "Patient Maria Garcia is due for a follow-up.", language="xx" + ) + assert "Maria Garcia" not in out + + def test_supported_languages_populated(self): + assert policies.SUPPORTED_LANGUAGES + assert "en" in policies.SUPPORTED_LANGUAGES + + +class TestLanguageHint: + def test_reads_tool_config_metadata(self): + assert language_hint({"metadata": {"language": "es"}}, None) == "es" + + def test_falls_back_to_policy_config(self): + assert language_hint(None, {"language": "fr"}) == "fr" + + def test_tool_config_wins(self): + assert ( + language_hint({"metadata": {"language": "es"}}, {"language": "fr"}) == "es" + ) + + @pytest.mark.parametrize( + "tool_config,policy_config", + [ + (None, None), + ({}, {}), + ({"metadata": {}}, {}), + ({"metadata": {"language": " "}}, {}), + ({"metadata": {"language": 42}}, {}), + ("not-a-dict", None), + ], + ) + def test_absent_or_malformed_hint_is_none(self, tool_config, policy_config): + assert language_hint(tool_config, policy_config) is None + + +class TestSettings: + def test_language_list_parsed_and_deduped(self): + s = Settings(presidio_languages="en, es ,EN,fr", debug=True) + assert s.presidio_language_list() == ["en", "es", "fr"] + + def test_empty_language_list_defaults_to_english(self): + assert Settings( + presidio_languages=" , ", debug=True + ).presidio_language_list() == ["en"] + + def test_invalid_language_mode_rejected(self): + with pytest.raises(ValueError, match="PRESIDIO_LANGUAGE_MODE"): + Settings(presidio_language_mode="sometimes", debug=True) + + @pytest.mark.parametrize("mode", ["hint", "union"]) + def test_valid_language_modes_accepted(self, mode): + assert ( + Settings(presidio_language_mode=mode, debug=True).presidio_language_mode + == mode + ) + + +@pytest.mark.multilingual +class TestNonEnglishRedaction: + """Requires the es/fr/de spaCy models; run inside the precheck image.""" + + @pytest.mark.parametrize( + "lang,model,text,secret", + [ + ( + "es", + "es_core_news_sm", + "El paciente Juan Martínez necesita revisión.", + "Juan Martínez", + ), + ( + "fr", + "fr_core_news_sm", + "Le patient Jean Dupont doit être revu.", + "Jean Dupont", + ), + ( + "de", + "de_core_news_sm", + "Der Patient Hans Müller braucht eine Untersuchung.", + "Hans Müller", + ), + ], + ) + def test_person_redacted_in_language(self, lang, model, text, secret): + if not _has_model(model): + pytest.skip(f"{model} not installed; run inside the precheck image") + if lang not in policies.SUPPORTED_LANGUAGES: + pytest.skip(f"analyzer not built for {lang}; set PRESIDIO_LANGUAGES") + + out, _ = anonymize_text_presidio(text, language=lang) + assert secret not in out diff --git a/tests/test_policy_invalidate_endpoint.py b/tests/test_policy_invalidate_endpoint.py index 255853b..afb0b7b 100644 --- a/tests/test_policy_invalidate_endpoint.py +++ b/tests/test_policy_invalidate_endpoint.py @@ -1,5 +1,6 @@ # SPDX-License-Identifier: MIT """HMAC-gated /api/v1/internal/policy/invalidate endpoint tests (ADR-005).""" + from __future__ import annotations import hashlib diff --git a/tests/test_policy_source.py b/tests/test_policy_source.py index d0ad0e9..34b6642 100644 --- a/tests/test_policy_source.py +++ b/tests/test_policy_source.py @@ -4,6 +4,7 @@ See ADR-005 for design. Tests cover cache hit/miss/expiry, missing-org, priority ordering, the dashboard-shape → precheck-shape translation, and invalidate(). """ + from __future__ import annotations import time @@ -117,8 +118,12 @@ def test_unknown_pii_action_falls_back_to_redact(db_session): # ─── priority order ─────────────────────────────────────────────────────── def test_highest_priority_active_policy_wins(db_session): - _make_policy(db_session, org_id="org-9", name="lo", defaults={"pii": "redact"}, priority=1) - _make_policy(db_session, org_id="org-9", name="hi", defaults={"pii": "block"}, priority=10) + _make_policy( + db_session, org_id="org-9", name="lo", defaults={"pii": "redact"}, priority=1 + ) + _make_policy( + db_session, org_id="org-9", name="hi", defaults={"pii": "block"}, priority=10 + ) cfg = policy_source.get_active_policy("org-9") # priority=10 ("block") must beat priority=1 ("redact") assert cfg["defaults"]["ingress"]["action"] == "deny" @@ -126,8 +131,22 @@ def test_highest_priority_active_policy_wins(db_session): def test_inactive_policy_ignored(db_session): - _make_policy(db_session, org_id="org-10", name="dead", defaults={"pii": "block"}, priority=999, is_active=False) - _make_policy(db_session, org_id="org-10", name="live", defaults={"pii": "redact"}, priority=1, is_active=True) + _make_policy( + db_session, + org_id="org-10", + name="dead", + defaults={"pii": "block"}, + priority=999, + is_active=False, + ) + _make_policy( + db_session, + org_id="org-10", + name="live", + defaults={"pii": "redact"}, + priority=1, + is_active=True, + ) cfg = policy_source.get_active_policy("org-10") assert cfg["_policy_name"] == "live" @@ -159,13 +178,19 @@ def test_invalidate_forces_refetch(db_session): assert first["defaults"]["ingress"]["action"] == "redact" # mutate the underlying row - row = db_session.query(DashboardPolicy).filter(DashboardPolicy.org_id == "org-i").one() + row = ( + db_session.query(DashboardPolicy) + .filter(DashboardPolicy.org_id == "org-i") + .one() + ) row.defaults = {"pii": "block"} db_session.commit() # without invalidation, the cache should still serve the old value cached = policy_source.get_active_policy("org-i") - assert cached["defaults"]["ingress"]["action"] == "redact", "cache should still hold stale entry" + assert ( + cached["defaults"]["ingress"]["action"] == "redact" + ), "cache should still hold stale entry" # invalidate → next call refetches and reflects the change policy_source.invalidate("org-i") @@ -189,7 +214,11 @@ def test_ttl_expiry_forces_refetch(db_session, monkeypatch): monkeypatch.setattr(time, "monotonic", lambda: base + 5) # mutate the row; without a hit-spy, the new value proves a refetch happened - row = db_session.query(DashboardPolicy).filter(DashboardPolicy.org_id == "org-t").one() + row = ( + db_session.query(DashboardPolicy) + .filter(DashboardPolicy.org_id == "org-t") + .one() + ) row.defaults = {"pii": "block"} db_session.commit()