Skip to content

feat(scan): persist + propagate the scanner's trust/legitimacy signal (it's computed then dropped) #1743

Description

@abankar1

Problem

scan.mjs computes a legitimacy/trust signal for every scanned job and then throws it away.

  • scan.mjs:1133–1136: trustValidator(job) runs before any filter and sets job.trustScore (0–100), job.trustFlags (e.g. missing_apply_url, invalid_url, suspicious_domain), and job.trustLevel (high/medium/low).
  • But that's the end of its life: it's used only for a console summary at scan time (scan.mjs:1319–1380). Neither formatPipelineOffer() nor formatScanHistoryRow() includes it, so it never reaches data/pipeline.md or data/scan-history.tsv.
  • And Block G (Posting Legitimacy) in modes/oferta.md / modes/_shared.md has no idea these flags existgrep -i "trust\|missing_apply_url\|suspicious_domain" modes/oferta.md → nothing.

So the moment the scan output scrolls away, the signal is gone. When the user later runs /career-ops pipeline to evaluate the queued URLs, the evaluation re-derives legitimacy from scratch (tokens) — or misses signals the free scanner already caught (a posting with no apply URL, or a bit.ly-shortened apply link, is a classic ghost/scam tell).

This is the same "compute-then-drop" pattern as #1578 (postedAt), but for the scam/ghost-job dimension — arguably higher user value, since it protects people from wasting an application (and a recruiter's time) on a fake posting.

Proposed solution (mirror the #1578 mechanism)

  1. Persist it — same shape scan.mjs drops providers' postedAt — posting dates never reach pipeline.md / scan-history.tsv #1578 uses for the posting date:
    • scan-history.tsv: trailing trust_score + trust_flags (pipe-joined) columns. Trailing, so existing readers keying off fixed indices are unaffected.
    • pipeline.md: a labeled (not positional) trust: segment, e.g. … | trust: 60 missing_apply_url,suspicious_domain. Omitted (byte-identical output) when the posting is clean/high — exactly like note: / posted:.
  2. Consume it in Block G — teach oferta.md Block G to read the scanner's flags as prior evidence rather than re-deriving them, so a missing_apply_url / suspicious_domain posting starts from a lower legitimacy tier.
  3. (Optional phase 2, opt-in) — a scan.quarantine_below threshold that routes very-low-trust postings (e.g. invalid apply URL) to a separate list instead of the eval queue, so they never consume an evaluation. Pure token savings + fewer scam applications. Default off.

Composes cleanly with #1578 (same labeled-segment plumbing; I'd sequence phase 1 after that lands to reuse it). Happy to bring the persistence half as a self-contained PR first (backward-compatible, like #1578); the Block G wording touches a shared system file so I'll want your steer there. Flagging per CONTRIBUTING before code. Searched open issues/PRs — nothing covers persisting the scanner trust signal.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions