From 0e51c2e7f76206e6b6424da8972499b05ceee041 Mon Sep 17 00:00:00 2001 From: Greg Anderson Date: Fri, 21 Aug 2026 18:10:19 -0600 Subject: [PATCH 1/2] docs: 3.3.x upgrade notes for the three deduplication identity changes The 3.3 upgrade page said "no special instructions", which this release makes wrong three times over: Xeol Parser, Checkmarx One Scan and Checkmarx Scan detailed all gain HASHCODE_FIELDS_PER_SCANNER registrations, so findings imported before the upgrade carry hashes that no longer match what an import computes after it. Follows the shape the 3.2 notes set: what changed and why per scan type, then two paths. Instances running the identity signature ledger (default from Pro 3.2.300) need no action because drift detection, the scoped rehash and signature matching bridge the change; the one caveat spelled out is that a large never-backfilled instance should run the backfill before upgrading, since a rehash replaces the identities the backfill exists to record. Everyone else gets the three manage.py dedupe commands in the same form the 2.4x and 3.2 notes use. --- docs/content/releases/os_upgrading/3.3.md | 46 ++++++++++++++++++++++- 1 file changed, 44 insertions(+), 2 deletions(-) diff --git a/docs/content/releases/os_upgrading/3.3.md b/docs/content/releases/os_upgrading/3.3.md index c857f839097..db22aae9351 100644 --- a/docs/content/releases/os_upgrading/3.3.md +++ b/docs/content/releases/os_upgrading/3.3.md @@ -2,6 +2,48 @@ title: 'Upgrading to DefectDojo Version 3.3.x' toc_hide: true weight: -20260803 -description: No special instructions. +description: Three parsers gain deduplication registrations, which changes the stored identity of their findings. Xeol Parser moves off the legacy hash so its identity stops depending on the wall clock; Checkmarx One Scan hashes the vendor id it already matches on; Checkmarx Scan detailed gains the same field list as its CxFlow sibling. Instances with the identity signature ledger enabled bridge these changes automatically; others should rehash the affected scan types after upgrading. --- -There are no special instructions for upgrading to 3.3.x. Check the [Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/3.3.0) for the contents of the release. + +## Deduplication identity changes + +Three scan types gain `HASHCODE_FIELDS_PER_SCANNER` registrations in this release. A +registration changes how `hash_code` is computed, so findings imported by these parsers +before the upgrade carry a hash that no longer matches what an import computes after it. +Left alone, the next re-import of an unchanged report would treat every affected finding +as new, close the stored one as absent, and recreate it, detaching false positive +history and risk acceptances along the way. + +| Scan type | New hash fields | Why | +|---|---|---| +| `Xeol Parser` | `title`, `component_name`, `component_version` | Its severity is derived from the current date against the component's EOL date, and the legacy hash meant finding identity moved on its own as time passed. Severity still escalates as an EOL date recedes; it just no longer decides what the finding is. | +| `Checkmarx One Scan` | `unique_id_from_tool` | The parser already deduplicates on the vendor id. The stored hash previously fell through to the legacy field set, whose title and description are the same volatile text for every result family, so the stored identity moved whenever Checkmarx reworded a finding even though matching did not. | +| `Checkmarx Scan detailed` | `vuln_id_from_tool`, `file_path`, `line` | The same three fields as its sibling `Checkmarx CxFlow SAST`, which the parser populates on every detailed-mode finding. None of them carries scan text. | + +### If the identity signature ledger is enabled + +DefectDojo Pro 3.2.300 and later enable the identity signature ledger by default +(`identity_signature_dual_write` and `identity_signature_matching`). On these instances +no action is needed: drift detection notices the definition change, the scoped rehash +recomputes the stored hashes, and signature matching bridges imports that still carry +the old identity during the transition. The nightly backfill sweep closes small ledger +gaps on its own; a large never-backfilled instance logs a notice instead, and should run +`python manage.py identity_signatures_backfill` in a maintenance window, ideally +**before** this upgrade, because the backfill records the identities findings currently +have and a rehash replaces them. + +### If it is not + +Open-source installs, and Pro installs where the ledger was explicitly disabled, should +rehash the affected scan types after upgrading so stored hashes match what imports now +compute: + +```bash +docker compose exec uwsgi /bin/bash -c "python manage.py dedupe --parser 'Xeol Parser' --hash_code_only" +docker compose exec uwsgi /bin/bash -c "python manage.py dedupe --parser 'Checkmarx One Scan' --hash_code_only" +docker compose exec uwsgi /bin/bash -c "python manage.py dedupe --parser 'Checkmarx Scan detailed' --hash_code_only" +``` + +Installs that never imported these scan types are unaffected. Check the +[Release Notes](https://github.com/DefectDojo/django-DefectDojo/releases/tag/3.3.0) for +the full contents of the release. From fa943f4db1d61bc3f50f3493380d095d2e6c1edb Mon Sep 17 00:00:00 2001 From: Greg Anderson Date: Fri, 21 Aug 2026 18:31:34 -0600 Subject: [PATCH 2/2] docs: the drift watch reports, it does not repair The Pro section overpromised twice. It claimed both ledger flags default on in 3.2.300, and matching is deliberately opt-in. It also claimed the definition change is repaired automatically, but identity_drift_watch notifies and stops there; the repair is the operator accepting the change or running the same dedupe commands everyone else uses. Rewritten so the Pro path is honest: recording is on by default, the drift watch will tell you, and the fix is the same three commands or a Tuner acceptance. The backfill-before-upgrade guidance stays, scoped to instances where matching is enabled, because that is where the preserved identities are actually consulted. --- docs/content/releases/os_upgrading/3.3.md | 38 +++++++++++++---------- 1 file changed, 22 insertions(+), 16 deletions(-) diff --git a/docs/content/releases/os_upgrading/3.3.md b/docs/content/releases/os_upgrading/3.3.md index db22aae9351..44b1dbc98aa 100644 --- a/docs/content/releases/os_upgrading/3.3.md +++ b/docs/content/releases/os_upgrading/3.3.md @@ -20,22 +20,28 @@ history and risk acceptances along the way. | `Checkmarx One Scan` | `unique_id_from_tool` | The parser already deduplicates on the vendor id. The stored hash previously fell through to the legacy field set, whose title and description are the same volatile text for every result family, so the stored identity moved whenever Checkmarx reworded a finding even though matching did not. | | `Checkmarx Scan detailed` | `vuln_id_from_tool`, `file_path`, `line` | The same three fields as its sibling `Checkmarx CxFlow SAST`, which the parser populates on every detailed-mode finding. None of them carries scan text. | -### If the identity signature ledger is enabled - -DefectDojo Pro 3.2.300 and later enable the identity signature ledger by default -(`identity_signature_dual_write` and `identity_signature_matching`). On these instances -no action is needed: drift detection notices the definition change, the scoped rehash -recomputes the stored hashes, and signature matching bridges imports that still carry -the old identity during the transition. The nightly backfill sweep closes small ledger -gaps on its own; a large never-backfilled instance logs a notice instead, and should run -`python manage.py identity_signatures_backfill` in a maintenance window, ideally -**before** this upgrade, because the backfill records the identities findings currently -have and a rehash replaces them. - -### If it is not - -Open-source installs, and Pro installs where the ledger was explicitly disabled, should -rehash the affected scan types after upgrading so stored hashes match what imports now +### DefectDojo Pro + +Pro 3.2.300 and later record identity signatures by default +(`identity_signature_dual_write`), and the nightly drift watch will notice these +definition changes and notify. The notification is a report, not a repair: recompute the +stored hashes with the same three commands below, or accept the definition change in the +Tuner, which runs the equivalent scoped rehash. + +Two Pro-specific notes: + +* If your instance has a signature ledger worth keeping (it has been recording for a + while, or you ran a backfill), run `python manage.py identity_signatures_backfill` in a + maintenance window **before** this upgrade. The backfill records the identities + findings currently have, and a rehash replaces them; done in this order, the previous + identities stay in the ledger and remain matchable during the transition on instances + where signature matching is enabled. +* `identity_signature_matching` remains opt-in per instance. Where it is enabled, imports + still carrying the old identity bridge through the ledger during the transition. + +### Open source, and Pro instances that skip the rehash + +Rehash the affected scan types after upgrading so stored hashes match what imports now compute: ```bash