You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
feat(api-v3): FindingSlim carries vulnerability_ids and cwes
Architect directive: a finding's identity fields must be present on
lists without per-row detail fetches. FindingSlim (and Detail via
inheritance) gains:
- vulnerability_ids: list[str] - flat strings in storage order (first
= the cve mirror), NOT v2's object-wrapper list; symmetric with what
FindingWrite accepts
- cwes: list[int] - parsed from Finding_CWE rows (reliably CWE-<n>
format via save_cwes), primary first; int chosen for consistency
with the scalar cwe field
Both prefetch-backed (reverse accessors vulnerability_id_set /
finding_cwe_set): +2 fixed in-batch queries on findings lists, pins
updated deliberately (offset 5->7, cursor 4->6, offset-1==cursor
relation preserved). Relations verified excluded from the defer set.
CSV gains two semicolon-joined columns; examples regenerated; write
asymmetry (no cwes list on writes) recorded as backlog bullet.
476 tests green + 6 skipped.
Copy file name to clipboardExpand all lines: API_V3_PLAN.md
+4-1Lines changed: 4 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -132,7 +132,8 @@ Every list response:
132
132
### 4.5 Slim shapes (starting field lists)
133
133
Slim = identity + primary status fields + parent refs + timestamps. **No per-row computed fields** (no `age`, no SLA math, no counts requiring extra queries — `locations_count` is a queryset annotation, which is allowed). Implementer: verify each field exists on the model before wiring; adjust only by removing, not adding, without a §12 entry.
-`cwes` (flat `list[int]`) and `vulnerability_ids` (flat `list[str]`) are identity-critical and always present on slim (no per-row detail fetch); populated by two fixed `prefetch_related` paths (`finding_cwe_set`, `vulnerability_id_set`). See §12 (2026-07-21).
136
137
-`engagement/product/product_type` are denormalized parent refs populated by one `select_related("test__engagement__product__prod_type")` — no extra queries.
@@ -355,6 +356,7 @@ layer and the closed contract, so all are additive:
355
356
- [ ] delete-time `push_to_jira` param, `configuration_permissions` on user
356
357
writes, v3 self-profile endpoint, filter vocabularies for the location-edge sub-resources
357
358
(PUT full replace was pulled forward out of this bundle and shipped — §12, 2026-07-20)
359
+
- [ ] **Accept a `cwes`list on finding writes** — the read side exposes `cwes` (flat `list[int]`), but writes still accept only the scalar `cwe` (+`vulnerability_ids`); make finding create/update accept a `cwes`list symmetric with the read shape (§12, 2026-07-21)
358
360
-**Port the v2 endpoint-level test corpora to v3** (architect-requested). Priority order:
359
361
- [x] **(1) the import/reimport corpus** (`test_import_reimport.py`'s mixin, `test_apiv2_scan_import_options.py`,
360
362
`test_importers_closeold.py`) — DONE via the dual-endpoint adapter `unittests/api_v3/import_corpus_shim.py`
@@ -563,3 +565,4 @@ tracked as a §6 backlog checkbox — so nothing was implemented here (conservat
563
565
| 2026-07-21 | **V2 IMPORT/REIMPORT TEST CORPUS PORTED TO v3 (§10 backlog #1; dual-endpoint adapter, no copy).** The v2 import corpus now proves the *same* DB behaviour against the consolidated `POST /api/v3-alpha/import`. **Adapter (tests only):** `unittests/api_v3/import_corpus_shim.py::ApiV3ImportShim` overrides ONLY the two v2 helper methods `import_scan_with_params` / `reimport_scan_with_params` — mapping v2 wire→v3 wire (`product_name`→`asset_name`, `product_type_name`→`organization_name`, import/reimport→one POST with `mode=import|reimport|auto`; `reimport` with `test_id=None`+auto-create fields→`mode=auto`; when `asset_name` is present the numeric `engagement` default is dropped so the v3 consistency check does not trip) and translating the v3 response `{mode_resolved, test:{id,name}, statistics, close_old_findings}`→the v2-shaped `{"test": <id>, "statistics": ...}` the corpus reads. The endpoint→location count redirect (`db_endpoint_count`/`db_endpoint_status_count`) and `block_execution` synchronous-dedupe setup are inherited unchanged from `ImportReimportTestAPI` (they are already `V3_FEATURE_LOCATIONS`-aware). **Modules:** `test_apiv3_import_corpus.py` (subclasses the v2 `ImportReimportTestAPI` via a module attribute so `test*.py` discovery does not re-collect the v2 class — 71 cases: mixin 61 [`@parameterized.expand`→6] + API-only 10) and `test_apiv3_import_options_corpus.py` (2 ported `ScanImportOptionsTest` scenarios via a one-method `import_zap_scan` override + 4 endpoint-level ports of `test_importers_closeold.py`). **ZERO v2 test files modified** (the narrow §10-item-2 exception was not needed). **Form field added (contract-compliant, §4.13 shared CommonImportScan):** `ImportForm.close_old_findings_product_scope: bool = False` routed through the `common` dict to `dojo/importers/services.py` (already accepted by `import_scan`/`reimport_scan`/`auto_import_scan`) — exercised end-to-end by the two product-scope close-old ports. `group_by` was inventoried but NOT added (no ported scenario needs it). OpenAPI guard (`test_apiv3_openapi`) unaffected (asserts path/tag presence only, never form fields). **Skips (honest, enumerated):** 2× `*_statistics` (v3 emits the delta statistics `{new,reactivated,closed,untouched}`, not the v2 before/after per-severity envelope), 2× `*_additional_endpoint` (`endpoint_to_add`/legacy Endpoint wiring out of v3 scope); `push_to_jira`/`group_by`/`endpoint_to_add` also guarded with `skipTest` in the shim. **Not ported (judgment, §5 item 5):** `test_close_old_findings_only_fetches_needed_columns` (spies on a private importer method via `mock.patch` — importer-internal, not endpoint-observable). Full `unittests.api_v3` = **459 executed + 6 skipped, 0 failures** (`/tmp/apiv3_t8a_full.log`; +77 over the 382 baseline = 73 pass + 4 skip). Ruff clean. No v2 production/UI/v2-test code modified (`dojo/api_v3/import_routes.py` is v3, the sanctioned form extension). |
564
566
| 2026-07-21 | **JIRA PUSH FLOWS PORTED TO v3 (§10 backlog #2).** **v3 wiring (sanctioned, all in `dojo/api_v3/import_routes.py` — v3 code):** `ImportForm.push_to_jira: bool = False` added and routed through the `common` dict to the `dojo/importers/services.py` facade (which already accepted `push_to_jira`; §4.13 shared CommonImportScan). The route now OR-s `payload.push_to_jira` with the resolved JIRA project's `push_all_issues` when JIRA is enabled — mirroring the v2 import/reimport viewsets (`dojo/api_v2/views.py:391-404`/`517-535`) and the v3 finding route's identical OR (D7 canonical). The OR uses the resolved driver per mode (engagement / test / `test or engagement or product` for auto); `_check_auto_permission` now returns that driver. **Port strategy:** the v2 corpus (`test_jira_import_and_pushing_api.py`) is a VCR/cassette suite entangled with v2-only surfaces (finding **groups**, **epics**, **webhooks**, UI `/finding/bulk`) — not economical to shim, so targeted v3 ports in `unittests/api_v3/test_apiv3_jira_push.py` (10 tests) mock JIRA at `dojo.jira.services.push` (import path — what `default_importer` + `finding.helper.post_process_findings_batch` both call) and `dojo.finding.services.jira_services.push` (write path, as `test_apiv3_finding_writes` does); `block_execution=True` forces the post-processing sync so the push is observable. **Coverage map (corpus → v3):** `test_import_with_push_to_jira`→`test_import_with_push_to_jira_pushes_each_finding` (set-equality: every finding in the new test is pushed); `test_import_no_push_to_jira`/`..._is_false`→`test_import_without_push_to_jira_does_not_push`/`..._false_does_not_push`; `test_import_no_push_to_jira_but_push_all`→`test_import_push_all_issues_forces_push`; `test_import_no_push_to_jira_reimport_with_push_to_jira`→`test_reimport_with_push_to_jira_pushes`; `test_import_no_push_to_jira_reimport_no_push_to_jira`→`test_reimport_without_push_to_jira_does_not_push`; PATCH/PUT push subtests of `test_create_edit_update_finding`→ existing `test_apiv3_finding_writes` (create push, keep-in-sync `force_sync`, push-failure-400) **+** new `TestApiV3FindingWriteJiraProjectSetting` (the `push_all_issues` OR-on-write gap: PATCH/PUT force push, negative when not push_all). **Import push-failure divergence (documented):** `test_import_jira_push_failure_does_not_fail_import` — a failed import-time push is logged, not raised, so the import still returns 200 (mirrors v2; the importer never propagated JIRA errors). This differs from finding PATCH/PUT, which surface a push failure as a 400 (`test_apiv3_finding_writes.test_update_jira_push_failure_is_400`). **Skipped with reasons (out of v3 alpha scope):** all `group_by`/finding-**group** scenarios (~half the corpus; no `group_by` on the v3 form, groups are §2 non-goal), `test_engagement_epic_*` (epics), `*bulk_edit*` (UI `/finding/bulk`, DS3), `test_import_with_push_to_jira_add_comment`/`add_tags` (note/tag→JIRA sync — the note-create JIRA `add_comment` side-effect is already covered by `test_apiv3_subresources` note side-effects), the `enforce_verified_status[_jira]` matrix (JIRA-layer verified-status gating, independent of the v3 wire surface, covered by v2), the `keep_sync` reimport status/priority-transition asserts and risk-acceptance reopen (require real JIRA issue *status* via cassettes — not reproducible with a mocked push), and `*_auto_create_context_fetches_all_objects_for_push_to_jira` (asserts v2 `import_settings["push_to_jira"]`, a v2 Test_Import audit-record shape, not a v3 response field). Full `unittests.api_v3` re-run green (see the T8b row). Ruff clean. No v2/UI/v2-test code modified. |
565
567
| 2026-07-21 | **BACKLOG PRIORITY 3 — v2 API test-corpus evaluation delivered as the §9.1 disposition table** (no code, per the task). Every remaining `test_apiv2_*` + the named API-adjacent suites + an `APIClient`/`rest_framework`/`-list` scan were inventoried and classified PORTED / PORT-LATER / SKIP / SUPERSEDED, with `test_rest_framework.py` split by area. Two open **PORT-LATER** checkboxes remain (both new §6 checkboxes): (a) `TestDetail` matching-policy read fields (ports `test_apiv2_test_dedupe_policy.py`), (b) `authorized_users` member-management on asset writes gated by `Product_Manage_Members` (ports `test_product_authorized_users_api_authz.py`); `configuration_permissions`-on-user and the v3 self-profile endpoint (which also close `ConfigurationPermissionTest`/`UserProfileTest`) were already tracked in the pre-existing backlog bundle. Notable SUPERSEDED mappings: prefetch-RBAC→`test_apiv3_expand_rbac`, methods-and-endpoints smoke→OpenAPI guard + authz/query sweeps, per-endpoint SchemaChecker→OpenAPI guard, FK-reassign authz→v3 write RBAC re-check. **No trivial ports were implemented** — every PORT-LATER item needs new schema/route surface beyond the <30-min bar (§10.2 conservative). |
568
+
| 2026-07-21 | **FINDINGSLIM EXTENDED WITH `vulnerability_ids` + `cwes` (architect directive — identity-critical fields available on lists without per-row detail fetches).** Two resolver-backed READ fields added to `FindingSlim` (so `FindingDetail` inherits them): **`vulnerability_ids: list[str]`** — flat strings (`["CVE-2020-1234", ...]`), NOT v2's object list; read directly from the `vulnerability_id_set` reverse relation in **storage order** (first = the id mirrored into `cve`), symmetric with what `FindingWrite` already accepts. **`cwes: list[int]`** — the numeric part of each `finding_cwe_set` row parsed via `dojo.finding.cwe.cwe_number`. **Shape choice `list[int]` (not the stored strings):** `Finding_CWE.cwe` is a `varchar(11)` that `save_cwes` always writes as the canonical `CWE-<n>` label (via `finding_cwe_labels`/`cwe_label`), so the stored format is reliably `CWE-<int>`; parsing to int is consistent with the slim's scalar `cwe: int | None`. **Primary is included:** `save_cwes` persists the primary `Finding.cwe` as a `Finding_CWE` row too (first), so reading the rows directly (no scalar-merge) yields the primary first — mirroring v2's `finding_cwe_set` row semantics. Both resolvers read only the prefetched reverse relations (mirror the `resolve_tags` pattern) — never a per-row query; empty lists (not null) when a finding has no rows. **related_names used:** `vulnerability_id_set` / `finding_cwe_set` (default reverse accessors — neither FK on `Vulnerability_Id`/`Finding_CWE` declares `related_name`; identical to v2's serializer `source=`). **Query-pin deltas (+2 fixed in-batch prefetches on findings lists):** `FindingSlim.PREFETCH_RELATED` grew from `("tags",)` to `("tags", "vulnerability_id_set", "finding_cwe_set")`. Offset findings list 5→7 queries; cursor 4→6 — `test_apiv3_cursor.EXPECTED_CURSOR_QUERIES` 4→6 (comment updated; the `offset−1==cursor` relation still holds). No other absolute pin exists (the `test_apiv3_findings_queries` tests are row-independence comparisons, still green; the whole-surface N+1 sweep threshold is 4 and the two new prefetches are distinct single-shot `IN` batches, so no shape repeats). **Defer:** `vulnerability_ids`/`cwes` are relations, not `model._meta.concrete_fields`, so `plan_list_fields` never enters them into the defer set (asserted); `?fields=` accepts them as ordinary slim names. **CSV:** the two list fields flatten to one semicolon-joined column each (`cwes`→`"79;89"`, `vulnerability_ids`→`"CVE-..;GHSA-.."`); `_FINDING_DEFAULT_COLUMNS` updated (inserted after `cwe`). **Write asymmetry (pre-existing, NOT changed here):** finding writes accept `vulnerability_ids` (flat, mirrored into `cve`) but only the scalar `cwe` — there is no `cwes` list on write. Documented + a backlog bullet added ("accept a `cwes` list on finding writes"); not implemented (directive). **Examples:** `api_v3_examples.md` regenerated via the CI-excluded harness (`DD_API_V3_EXAMPLES=1`); the new `cwes`/`vulnerability_ids` keys appear on every captured finding body. **Tests (+7):** `test_apiv3_findings` `_SLIM_KEYS` += both names + new `TestApiV3FindingsVulnIdsAndCwes` (2-vuln-id + 2-CWE-row render on list AND detail, empty-lists-not-null, `?fields=id,cwes,vulnerability_ids` projection = 4) + a defer-set assertion (1); `test_apiv3_findings_queries` new constant-query test (identity rows on every finding stay row-independent, 1); `test_apiv3_csv_export` header pin + new semicolon-join value test (1). Full `unittests.api_v3` = **476 executed + 6 skipped, 0 failures** (`/tmp/apiv3_slimext_full.log`; +7 over the 469 baseline). Ruff clean. No v2 code/UI/v2-test modified. |
Copy file name to clipboardExpand all lines: api_v3_examples.md
+32-19Lines changed: 32 additions & 19 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -2,7 +2,7 @@
2
2
3
3
> **Auto-generated, do not hand-edit.** Every request/response below was captured by `unittests/api_v3/test_apiv3_examples.py` (`DD_API_V3_EXAMPLES=1`, CI-excluded) making **real** in-process requests against the test fixture. Tokens are redacted; long lists are truncated to ~3 rows. Regenerate with the command in that file's docstring.
0 commit comments