feat: API v3 (alpha) — parallel /api/v3-alpha/ with slim refs, expand, and a tested query-count contract#15304
Draft
valentijnscholten wants to merge 35 commits into
Draft
feat: API v3 (alpha) — parallel /api/v3-alpha/ with slim refs, expand, and a tested query-count contract#15304valentijnscholten wants to merge 35 commits into
valentijnscholten wants to merge 35 commits into
Conversation
…alpha) Phase OS1 of API_V3_PLAN.md, mounted at /api/v3-alpha/ (gated on V3_FEATURE_LOCATIONS): - dojo/api_v3/ kernel: token+session auth (reuses v2 token store), pagination envelope with hybrid exact/planner-estimate counts, RFC 9457 problem+json errors, expand/fields engine (cycle guard + budget, expand-driven select_related/prefetch_related), django-filter adapter, include=counts - dojo/finding/api_v3/: FindingSlim/FindingDetail schemas + build_findings_router() factory (GET list/detail) - dojo/importers/services.py: framework-neutral import facade returning structured ImportResult; POST /import with mode auto|import|reimport - unittests/api_v3/: 37 tests incl. constant-query-count guarantee (5 slim / 7 expanded, independent of row count) and v2 import DB-state equivalence New dependency: django-ninja==1.6.2 (pydantic v2 already present transitively).
- severity rank ordering (Case/When, mirrors v2 numerical_severity)
- strict unknown-filter-param rejection (400; typo'd filters must not
silently return unfiltered data)
- FilterSpec registry + vocabulary snapshot test (contract drift fails CI)
- expand=locations: swaps locations_count for edge rows
[{location, status, audit_time}] via special renderer with declared
prefetch paths (query count stays constant)
- dedicated 'fields' problem type; full problem+json error-path test sweep
- OpenAPI schema-generation guard test
65 tests green (OS1 37 + OS2 28).
- slim/detail/write schemas per plan §4.5 with canonical slims relocated out of the finding module (finding expand targets re-export them) - router factories with GET list/detail, POST, PATCH, DELETE (no PUT in alpha, additive later) - RBAC: authorized querysets + v2-parity permission checks; product/ product_type deletion mirrors v2 exactly (async_delete or Endpoint.allow_endpoint_init context) - user visibility: v2-parity view_user configuration-permission gate with guaranteed self-read (plain users see exactly themselves); UserSerializer.validate() rules ported (superuser/staff gating, password write-only, no self-delete) - filter specs registered + vocabulary snapshot extended (additive diff) 129 tests green (65 prior + 64 new).
…e layer - dojo/finding/services.py (D7 flagship): create/update/delete_finding extracted by reconciling FindingSerializer.update()/validate() AND the UI edit_finding flows; 17-row divergence table in the phase report, serializer semantics canonical, UI-only behaviors deferred to CONV2 - POST/PATCH/DELETE /findings: thin routes (authorize -> service -> serialize), 404-then-403 permission ladder, JIRA mocked in tests (push failure -> problem+json 400) - engagement + test resources: CRUD, canonical slims relocated out of the finding module, deletion mirrors v2 exactly (no Endpoint.allow_endpoint_init wrapper, unlike products - per v2) - filter snapshot extended additively 197 tests green (129 prior + 68 new).
- query_report.py: captures per-request SQL, normalizes literals, flags the N+1 signature (same query shape >= 4x in one request) - test_apiv3_query_report.py: sweeps every mounted v3 GET route with fanned-out rows so per-row queries cannot hide; OpenAPI completeness check forces every new GET endpoint to be query-profiled; writes /tmp/apiv3_query_report.md artifact - current surface: zero N+1 flags 198 tests green.
- GET /locations, /locations/{id} (read-only; superuser gate mirrors v2
LocationViewSet, rows via get_authorized_locations as the future seam)
- GET /findings/{id}/locations and /products/{id}/locations: edge rows
with status (+audit_time/auditor for findings; product edge has no
audit columns), parent-inherited authorization, constant query count
- auditor ref added to expand=locations edge rows (closes OS2 deferral)
- ?fields= allowlist now includes expandable keys, so
expand=locations&fields=id,title,locations works (OS2 open question)
- flag-off test: V3_FEATURE_LOCATIONS=False unmounts all of
/api/v3-alpha/ (in-process URLconf reload)
- query sweep extended with the 4 new endpoints, zero N+1 flags
229 tests green (198 prior + 31 new).
- dojo/api_v3/subresources.py: three parameterized kernel factories, attached only where models have real storage (verified matrix in plan \$12): notes+files on finding/engagement/test, tags on finding/engagement/test/product - authorization inherited from parent: authorized-view queryset (404) then per-method permission (403), values mirror v2 related-object permission classes - note privacy mirrors v2: private = report-exclusion only, not a per-user read filter; tags mirror tagulous force_lowercase + inheritance write path; files mirror FileUpload.clean() validation and streamed download - note side-effects (JIRA comment, last_reviewed, mentions) deferred to the service layer per D7 - recorded as a known alpha parity gap - query sweep extended (13 new GET endpoints), zero N+1 flags 252 tests green (229 prior + 23 new).
- RBAC expand sweep (15 tests): no expanded object, included count, denormalized ref, or sub-resource row from outside authorized querysets (ports the v2 prefetch-RBAC test intent) - honest benchmark (1021 findings, limit=100, in-process): v3 constant 5 queries / 37.9ms median vs v2 636 queries / 521ms with ?prefetch; latency directional, query counts load-bearing; CI-excluded harness (DD_API_V3_BENCH=1) - api_v3_examples.md: auto-generated verbatim request/response pairs (findings incl. expand/filters/pagination/counts/notes/locations/ import/PATCH; products as simple contrast; DD_API_V3_EXAMPLES=1) - docs page: automation/api/api-v3-alpha-docs.md with known-alpha-gaps section and beta URL-migration notice - invariant checklist I1-I10 verified: all pass - v2 regression untouched: test_rest_framework 879 OK, test_apiv2_prefetch_rbac 10 OK 267 tests green (+2 CI-excluded harnesses).
19 catalogued divergences (D1-D19) between the v2 API serializer, the classic UI views, and the v3 finding service - every row verified in code with file:line anchors; 3 rows of the original extraction table materially corrected, 2 new divergences surfaced. Key outcomes: - D17: v3 delete skips delete-time JIRA sync (Finding.delete() sentinel default) - confirmed v3 regression, scheduled for the alpha PR - D8 (auto-mitigation on deactivation) and D7 (last_reviewed stamping) proposed canonical behaviors for the convergence track - consolidated v2-consumer impact assessment: 1 potentially breaking, 4 behavioral, 12 invisible - sequencing: alpha fixes only clear v3 bugs; CONV1 refactor-then- converge with release notes; CONV2 with behavior-pinning tests
Note side-effects (architect directive - closes the OS5 alpha gap): - generic notes factory gains an optional on_note_created callback; kernel stays free of JIRA/notification imports (I5) - finding: process_note_added service fires JIRA comment sync (linked issue or finding-group issue), last_reviewed stamping, and @mention notifications - mirrors v2's notes action exactly - engagement/test: mention notifications only (verified v2 parity - their v2 actions fire nothing else) - mentions use the shared v2 process_tag_notifications helper via the crum request bridge; skipped only for non-HTTP service calls (I6) D17 fix (API_V3_DIVERGENCE_ANALYSIS.md - confirmed v3 regression): - delete_finding now passes the v2 tri-state default push_to_jira=None so the JIRA delete close/reassign runs; bare Finding.delete() hits the suppress-sentinel and silently skipped it - regression-pinning test asserts finding_delete receives None Docs: known-gaps section updated (note side-effects now at parity; locations reclassified as platform limitation; count estimation reclassified as design decision); plan gains an explicit post-alpha OS backlog with architect-confirmed TODOs. 276 tests green (+2 CI-excluded).
Adds an 'API v3 Docs (alpha)' entry next to the existing v2 OpenAPI link, pointing at the interactive docs (/api/v3-alpha/docs via the api_v3:openapi-view URL name). Guarded by V3_FEATURE_LOCATIONS - the v3 mount is conditional on that flag (D5), so an unguarded url tag would raise NoReverseMatch on every page when the flag is off. Verified: URL name resolves to /api/v3-alpha/docs; base.html compiles.
v3 speaks the product's new domain language (the UI relabel is already
default-on): product_type -> organization, product -> asset, across the
entire wire surface, all-or-nothing:
- paths: /organizations, /assets (+ /assets/{id}/locations|tags)
- ref keys + expand: finding.asset, finding.organization,
engagement.asset, asset.organization
- filter params: asset=, asset__in=, organization= (+ registry names,
snapshot regenerated - diff is exactly the rename)
- schema classes (AssetSlim, OrganizationSlim, ...), router factories,
OpenAPI tags/operationIds, error messages
- import form: asset_name, organization_name (mapped internally to the
AutoCreateContextManager's product_* context keys)
- asset user-role ref renamed to asset_manager (relabel's canonical
term); critical_product/key_product kept (DB-column booleans the UI
relabel also keeps)
- docs: mapping table doubles as rename documentation + explicit rule
that API naming does NOT follow the UI relabel flag (per-instance
API contracts are codegen poison); examples regenerated from real
requests
NOT renamed (D11 scope): Django models, DB tables, dojo/product*/
module paths, and services/queries internals - the DTO layer decouples
wire names from models.
Verified: OpenAPI render contains zero product/product_type wire
tokens; regenerated api_v3_examples.md contains zero legacy wire keys;
276 tests green (+2 CI-excluded), same counts as pre-rename.
- errors.py: comment on why DRF constants exist in a ninja API (v3 reuses v2 helpers/services that raise DRF exceptions; the adapter maps them onto the closed problem+json contract) - plan D3: record why schemas are hand-declared DTOs, never ModelSchema-derived (auto-derivation recreates v2's heavy-by-default disease); Slim/Detail/Write/Update roles; idiomatic ninja/FastAPI - plan backlog: port v2 endpoint-level test corpora via dual-endpoint adapter (import/reimport scenarios first, then JIRA push flows)
Two guards ensuring a v3 endpoint can never ship without authorization (enforces invariant I8 structurally): - test_apiv3_authz_sweep.py: probes all 66 OpenAPI operations as anonymous (must 401) and as a zero-permission user (must never receive data: empty RBAC-scoped lists, 404/403 on details+writes). Write probes carry minimal valid payloads from a completeness-gated registry so they genuinely reach the authz gate - a validation 400 counts as sweep failure. Any future endpoint fails the completeness gate until probed. Sanity test proves empty lists are RBAC-empty, not fixture-empty. - test_apiv3_authz_static.py: source-scan over the route layer - bans direct .objects. access outside a justified 5-entry allowlist (which fails on stale entries) and AST-verifies every route operation references an authz primitive (one module-local helper deep). Audit result: ZERO authorization gaps found - all 66 operations already deny by default. Also: filter-contract test docstring now explains why the vocabulary is a tested contract (silent-typo'd-filter failure mode, D6 one-vocabulary-many-projections, DD_API_V3_UPDATE_SNAPSHOTS workflow). 284 tests green (+2 CI-excluded).
…ization Part A - lists accept any Detail field via ?fields= (Jira-style): - GET /findings?fields=id,title,impact,references works on a list; default stays byte-identical slim (pinned by tests); unknown -> 400; applied uniformly to all seven resources - detail-only relation refs (mitigated_by, asset user roles, URL subtype) declare DETAIL_SELECT_RELATED - fixed join, never per-row Part B - defer() on list queries (architect-suggested): - default lists no longer fetch heavy detail columns from the DB at all (finding drops 9 columns incl. description/impact/references; engagement 15); a field named in ?fields= un-defers exactly that column - defer sets computed at runtime from model._meta.concrete_fields (never properties/relations/annotations) - defer over only(): only() requires enumerating every ref column across select_related paths and silently drops joins; defer composes safely (rationale in plan \$12) - serialize_list_row splices only requested detail fields onto the slim base, so unrequested deferred columns are never read - deferred-load N+1 structurally impossible - defer-proof tests assert deferred columns absent from the list SQL by default and present when requested; constant-query assertions in both modes; zero un-defer exceptions needed 296 tests green (+2 CI-excluded).
The complete v3 plan (decisions D1-D11, contract spec, invariants, phase log, post-alpha backlog) and the consolidated implementation debrief (commit timeline, per-phase deliverables, review findings, benchmark, invariant verdicts).
Architect directive: v3 imports are synchronous - no job resource, no deferred-execution path, no reserved grammar. Removes the background form field, its 400 reject branch, the \$4.13 reservation, the backlog job-resource item, and the docs-page reservation wording. Capability checklist row 13 disposition: intentionally removed from v3 scope. Also resolves the two \$12 OPEN rows with architect sign-off: django-ninja/pydantic pins accepted; COUNT_CAP/EXPAND_BUDGET defaults accepted. 296 tests green (+2 CI-excluded).
Reverses the alpha's no-PUT decision (architect directive). Semantics: PUT validates the create-shaped schema (extra=forbid, required fields enforced) and applies the payload WITHOUT exclude_unset, so omitted optionals reset to schema defaults - a true full replace mirroring v2's DRF update(partial=False). Permission ladder identical to PATCH (404 authorized-resolve, then 403 edit permission). - findings/assets/engagements/tests get dedicated *Replace schemas: create-Write schemas declare non-null columns as nullable-with-None (create drops None -> model default) but a full replace would setattr None onto an existing row and violate NOT NULL; Replace schemas default those to the model default. findings/tests also drop their immutable parent FK (editable=False, like PATCH/v2) - organizations/users reuse their Write schemas unchanged - finding PUT flows through dojo/finding/services.py (side-effects: JIRA force_sync, risk acceptance, vuln ids); reporter/vuln-ids follow the service's no-reset-on-omit semantics - +37 tests incl. reset-of-omitted-optionals proofs; authz sweep grows 66 -> 72 operations, all deny-by-default 335 tests green (+2 CI-excluded).
Implements the D4 reserved grammar: ?pagination=cursor on every top-level list. Same envelope with count: null; previous: null (forward-only, GitLab-style); next carries an opaque tamper-proof cursor (django signing, salt dojo.api_v3.cursor); page fetched as limit+1 for has-next - no COUNT query, so cursor pages cost one query FEWER than offset pages (pinned in tests). - keyset-safe orderings derived per FilterSpec (id always; created/ updated where declared), each with deterministic id tiebreaker; other o= values in cursor mode -> 400 - NULL-aware keyset predicate: Finding.updated is NULL in practice despite model metadata, so the tuple comparison handles the NULL group per Postgres placement (NULLS LAST asc / FIRST desc) - a mixed walk never skips or repeats a row - kernel-central: paginate() gained one optional filter_spec kwarg; one-line change per list route; sub-resource lists stay offset-only (clean 400) - 19 new tests incl. full-walk exactly-once proofs, tamper/mismatch 400s, filter/fields/expand/include composition, per-page constant queries 353 tests green (+2 CI-excluded).
GET /<resource>/export.csv on all seven list resources - the first non-list projection of the D6 filter contract: identical filters, full o= vocabulary, q=, and ?fields= incl. the detail opt-up. No pagination params; expand/include/limit/offset/pagination/cursor -> 400 (new 'export' problem type URI). - streamed via csv.writer over queryset.iterator(chunk_size=2000): memory-bounded, query count independent of row count (pinned) - row cap DD_API_V3_EXPORT_MAX_ROWS (default 100000) via the shared capped-count helper: over-cap -> 400 'narrow your filter', never a silently truncated file - generic kernel flattener (no per-resource column lists): refs -> <key>_id,<key>_name columns; tags semicolon-joined; ISO-8601 Z; header row always emitted - CSV-injection hardening: cells starting with = + - @ or TAB get quote-prefixed (spreadsheet formula defense), tested - authz sweep 72 -> 79 operations; query sweep +7 entries; backlog item reduced to XLSX-only 382 tests green (+2 CI-excluded).
Backlog priority 1 of the v2 test-corpus port. The corpus scenarios and assertions run VERBATIM against v3 through an adapter mixin (unittests/api_v3/import_corpus_shim.py) that overrides only the two endpoint helpers: v2 field names map to the v3 wire (product_name -> asset_name, product_type_name -> organization_name, import/reimport -> consolidated mode=), and the v3 response translates back to the v2-shaped dict the assertions read. Zero v2 test files modified. - 73 scenarios ported green, 4 honest skips (2x legacy endpoint_to_add, 2x v2 before/after statistics envelope vs v3 delta stats), 1 importer-internal test re-expressed as endpoint-level delta checks - ImportForm gains close_old_findings_product_scope (facade already accepted it; \$4.13 declares shared CommonImportScan fields in scope) - port surfaced one contract nuance: v2 silently ignores a mismatched engagement id when product_name is set; v3 rejects - handled in the shim, production untouched 457 tests green + 6 skipped (2 CI-excluded harnesses + 4 corpus skips).
Completes the architect-requested v2 test-corpus port (backlog priorities 2 and 3). JIRA flows (10 targeted tests; the v2 corpus is VCR/cassette- and v2-entangled, so intent-ports rather than a shim): - ImportForm gains push_to_jira (facade already accepted it), and the import route now ORs it with the resolved JIRA project's push_all_issues - mirroring v2's import/reimport viewsets; the batch importer path does not apply that OR itself (real gap found by the port). Driver resolution per mode: test -> engagement -> product. - covered: push-per-finding on import, no-push default, push_all forcing (import + PATCH/PUT writes), reimport push semantics, import push-failure logs-and-continues (200, mirrors v2) vs finding writes surfacing 400 - divergence documented - skipped with reasons: finding groups, epics, webhooks, bulk edit, cassette-dependent status transitions Corpus disposition (plan \$9.1): every remaining test_apiv2_* and API-adjacent suite classified PORTED / PORT-LATER / SKIP / SUPERSEDED; backlog reduced to two PORT-LATER checkboxes (test dedupe-policy read fields, authorized_users member management). 469 tests green + 6 skipped.
Two couplings that must be resolved before the v2 test files can ever be deleted: - test_jira_import_and_pushing_api.py is the only consumer of the JIRA VCR cassettes; deleting it orphans the sole coverage of the shared JIRA engine behaviors (v3 tests cover dispatch only, by design). Prerequisite: v3-keyed cassettes, naturally paired with the workflow-actions backlog item. - test_import_reimport.py is imported by the v3 corpus shim (subclasses ImportReimportMixin). Prerequisite: relocate the mixin to a shared module first.
Architect directive: a finding's identity fields must be present on lists without per-row detail fetches. FindingSlim (and Detail via inheritance) gains: - vulnerability_ids: list[str] - flat strings in storage order (first = the cve mirror), NOT v2's object-wrapper list; symmetric with what FindingWrite accepts - cwes: list[int] - parsed from Finding_CWE rows (reliably CWE-<n> format via save_cwes), primary first; int chosen for consistency with the scalar cwe field Both prefetch-backed (reverse accessors vulnerability_id_set / finding_cwe_set): +2 fixed in-batch queries on findings lists, pins updated deliberately (offset 5->7, cursor 4->6, offset-1==cursor relation preserved). Relations verified excluded from the defer set. CSV gains two semicolon-joined columns; examples regenerated; write asymmetry (no cwes list on writes) recorded as backlog bullet. 476 tests green + 6 skipped.
Ports test_apiv2_test_dedupe_policy.py (\$9.1 PORT-LATER -> PORTED). TestDetail gains deduplication_algorithm and hash_code_fields, computed by reusing the Test model properties (settings-driven lookups keyed by scan type) - zero duplicated logic, zero extra queries on detail. Writes supplying either field are REJECTED with 400 (extra=forbid) - a deliberate hardening over v2, which silently ignores them: silent- ignore is the failure mode v3 rejects everywhere. Kernel accommodation: computed detail fields that read deferred columns would lazy-load per row when requested on lists via ?fields= - new optional DETAIL_FIELD_COLUMNS map lets a schema declare the concrete columns its computed fields read, and plan_list_fields un-defers exactly those (additive; schemas without it unchanged). Constant-query verified for the list opt-up; CSV columns flow automatically. 16 new tests. 492 tests green (+6 skipped) before Scalar addition.
…cision Scalar reference page at /api/v3-alpha/reference (architect-approved, supersedes the OS6 deferral) WITHOUT vendoring: - version-pinned jsDelivr URL (@scalar/api-reference@1.63.0) + sha384 Subresource Integrity hash - the browser refuses to execute the bundle if the CDN ever serves different bytes; only the hash lives in the repo - hand-rolled ~50-line view instead of the scalar-django-ninja package (avoids a new PyPI dependency; keeps built-in Swagger at /docs as the locally-served, air-gap-safe default - Scalar is progressive enhancement, documented in the docs page) - plain Django view: not an OpenAPI operation, so no authz/query completeness-gate obligations (asserted in tests) Also records the D7 product decision in the divergence analysis: API field-writes do NOT stamp last_reviewed (architect: NO) - v3's current behavior is canonical; stamping remains an explicit-action concern (notes today, workflow actions later); CONV2 guardrail noted. 495 tests green (+6 skipped).
Closes the read/write asymmetry from the FindingSlim identity-fields change: FindingWrite/Update/Replace accept cwes: list[int], popped at the routes and passed as a service kwarg - fully parallel to vulnerability_ids. Contract (\$12): - precedence mirrors v2 and the vuln-ids->cve pattern: explicit non-None scalar cwe stays primary; else cwes[0] is mirrored into cwe BEFORE save so the primary persists; rows primary-first via the existing save_cwes helper (no duplicated label logic - plain ints go onto unsaved_cwes verbatim, the helper canonicalizes) - omission (None) never touches rows beyond the existing scalar-change resync (no-reset-on-omit, symmetric with vulnerability_ids/reporter) - explicit cwes: [] clears the extras, resyncing to the scalar-derived primary - an explicit statement, unlike omission - PUT subtlety: the full-replace dict always carries cwe (reset-default None), so the mirror keys off explicit-non-None, not key presence +9 tests (precedence, mirror, omission, empty-list, PUT semantics, read-back symmetry). 504 tests green (+6 skipped).
Supersedes the CDN+SRI approach (architect directive): @scalar/ api-reference is now an exact-pinned yarn dependency in components/ package.json (1.63.0, lockfile-verified integrity), installed by the EXISTING yarn step in Dockerfile.nginx-alpine - zero Dockerfile changes because STATICFILES_DIRS already includes components/node_modules. - reference view points at the local static path; SRI attribute dropped (same-origin, image-baked); no runtime third-party, air-gapped deployments now get a working reference page, no metadata leaks - trust model identical to every other yarn-managed frontend dep in this repo; integrity enforced by yarn.lock hashes at image build - engine note: Scalar 1.63.0 wants node>=22; the image base ships v22.23.0 (verified), local lockfile regen on older node needs --ignore-engines - tests: static-URL + no-CDN/no-SRI assertions + exact-pin guard reading components/package.json 504 tests green (+6 skipped).
- 'API v2 OpenAPI3 Docs' -> 'API v2 Docs' - 'Open API v3 alpha Docs (Swagger)' -> /api/v3-alpha/docs - 'Open API v3 alpha Docs (Scalar)' -> /api/v3-alpha/reference Both v3 entries stay inside the V3_FEATURE_LOCATIONS guard (the mount is conditional; unguarded url tags would break every page flag-off). Verified: both URL names resolve; base.html compiles.
The UIPreferenceLoader serves base.html from dojo/templates/ OR dojo/templates_classic/ per user preference - the previous menu commit only covered the modern template, so classic-UI users saw no v3 links (and the old v2 label). Classic now matches: 'API v2 Docs' rename plus the two flag-guarded v3 entries (Swagger + Scalar). Verified: authenticated classic render shows all three, old label gone.
github-actions
Bot
added
settings_changes
![accesslint[bot]](https://avatars.githubusercontent.com/in/1502?s=60&v=4){kind=small}
v3 security-parity review of three open v2 hardening PRs (#15300, #15296, #15191): #15300 IMMUNE (v3 locations read-only, no reference- write surface); #15296 IMMUNE (configuration_permissions not on the v3 user write surface); #15191 identity half was a REAL GAP now fixed. Gap: get_authorized_users returns co-members, so a non-superuser with view_user+change_user could PATCH/PUT a visible co-member's email or username -> account takeover via password reset. Adds _enforce_identity_field_rules (mirrors UserSerializer.validate()): superuser unrestricted, self-edit allowed, email/username change to another account -> 400; create is a no-op. Wired into PATCH and PUT after the superuser/staff gate; deny-by-default sweep unchanged. +6 tests. (API_V3_PLAN.md also carries the examples-refresh §12 row committed next.)
Regenerates api_v3_examples.md from real in-process requests: cursor pagination (page-1 -> next -> page-2), CSV export (headers + flattened ref columns + semicolon-joined cwes/vulnerability_ids/tags), PUT full- replace, cwes-on-write PATCH + read-back; import section corrected to the current form. Flagship finding seeded so bodies show populated tags/cwe/cwes/vulnerability_ids. Harness stays CI-excluded.
The unit-test matrix runs a flag-off leg (unit-tests.yml v3_feature_locations: [false, true]); with the flag off dojo/urls.py does not mount /api/v3-alpha/, so mount-dependent v3 tests hit the SPA catch-all (200 HTML) and failed - 66 failures on the PR's test-rest-framework (false) job, invisible locally because the settings default is True. Guards (skipUnless settings.V3_FEATURE_LOCATIONS): - ApiV3TestCase: covers every HTTP contract test transitively - ApiV3ImportShim mixin: covers the import-corpus tests (they subclass the v2 corpus mixins, not ApiV3TestCase) - TestApiV3OpenApi: get_openapi_schema() resolves the mounted namespace (KeyError 'api_v3' when unmounted) Genuinely mount-independent unit tests keep running (static authz tripwire, expand cycle guard). Verified: flag-off Ran 511 OK (skipped=506); flag-on unchanged Ran 511 OK (skipped=6). Plan §12 also records: PR #15307 verdict (v3 IMMUNE - authorized_users not on the write surface) + the recommendation to implement member management as a sub-resource, not an inline write field.
Serialization audit (2026-07-21) confirmed no manager/model/tagulous object reaches Pydantic — every Ref/collection field has a resolver, every fall-through field is a plain scalar. The one latent soft-spot: ref_label() falls back to str(obj) for models not in _LABELERS, so a future Ref to an unregistered model would silently mislabel rather than fail. Recorded as a backlog item: a test walking every Ref-typed field and asserting its target model is registered.
The deploy job's in-app-docs link check (validate_docs_build.yml, lychee) requires every docs.defectdojo.com URL under dojo/ to resolve against the built docs site. The RFC 9457 error 'type' base pointed at https://docs.defectdojo.com/api/v3/errors/<type>, which has no backing page -> broken link -> deploy failed. Repoint the base at the v3 alpha docs page via its committed front-matter alias (/en/api/api-v3-alpha-docs), with a per-type fragment (#error-<type>) preserving distinct RFC 9457 type identity. Using the literal alias path guarantees the URL resolves (Hugo writes alias stubs at the exact path given, independent of the language-prefix config); lychee checks the page, not the fragment. Updated the error-type test assertion and the docs-page example to match. No functional/API-response-shape change beyond the type URL.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Introduces API v3 (alpha) — a new, parallel API mounted at
/api/v3-alpha/next to the existing v2. v2 is not modified: apart from the mount indojo/urls.py, one dependency, a handful of settings constants, and menu/docs entries, every change is new code (83 files, ~18k lines, 30 commits — one per implementation phase).organizations,assets,engagements,tests,findings,locations,users— plus one consolidatedPOST /import(mode=auto|import|reimport). Everything else stays on v2 until v3 earns parity.{id, name}refs;?expand=inlines related objects and drives realselect_related/prefetch_related(replacing v2's post-serialization?prefetch=);?fields=narrows or opts lists up into detail fields;?include=countsreturns filtered aggregate totals inmeta.?prefetch=test). Guarded byassertNumQueriestests and a whole-surface N+1 sweep.{count, next, previous, results, meta?}) with hybrid counts (exact below a cap, Postgres planner estimate above it, flaggedcount_exact: false); offset pagination by default plus opt-in cursor (keyset) pagination (?pagination=cursor, signed opaque cursors, forward-only).GET /<resource>/export.csv) reuses the identical contract, streams, refuses (never truncates) above a row cap, and hardens cells against spreadsheet formula injection.dojo/finding/services.py,dojo/importers/services.pyreturning a structuredImportResult) shared with — but not yet wired into — v2. Findings expose and acceptvulnerability_idsandcwessymmetrically. Note creation fires the same side-effects as v2 (JIRA comment sync,last_reviewed, @mentions).Endpointmodel; findings↔locations are M2M with status on the edge. The whole mount is gated onV3_FEATURE_LOCATIONS(default on).get_authorized_*querysets and permission semantics as v2. A deny-by-default authorization sweep probes every operation (anonymous → 401; zero-permission user → never data) with a completeness gate, plus a static tripwire banning unauthorized model-manager access in route code. Audit result: zero gaps.application/problem+jsonwith afieldsextension./api/v3-alpha/docsand a Scalar reference at/api/v3-alpha/reference(served from image-built static files — exact-pinned yarn dependency, no CDN at runtime); docs-site page underdocs/content/automation/api/; both linked from the profile menu.api_v3_examples.mdcontains captured-from-real-requests examples./api/v3/and stays there (one client migration, at contract freeze). Every response carriesX-API-Status: alpha.API_V3_PLAN.md(decisions D1–D11, the full contract, invariants, phase log, backlog),API_V3_DIVERGENCE_ANALYSIS.md(19 verified divergences between the v2 serializer, UI views, and the v3 service, with a convergence proposal), andapi-v3-alpha-debrief.md(implementation record).unittests/api_v3/(plus 2 env-gated harnesses for benchmarks/example capture), including the v2 import/reimport scenario corpus running verbatim against v3 through an adapter shim, JIRA push-flow ports, and v3 import/reimport DB-state equivalence against v2. The existing v2 suites are untouched and green.Known alpha gaps and deliberately deferred items (bulk operations, workflow actions, delete-impact preview, XLSX export) are documented in the docs page and tracked in
API_V3_PLAN.md's backlog.