| Status | Approved for OS implementation (alpha). Downstream adoption phases are outline-only, not started. |
| Audience | Developers and architects implementing or reviewing v3. Written to be executable by an implementation-focused engineer or AI model without re-deriving decisions. |
| Companion | "API v3 Idea and Design Spec" (the original design proposal). Where this plan and the spec disagree, this plan wins — it incorporates codebase and downstream-consumer audits done after the spec was written. |
| How to update | Decisions live in §1 (numbered D1…). Don't silently change a decision — add a row to the Decision Log (§12) with rationale, then update the affected sections. Open questions also go to §12. |
v3 is a new, parallel API mounted at /api/v3/ next to the existing v2. v2 stays fully supported and is not modified by this work. The alpha covers seven objects (product_type, product, engagement, test, finding, location, user) plus a consolidated import endpoint. Everything else stays on v2 until v3 earns parity.
The design decisions, with the reasoning that produced them:
D1 — Parallel, additive-only mount; alpha carries "alpha" in the URL. v3 mounts in dojo/urls.py next to v2 — during alpha at /api/v3-alpha/, so the instability is impossible to miss (headers and docs banners are ignorable; the URL isn't). At beta it moves to /api/v3/ and never changes again — one client-side URL migration total, timed to the additive-only contract freeze (§4.1). The entire OS deliverable is new code: no v2 serializer, viewset, or URL changes. This is what makes one big, easily-testable PR viable (§7) — nothing existing can regress.
D2 — Framework: Django Ninja pilot with a hard decision gate; DRF fallback. Ninja (Pydantic v2 DTOs) is recommended because explicit schemas make the ref/expand model natural and schemas are subclassable (a downstream requirement, see D10). It is not justified by async — DefectDojo deploys WSGI (uwsgi), so async endpoints yield no concurrency win. Neither ninja nor pydantic is currently a dependency; adding them is a deliberate supply-chain decision surfaced in the PR. Phase OS1 ends with a go/no-go gate against seven exit criteria (§6, OS1); on failure we implement the identical contract on DRF. Everything in §4 is framework-agnostic.
Ecosystem trade-off (accepted): Ninja has a much smaller third-party ecosystem than DRF — there is no drop-in equivalent for much of the DRF plugin landscape (filter integrations, throttling add-ons, spectacular extensions), so v3 writes more of its own kernel code (pagination, filter adapter, error handling — §6 OS1/OS2). We accept this deliberately: (a) the kernel pieces are small, contract-driven, and easier to own than to adapt — and with AI-assisted implementation the cost of writing well-specified glue code is far lower than it historically was; (b) the DRF plugin ecosystem is itself a liability in places — several widely-used packages are aging or effectively unmaintained, which is how v2 accumulated debt like the bespoke prefetch framework. Owning ~6 small kernel modules outright is preferable to depending on third-party code with uncertain maintenance. The OS1 gate still checks that this self-built surface stays small (exit criterion 4: django-filter reuse or costed equivalent).
D3 — Response model: slim default + {id, name} refs + ?expand= + ?fields=. No in-band memoization. Every relation renders as a ref by default; ?expand=test.engagement swaps refs for fuller objects and drives real select_related/prefetch_related (this is the actual N+1 fix — v2's ?prefetch= is post-serialization and does per-row queries). We surveyed how major APIs handle nested relations: Stripe (id + expand[]), ServiceNow ({value, link} refs + display values), Jira (fields=/expand=), GitLab/GitHub (small objects inlined and repeated per row). No mainstream API deduplicates repeated nested objects in-band the way one downstream client implementation does; the only standardized dedup is JSON:API side-loading, which carries the client-rehydration complexity we're explicitly rejecting. gzip makes repeated {id, name} refs nearly free on the wire. Consequence: no memoization, no prefetch blob, no client rehydration contract.
Schemas are hand-declared DTOs, never auto-derived from models. Ninja offers ModelSchema
(derive every field from the Django model); v3 deliberately does not use it — auto-derivation would
recreate v2's core disease: heavy-by-default responses where every model field leaks into the
contract and every model change is a silent breaking change. Each resource declares up to four
explicit classes — Slim (list/expansion shape: identity + status + refs + timestamps),
Detail(Slim) (adds the heavy fields; inherits so nothing is duplicated), Write and Update
(create/patch payloads — separate because writes take integer ids where reads render refs, forbid
unknown fields, and never expose server-managed fields). Pydantic auto-generates the validation
and OpenAPI from these declarations; the declarations themselves ARE the curated contract. Four
DTOs per resource is idiomatic ninja/FastAPI style, and the named classes are exactly what makes
schemas subclassable downstream (I4).
D4 — Pagination: one envelope, two modes; offset is the default; no two-phase pagination. Envelope is always {count, next, previous, results} (+ optional meta). Default mode is limit/offset because the primary future consumer (a SPA with data tables) needs page-jump, offset URL state, and a count. Cursor/keyset mode is reserved as ?pagination=cursor (GitLab precedent: same endpoints, opt-in keyset) for export/sync consumers: same envelope, count: null, opaque cursor in next, sort restricted to keyset-safe orderings. Alpha ships offset only; the reserved parameter makes cursor a later non-breaking addition. Clients must treat next/previous as opaque URLs — that's what makes both modes interchangeable. (Page-jump itself is a UI choice, not a law — if a future UI drops it, cursor becomes the recommended default for large collections with zero contract change; the dual-mode envelope is the hedge.) Counts follow industry practice (Jira's new endpoints drop totals; GitLab omits x-total >10k; Elasticsearch caps at 10k): exact below a documented threshold, planner-estimated above it. Below COUNT_CAP the count is exact (capped counting bounds the cost); above it, count is the Postgres planner's row estimate for the filtered query (EXPLAIN-based, the PostgREST count=estimated technique), clamped to ≥ CAP+1 and flagged count_exact: false in meta. Users see "≈1.2M" instead of "10000+"; cost stays bounded; never a full COUNT(*) and never a second rehydration query phase like the two-phase paginator observed in a downstream implementation, whose root cause (heavy list annotations) the slim default removes.
D5 — Locations, not Endpoints; M2M with status on the edge; URL-type only; flag-required. v3 never exposes the legacy Endpoint model. The audit showed Finding↔Location is many-to-many via LocationFindingReference, which carries the status (Active/Mitigated/FalsePositive/RiskAccepted/OutOfScope), auditor, audit_time — so the original spec's singular "location": {id, name} field on Finding is wrong. Slim findings carry locations_count; the full list is the /findings/{id}/locations sub-resource whose rows are (location ref + edge status). Only the URL location subtype has a persistable model today (LocationData.code/.dependency DTOs exist but are dropped by LocationManager on import), so v3 documents location support as URL locations only until Code/Dependency models land. v3 requires V3_FEATURE_LOCATIONS=True (already default-on in dev); with the flag off, /api/v3/ is not mounted at all — v3 carries no legacy-endpoint code path.
D6 — One filter contract, many projections. The most load-bearing pattern found in downstream API usage: the same filter parameters must drive the list, aggregate counts, future chart/aggregation endpoints, and future exports. Therefore (a) the per-object filter vocabulary is a documented, snapshot-tested artifact (django-filter-compatible grammar: field, field__gte, field__in, …, multi-sort o=, free-text q=); (b) ?include=counts returns severity/status totals computed over the filtered, authorized queryset in the same response's meta — no second round-trip for "N active / M verified" headers; (c) future aggregation endpoints take identical filter params. Only include=counts ships in alpha; the grammar is what's locked.
Expressiveness ceiling (learned from downstream workarounds, not copied from them): flat field__lookup params cannot express boolean OR/AND groups. When a downstream consumer hit this it accreted bespoke any/all-style suffix params and its dashboard layer abandoned GET params for POSTed filter dicts. v3 does not rebuild that ceiling: the flat grammar stays for the common case, and complex boolean filtering gets exactly one structured representation later — a reserved POST /<resource>/query accepting a canonical JSON filter tree (also the future substrate for saved views and aggregation filters). Bespoke _any/_all-style suffix params are banned; if a filter need can't be expressed in the flat grammar, it waits for the structured representation rather than growing a one-off param.
D7 — Service layer is the kernel; routes are thin; v2 and UI views untouched (for now), converging later. Business logic currently lives in three places: the classic Django UI views (e.g. the close/edit/risk-acceptance flows in dojo/finding/views.py), the v2 serializers (FindingSerializer.update() doing JIRA/risk/vuln-ids), and — without this decision — soon a third copy in v3. The UI-vs-API duplication is a known drift source (the paths have diverged on notification triggers and JIRA sync behavior because each is patched independently). Therefore: all side-effects and orchestration (JIRA push, risk acceptance, status transitions, import) live in dojo/<resource>/services.py functions written to serve all three consumers, but in the alpha PR only v3 calls them — that preserves the additive property (D1). Schemas do field validation only; routes parse → call service → shape response. The import endpoint requires a new dojo/importers/services.py facade returning a structured ImportResult (the current importer returns a 7-tuple, and — correcting the spec — no service wrapper exists today; the v2 serializer calls DefaultImporter directly). Extraction rule: a service is extracted by reconciling both existing implementations (v2 serializer + UI view flow), not by copying the serializer — where they disagree, the canonical behavior is chosen consciously and recorded in §12; otherwise the later UI-view refactor silently becomes a behavior change instead of a pure refactor. v2 serializers and UI views migrate onto the services in the post-alpha convergence track (§7): end state is views, v2, and v3 as thin shells over one service layer — one place to fix a workflow bug.
D8 — Auth: token AND Django session+CSRF from day one; v2 tokens work unchanged. Two authentication modes, both active on every endpoint:
- Token — reuses the existing v2 tokens. Same
Authorization: Token <key>scheme, validated against the same store (rest_framework.authtoken.models.Token). No new token model, no migration, no re-issuing: a key that works on/api/v2/works on/api/v3/the same day. Implemented as a small header-auth class that parses theTokenprefix and resolvesrequest.user. - Django session + CSRF — session cookie plus
X-CSRFTokenon unsafe methods. Mandatory now because the eventual SPA consumer is session-authenticated; bolting it on later would ripple through every route's auth declaration.
Auth classes are registered as an ordered list (auth=[TokenAuth(), django_auth]); each returns None on no-match so the next gets a chance; all failing → 401 problem+json. The token scheme is intentionally replaceable: because routes only see the pluggable auth list (invariant I7), the token backend can later be extended or swapped (scoped/expiring keys, JWT, OAuth) as a config-level change with no route changes — v2-token compatibility is the alpha baseline, not a permanent commitment. Authentication is separate from authorization: after any auth mode succeeds, all object access still flows through the existing get_authorized_* querysets and permission semantics (I8), so RBAC is identical to v2. Both modes are tested on the same endpoints from OS1.
D9 — Boring, standard conventions. ISO-8601 UTC datetimes, explicit nulls (no key-stripping), raw fields (no server-side display formatting of users etc.), errors as RFC 9457 application/problem+json with a fields: {field: [messages]} extension for validation errors. Where downstream consumers currently rely on non-standard conventions (Unix-ms timestamps, stripped nulls, preformatted user strings), those are client-side adaptations — they do not shape the v3 contract.
D10 — Forward-compatibility invariants instead of downstream features. OS phases build zero downstream-specific features, but §5 defines invariants (composable schemas, router factories, pluggable auth, the filter contract, the include/meta mechanism, service-layer purity) that guarantee later downstream adoption phases are additions on top of v3, never changes to it. Every OS phase is reviewed against §5.
D11 — v3 speaks the new domain language: organizations and assets (wire surface only, unconditional). The product has already renamed Product_Type→Organization and Product→Asset (DD_ENABLE_V3_ORGANIZATION_ASSET_RELABEL, default on); a brand-new API must not bake the legacy names into the surface consumers will codegen against for years, and the alpha is the only time the rename is free (post-beta it becomes aliases + deprecation forever). Scope: wire surface only — paths (/organizations, /assets), ref keys (finding.asset, finding.organization, engagement.asset, asset.organization), filter params (asset=, organization=, …), expand paths, schema class names (AssetSlim, OrganizationSlim), router factories, OpenAPI tags, import form fields (asset_name, organization_name), docs, examples, snapshots. All-or-nothing — no half-rename; a renamed path with a legacy ref key is the worst outcome. Not renamed: Django models, DB tables, and dojo/product*/ module paths (the DTO layer is exactly what decouples wire names from models; internal relocation rides the module reorg later); engagement/test/finding/user/location are not part of the relabel. Unconditional: the API naming does NOT follow the UI relabel flag — a per-instance API contract is codegen poison; deployments that restore legacy UI labels still get organizations/assets on v3 (documented). The v2→v3 mapping table doubles as the rename documentation.
OS alpha delivers: /api/v3/ mount; CRUD for the 7 objects; refs/slim/expand/fields/include; offset pagination with capped counts; documented+tested filter contract; generic notes/tags/files sub-resources; locations with edge status; consolidated POST /api/v3/import (mode auto/import/reimport); token+session auth; problem+json errors; OpenAPI with alpha marking; query-count regression tests.
Explicit non-goals for OS (deferred, see §8): bulk operations, workflow actions (close/review/duplicate handling), aggregation/chart endpoints, saved views, CSV export, delete-impact preview, job-status resources, global search, permission-map/bootstrap endpoints, cursor pagination implementation (grammar reserved only), v2/UI-view refactors (deferred to the convergence track, §7).
Three audits inform this plan (details in PR discussion / architect's notes):
- OS v2 audit. ~49 endpoint groups, ~106 serializers, now organized per-module (
dojo/<resource>/api/);dojo/api_v2/serializers.pyis largely a re-export shim.?prefetch=is post-serialization with per-row-per-field queries (dojo/api_v2/prefetch/mixins.py:7-48) — its RBAC hole is already patched (deny-by-default registry,authorized_querysets.py; testsunittests/test_apiv2_prefetch_rbac.py), so the motivation to replace it is performance, not security. Business logic embedded inFindingSerializer.update()/validate()(dojo/finding/api/serializer.py:438-560): JIRA push, risk acceptance, vuln-ids/CWEs. Importer returns a 7-tuple (dojo/importers/base_importer.py:183-188); no service facade exists. Deps: DRF 3.17.1, drf-spectacular 0.30.0, django-filter 26.1; no ninja/pydantic. No existing v3 scaffolding. - Downstream API-consumer audit. A survey of how existing downstream consumers (a session-authenticated SPA among them) use the v2 API. Key findings: hard dependence on limit/offset +
count, the OS?prefetch=mechanism, filtered-list totals alongside lists, bulk operations with per-item skip reasons, workflow actions, aggregation endpoints, saved views, poll-based async jobs (no websockets anywhere), delete-impact previews, and CSV export. Full capability list with dispositions: §9. - Location audit. Full parallel Location subsystem behind
V3_FEATURE_LOCATIONS(default True):Location+LocationFindingReference(edge status incl. mitigation transitions indojo/importers/location_manager.py:230) +LocationProductReference;URLis the only persistable subtype (location_manager.py:402-405drops others); when the flag is on,Endpoint.__init__raises — hard cutover, no dual-write.
This section is normative. Implementers follow it verbatim; deviations require a §12 decision-log entry.
- Alpha mounts at
/api/v3-alpha/indojo/urls.pyalongside the v2 mount (near the existingre_path(r"^{}api/v2/"...), ~line 198). Entire mount is conditional onsettings.V3_FEATURE_LOCATIONS(D5). Everywhere else in this document,/api/v3/is the logical name; during alpha the actual prefix is/api/v3-alpha/(a single settings/urls constant — do not hardcode the prefix anywhere else). - Stability-in-URL policy (Kubernetes/Google precedent, one migration total):
/api/v3-alpha/for the whole alpha period — the URL itself signals "contract may break at any time; do not build production dependencies on this." At beta the API moves to/api/v3/and stays there through GA — beta's additive-only contract freeze is exactly the stability a permanent URL implies. Beta/GA status is conveyed by header + docs, not further URL changes. Clients therefore migrate URLs exactly once (alpha→beta), at the moment they must re-verify against the settled contract anyway. - OpenAPI
info.version = "3.0.0-alpha"; every response carries headerX-API-Status: alpha(laterbeta, then removed at GA); docs page states the alpha contract may change. - OpenAPI JSON at
/api/v3-alpha/openapi.json, interactive docs at/api/v3-alpha/docs.
- Token:
Authorization: Token <key>validated against the existing DRF token store (rest_framework.authtoken.models.Token) — same tokens work on v2 and v3. In Ninja: a smallAPIKeyHeadersubclass parsing theTokenprefix; setsrequest.user. - Session: Django session cookie + CSRF on unsafe methods. In Ninja:
ninja.security.django_authandNinjaAPI(csrf=True). - Both registered on the API instance:
auth=[TokenAuth(), django_auth]. Anonymous → 401 problem+json.
Every list response:
limitdefault 25, max 250.offset≥ 0.- Count strategy (hybrid exact → estimate),
COUNT_CAP = 10_000(settings-overridable):- Capped exact count:
COUNT(*)over aLIMIT COUNT_CAP+1subquery (qs.order_by()[:COUNT_CAP+1].count()-style) — bounded cost, and it detects whether the result is under the threshold. - ≤ CAP →
countis exact. - = CAP+1 →
count= the Postgres planner's row estimate for the same filtered queryset (qs.explain(format="json"), readPlan Rows), clamped tomax(estimate, COUNT_CAP+1)so the reported count never contradicts step 1; response gains"meta": {"count_exact": false}.
- Never a full
COUNT(*)on unbounded tables, never a second PK-prefetch phase. Estimates depend on planner statistics (autovacuum/ANALYZE freshness) and may drift — documented as approximate. Clients must tolerate an empty page when jumping near an estimated end (results: [],next: null). Tests assert the exact/estimate switchover, the clamp, and thecount_exactflag — never specific estimate values (nondeterministic across environments).
- Capped exact count:
?pagination=cursoris implemented (forward-only keyset mode, GitLab-style; see the 2026-07-20 §12 entry). Same envelope withcount: nullandprevious: null(forward-only — no backward cursor);nextcarries an opaque, tamper-proof signedcursor=token (django.core.signing, saltdojo.api_v3.cursor) encoding only the ordering + last-row key position (never the filters). Sorting is restricted to keyset-safe orderings —id,created,updated(each ± direction), whichever a resource'sFilterSpecdeclares (derived per spec; a resource withoutcreated/updatedallowsidonly) — always with a deterministicidtiebreaker appended; default isidascending. Any othero=(or a multi-fieldo=) → 400 problem+json (pagination type); a tampered/undecodable/ordering-mismatched cursor → 400. Pages are read aslimit+1rows to detect a next page with noCOUNTquery. Available on the seven top-level list endpoints; parent-scoped sub-resource/edge lists remain offset-only. Filters/q=/fields=/expand=/include=countsall compose unchanged.- Sorting:
o=comma-list,-prefix for descending (o=-severity,date). Allowed orderings are a curated per-object list (part of the filter contract, §4.9).
{ "id": 42, "name": "Q3 Pentest" }- Produced by one shared
Refschema. The relation key conveys the type — notypefield, except location refs, which add"type": "<location_type>"because one key can hold heterogeneous location subtypes. - Ref label registry (single source of truth,
dojo/api_v3/refs.py):
| Model | name source |
|---|---|
| Product_Type, Product, Engagement | .name |
| Test | .title if set, else str(.test_type) |
| Finding | .title |
| Dojo_User | .username (clients do their own display formatting — D9) |
| Location | .location_value |
| Test_Type, Development_Environment | .name |
Slim = identity + primary status fields + parent refs + timestamps. No per-row computed fields (no age, no SLA math, no counts requiring extra queries — locations_count is a queryset annotation, which is allowed). Implementer: verify each field exists on the model before wiring; adjust only by removing, not adding, without a §12 entry.
- FindingSlim:
id, title, severity, active, verified, false_p, duplicate, risk_accepted, out_of_scope, is_mitigated, date, cwe, cwes[], vulnerability_ids[], test{ref}, engagement{ref}, product{ref}, product_type{ref}, reporter{ref}, locations_count, tags[], created, updated.cwes(flatlist[int]) andvulnerability_ids(flatlist[str]) are identity-critical and always present on slim (no per-row detail fetch); populated by two fixedprefetch_relatedpaths (finding_cwe_set,vulnerability_id_set). See §12 (2026-07-21).engagement/product/product_typeare denormalized parent refs populated by oneselect_related("test__engagement__product__prod_type")— no extra queries.
- TestSlim:
id, name(=title), test_type{ref}, engagement{ref}, product{ref}, product_type{ref}, environment{ref|null}, lead{ref|null}, target_start, target_end, percent_complete, tags[], created, updated. - EngagementSlim:
id, name, product{ref}, product_type{ref}, lead{ref|null}, status, engagement_type, target_start, target_end, active, tags[], created, updated. - ProductSlim:
id, name, description, product_type{ref}, lifecycle, tags[], created, updated. - ProductTypeSlim:
id, name, description, critical_product, key_product, created, updated. - UserSlim:
id, username, first_name, last_name, email, is_active, is_superuser, last_login. - LocationSlim:
id, name(=location_value), type(=location_type), tags[]. URL-subtype detail fields (protocol, host, port, path, query, fragment) appear on retrieve/expand only.
Detail (GET /{id}) returns Slim + a documented per-object set of heavier fields (e.g. Finding adds description, mitigation, impact, steps_to_reproduce, severity_justification, references, file_path, line, mitigated, mitigated_by{ref}). List returns Slim only.
- Comma-separated dotted paths:
?expand=test.engagement,reporter. Each path segment must be a registered expandable relation of the current schema (per-schema registryEXPANDABLE: dict[str, tuple[django_relation_path, TargetSchema]]). - Expanding swaps the ref for the target's Slim schema inline (no side blob).
- No fixed depth cap. Two guards, both returning 400 problem+json:
- Cycle guard: while walking a path, track the model classes from the root; a segment whose target model already appears in its own ancestry chain is rejected (
finding → test → ... → findingsstyle loops). - Budget: total expanded relation nodes across all paths ≤
EXPAND_BUDGET = 10(settings-overridable).expand=test.engagement,reporter= 3 nodes.
- Cycle guard: while walking a path, track the model classes from the root; a segment whose target model already appears in its own ancestry chain is rejected (
- The expand set drives the queryset: FK/O2O segments append
select_relatedpaths; M2M segments appendprefetch_related. Implemented centrally indojo/api_v3/expand.py(plan_queryset(qs, paths)), never per-route. - Special case:
expand=locationson a finding swapslocations_countforlocations: [{location: {ref+type}, status, audit_time}](bounded by the same budget; usesprefetch_related).
- Comma-separated allowlist per object (documented enum = the Detail field names + the schema's
EXPANDABLEkeys). Unknown field → 400 (dedicatedfieldsproblem type).idalways included. This is the "picker/dropdown" facility (?fields=id,name), replacing the need for separate restricted-list endpoints. - On a list,
?fields=may opt UP into the Detail field set (Jira-style). The default (no?fields=) list is the Slim shape exactly. When a requested field is beyond Slim, the row serializes with the Detail shape and is then projected to the requested fields; when all requested fields are within Slim, behaviour is identical to Slim. Because?fields=names are plain row-columns, opting up is a widerSELECTon the same single query — never a per-row cost. List querysets thereforedefer()the heavy detail columns that were not requested, so the default list never fetches them and?fields=impactun-defers exactlyimpact(Detail-only relation refs, e.g.finding.mitigated_by, add a fixedselect_relatedjoin, applied only when requested). Guard: only row-columns are eligible for?fields=; anything computed per row (SLA/age math, per-row counts) is permanently excluded — it would reintroduce the N+1 the Slim default removes. DetailGET /{id}routes are never deferred. Implemented resource-agnostically by the kernel (plan_list_fields/serialize_list_rowindojo/api_v3/expand.py); the location-edge sub-resource lists serialize fixed edge shapes and are out of scope (§12).
- Namespaced response add-ons rendered into
meta. Alpha:include=countson findings lists only:
"meta": { "counts": { "total": 812, "active": 400, "verified": 123, "duplicate": 12,
"severity": {"Critical": 3, "High": 40, "Medium": 200, "Low": 500, "Info": 69} } }- Computed on the filtered, authorized queryset via one aggregate query. The mechanism (
dojo/api_v3/include.py) is generic: a registry ofinclude_name -> callable(filtered_qs) -> dict, so later includes (and downstream-defined ones) plug in without contract changes.
- Grammar (fixed): exact
field=, lookupsfield__gte/__lte/__gt/__lt/__in/__icontains/__isnull, multi-sorto=, free-textq=.__intakes comma-separated values. - Per-object vocabulary is curated and documented; alpha findings vocabulary:
id__in, title__icontains, severity, severity__in, active, verified, duplicate, false_p, risk_accepted, out_of_scope, is_mitigated, date__gte/__lte, cwe, cwe__in, product, product__in, product_type, engagement, test, reporter, tags__in (any-match), created__gte/__lte, updated__gte/__lte; orderings:o= id, date, severity, title, created, updated(± each). Other objects: analogous minimal sets (id/name/parent-FK/dates). - Implementation reuses django-filter
FilterSets where practical (framework-light:FilterSet(request_params, queryset).qs); severity ordering must sort by severity rank, not alphabetically (reuse v2's approach). - Snapshot-tested: a test renders each object's vocabulary (params + orderings) to a checked-in JSON snapshot; contract drift fails CI (§6 OS2).
RFC 9457 problem+json, Content-Type: application/problem+json:
{ "type": "https://docs.defectdojo.com/api/v3/errors/validation",
"title": "Validation failed", "status": 400,
"detail": "2 fields failed validation",
"fields": { "severity": ["Not a valid choice."], "date": ["Required."] } }- 401 unauthenticated, 403 forbidden (RBAC), 404 unknown-or-unauthorized object (do not leak existence), 400 validation/expand/filter errors. One exception handler in
dojo/api_v3/errors.py; routes never hand-build error bodies.
ISO-8601 UTC with Z suffix for datetimes; dates as YYYY-MM-DD; explicit nulls; tags as list[str]; enums as their stored string values; write payloads reference relations by integer id ("test": 9), reads render refs — asymmetry is intentional and documented.
Update verbs (PATCH + PUT). Every writable resource (findings, assets, organizations, engagements, tests, users; locations stay read-only) offers both. PATCH is a partial update (model_dump(exclude_unset=True) — only supplied fields change). PUT is a full replace: it validates against the resource's create-shaped Write schema (required fields enforced, extra="forbid") and applies payload.dict() without exclude_unset, so an omitted optional resets to its schema default — mirroring v2's DRF update(partial=False). The PUT schema default of a non-null / non-blank-with-default column is that column's model default (not None), so a full replace never violates a NOT NULL/blank constraint. Both verbs share the identical permission ladder (authorized-queryset resolve → 404, object edit permission → 403), reject unknown fields with a 400 problem+json, and treat an editable=False parent (finding test, test engagement) as non-reassignable. Finding writes (both verbs) flow through dojo/finding/services.py (I6). See §12 (2026-07-20 PUT reversal).
One generic factory each (dojo/api_v3/subresources.py), attached to all seven resources:
GET/POST /api/v3/<resource>/{id}/notes { "entry": "...", "private": false }
GET/PUT/POST /api/v3/<resource>/{id}/tags { "tags": ["pci"] } # PUT replaces, POST appends
DELETE /api/v3/<resource>/{id}/tags/{tag}
GET/POST /api/v3/<resource>/{id}/files (multipart: file, title)
GET /api/v3/<resource>/{id}/files/{file_id}/download
Authorization inherited from the parent object (mirror v2's related-object permission classes and note privacy rules). NoteSchema: id, entry, author{ref}, private, edited, created, updated. FileSchema: id, title, size, created.
POST /api/v3/import (multipart/form-data)
mode: auto | import | reimport # default auto
file, scan_type, engagement | test | (product_name+engagement_name+auto_create_context),
close_old_findings?, do_not_reactivate?, ...CommonImportScan fields
autoresolves via existingAutoCreateContextManager.get_target_test_if_exists(...).- Destructive flags are never implied by mode — if omitted, importer defaults apply and the response echoes effective values.
- v3 imports are synchronous — the response returns once the import completes; there is no job resource and no deferred-execution path for imports (removed from v3 scope by architect directive; §12, 2026-07-20).
- Endpoint delegates entirely to the new
dojo/importers/services.pyfacade (§6 OS1). Response:
{ "mode_resolved": "reimport", "test": { "id": 9, "name": "ZAP Scan" },
"statistics": { "new": 4, "reactivated": 1, "closed": 2, "untouched": 37 },
"close_old_findings": true }GET /api/v3/locations+GET /api/v3/locations/{id}(read-only in alpha; lifecycle is import-driven). Filter:type,name__icontains,product.GET /api/v3/findings/{id}/locationsandGET /api/v3/products/{id}/locationsreturn edge rows:{ location: {id, name, type}, status, audit_time }(+auditor{ref}on findings).- Documented: URL locations only in alpha; other types 501/absent until models exist.
GET /<resource>/export.csvon the seven top-level list resources (findings,assets,organizations,engagements,tests,users,locations—locationskeeps its superuser gate). Registered as a literal path so the{int:id}detail route's integer converter cannot matchexport.csv(no collision, verified).- Identical filter contract as the list: filters,
o=(the full offset-mode ordering vocabulary),q=, and?fields=(including the Detail opt-up). No pagination — the export is the whole filtered, authorized set.expand/include/limit/offset/pagination/cursorare not applicable → 400 problem+json (type=.../errors/export), keeping the reserved-param handling coherent with the list. - Row cap:
API_V3_EXPORT_MAX_ROWS(envDD_API_V3_EXPORT_MAX_ROWS, default 100 000). Measured with the shared capped-count helper; if the filtered count exceeds the cap the request is a 400 telling the client to narrow the filter — never a silent truncation. - Response:
StreamingHttpResponse,Content-Type: text/csv; charset=utf-8,Content-Disposition: attachment; filename="<resource>-export.csv",X-API-Status: alpha. Streamed by acsv.writeroverqueryset.iterator(chunk_size=…)(chunked prefetch), so memory is bounded and the query count is independent of the row count (pinned byassertNumQueries). - Column flattening (one generic, resource-agnostic kernel —
dojo/api_v3/csv_export.py; no per-resource column lists): columns are the serialized row's keys in schema order, restricted to the?fields=projection. A{id, name}ref → two columns<key>_id,<key>_name; a location{id, name, type}ref → three columns<key>_id,<key>_name,<key>_type;tags(any list field) → one semicolon-joined column; datetimes ISO-8601Z, datesYYYY-MM-DD, booleans lowercase;null→ empty cell. A zero-row export still emits the full header row. Row values are reused fromserialize_list_row(the same shaping as the list) then flattened, so the CSV never diverges from the JSON. - CSV-injection hardening (security product): any cell whose text starts with
=,+,-,@or a TAB is prefixed with a single quote (the standard spreadsheet formula-injection defense).
These guarantee downstream adoption phases are additions, never rework. PR review checklist — every item verified before merge:
- I1 Envelope is closed.
{count, next, previous, results, meta}and nothing else;pagination=,include=,expand=,fields=,o=,q=are reserved query params; new capabilities extendmeta/include, never the envelope. - I2 Filter contract is a tested artifact. Vocabulary snapshot in CI; grammar (§4.9) never varies per endpoint.
- I3 Ref shape is closed.
{id, name}(+typefor locations only). Extensions would break every consumer's ref handling. - I4 Schemas are composable. Every response schema is a named, importable class; no anonymous/inline schemas; downstream code must be able to
class ProFindingSlim(FindingSlim): epss_score: float | Noneand get correct OpenAPI. - I5 Routers are factories. Each resource exposes
build_<resource>_router(*, schema=None, filters=None, queryset_hook=None, auth=None) -> Routerwith OS defaults. The OS mount calls the factory; a downstream distribution can call it with overrides and mount the result under its own prefix. No route logic at module import time. - I6 Services own all writes/side-effects. Routes contain zero business logic (parse → authorize → service → shape). Service functions: keyword-only args, explicit
user, return domain objects or result dataclasses, importable and callable without any HTTP context. - I7 Auth is a pluggable list (§4.2); adding an auth class must require no route changes.
- I8 RBAC only via
get_authorized_*querysets + permission checks — never ad-hocuser.is_stafflogic in routes; expanded/included data is always drawn from authorized querysets. - I9 Error contract is closed (§4.10); new error kinds get new
typeURIs, not new shapes. - I10 No UI-flavored formatting in responses (D9) — formatting belongs to clients.
All phases land on one feature branch as one PR (§7), one merge-worthy commit (or small commit series) per phase. Each phase lists: goal, files, key details, tests, acceptance criteria.
Test suite layout — v3 is a separate package. Unlike the flat unittests/test_apiv2_*.py files, all v3 tests live in a dedicated package so the suite is separately runnable, shardable in CI, and never interleaved with v2:
unittests/api_v3/
__init__.py
base.py # ApiV3TestCase(DojoAPITestCase): v3 URL-prefix helper,
# token + session/CSRF client helpers
test_apiv3_<topic>.py # all files referenced in the phases below live here
snapshots/filters.json # filter-contract snapshot (OS2)
Django test discovery handles packages natively: unittests.api_v3 runs the whole v3 suite, unittests.api_v3.test_apiv3_findings one module. Implementer: verify the CI unit-test workflow discovers the subpackage (standard Django labels do; a hardcoded file glob would need a tweak). Run tests exactly as repo convention requires:
./run-unittest.sh --test-case unittests.api_v3.test_apiv3_findings 2>&1 | tee /tmp/test_output.logTest client policy. Ninja endpoints are plain Django views, so the standard in-process Django test client (via DojoAPITestCase) works against v3 unchanged — no socket, same thread: server-side exceptions propagate as real tracebacks (raise_request_exception default), logging is visible, mock.patch affects what routes see, assertNumQueries works, and the test transaction is shared (DB-state assertions for import equivalence). All contract/integration tests use it (full URLconf/middleware/auth/CSRF stack). ninja.testing.TestClient (invokes router operations directly, skipping URL resolution and middleware — and therefore real auth) is allowed only for kernel-internal unit tests (expand parser, pagination math).
Goal: working /api/v3/ skeleton proving the whole contract on its hardest consumer (findings), plus the import endpoint; produce the evidence for the framework decision.
Files:
requirements.txt: adddjango-ninja(pin latest stable 1.x at implementation time; it vendors/pulls pydantic v2 — note both in the PR description).dojo/api_v3/__init__.py—NinjaAPIinstance:csrf=True,auth=[TokenAuth(), django_auth], exception handlers registered,X-API-Status: alphavia a response hook/middleware, version string.dojo/api_v3/auth.py—TokenAuth(parseAuthorization: Token <key>→Token.objects.select_related("user"), setrequest.user; return None on mismatch so session auth can try).dojo/api_v3/refs.py—Refschema + label registry (§4.4) +to_ref(obj)helper.dojo/api_v3/pagination.py— envelope paginator (§4.3) incl. the hybrid exact→planner-estimate count (offset mode) and forward-only keyset cursor mode (?pagination=cursor, opaque signed cursors).dojo/api_v3/errors.py— problem+json handlers (validation, 401/403/404, expand/filter errors).dojo/api_v3/expand.py— parse/validate/plan (§4.6). OS1 may hard-scope the registry to finding relations; generic walker completed in OS2.dojo/api_v3/filtering.py— django-filter adapter: build FilterSet from a declarative per-object spec; applyo=/q=.dojo/api_v3/include.py— include registry +countsimplementation (§4.8).dojo/finding/api_v3/schemas.py—FindingSlim,FindingDetail(§4.5);dojo/finding/api_v3/routes.py—build_findings_router()(I5) withGET /findings,GET /findings/{id}. Queryset:get_authorized_findings(Permissions.Finding_View)+ baseselect_related("test__test_type", "test__engagement__product__prod_type", "reporter")+annotate(locations_count=Count("locations")).dojo/importers/services.py— the facade (framework-neutral; useful even if the gate fails):Internally constructs the importer context exactly as@dataclass(frozen=True) class ImportResult: test: Test mode_resolved: str # "import" | "reimport" new: int; closed: int; reactivated: int; untouched: int test_import: Test_Import | None close_old_findings: bool; do_not_reactivate: bool # effective values def import_scan(*, user, scan_file, scan_type, ...) -> ImportResult def reimport_scan(*, user, scan_file, test, ...) -> ImportResult def auto_import_scan(*, user, ...) -> ImportResult # resolves via AutoCreateContextManager
ImportScanSerializer.save()does today (dojo/api_v2/serializers.py:526-551is the reference implementation; do not modify it) and unpacks the 7-tuple intoImportResult.dojo/api_v3/import_routes.py—POST /import(§4.13): multipart parse, permission check mirroringUserHasImportPermission/UserHasReimportPermissionsemantics, delegate to facade.dojo/urls.py— conditional mount (D5/§4.1).
Tests (OS1): test_apiv3_auth.py (token OK, session+CSRF OK, anonymous 401, both on same endpoint); test_apiv3_findings.py (slim shape incl. denormalized parent refs + locations_count; expand=test.engagement; fields=; include=counts under RBAC; filter basics; pagination envelope; count switchover exact→estimate with a lowered COUNT_CAP override, incl. clamp + count_exact flag); test_apiv3_findings_queries.py (assertNumQueries constant for 100-row list, with and without expand — the headline guarantee); test_apiv3_import.py (import/reimport/auto DB-state equivalence vs the v2 endpoints on identical payloads — same counts, same finding states, incl. close_old_findings).
Gate (architect-reviewed, not self-certified): the implementer produces a gate report (criteria results + benchmark numbers, posted on the draft PR) and stops; the architect reviews, records the outcome in §12, and releases OS2. Criteria: (1) query-count tests green; (2) import equivalence green; (3) RBAC via authorized querysets works as an auth dependency; (4) django-filter reuse demonstrated or equivalent costed; (5) both auth modes green; (6) OpenAPI renders + client codegen sane for Ref/Slim/expand; (7) I4/I5 seam demo (subclass FindingSlim + remount router, no fork). Failure → re-implement OS1 surface on DRF with identical contract; OS2+ unchanged.
Goal: finish the generic machinery so OS3 resources are declaration-only.
expand.py: generic registry-driven walker, cycle guard, budget, M2M prefetch planning,expand=locationsedge-row special case.filtering.py: severity rank ordering;q=per-object field sets; vocabulary snapshot test (test_apiv3_filter_contract.pywrites/comparesunittests/api_v3/snapshots/filters.json).include.py: registry finalized; counts on any findings-shaped queryset.errors.py: full coverage sweep — every failure path returns problem+json (add tests for expand-budget 400, cycle 400, unknown field/filter 400, unauthorized 404-vs-403 policy).- OpenAPI polish: tags per resource, alpha banner text,
openapi.jsonserved; CI check that schema generation succeeds (mirror how v2's spectacular check runs in CI if present; otherwise add a unit test that renders the schema).
Acceptance: kernel modules have no per-resource imports (dependency direction: resources import kernel, never reverse — keeps I5 honest).
Goal: CRUD for product_type, product, engagement, test, user; write endpoints for finding.
- Per resource:
dojo/<resource>/api_v3/{schemas,routes}.py, router factory, filter spec, authorized queryset (dojo/<resource>/queries.py— all exist), permission checks mirroring the v2 permission classes for create/update/delete. - Write schemas (create/update): the editable subset of the detail fields; required-vs-optional mirrors the v2 serializer for that resource; relations by integer id (§4.11); server-managed fields (
id,created,updated, computed refs) are never writable. Ambiguity → conservative choice + §12 entry (per §10.2). - Finding writes go through a new service (
dojo/finding/services.py), extracted as new code from the logic inFindingSerializer.update()/validate()(dojo/finding/api/serializer.py:438-560is the reference; leave it untouched):Must reproduce: JIRA push + keep-in-sync (def update_finding(finding, *, changes: dict, vulnerability_ids: list[str] | None = None, push_to_jira: bool = False, user) -> Finding def create_finding(*, test, data: dict, user, push_to_jira: bool = False) -> Finding def delete_finding(finding, *, user) -> None
jira_services.push,is_keep_in_sync), risk-acceptance processing, vuln-id/CWE persistence, mitigated-field edit rules, active/verified/duplicate/false_p invariants. Port the validation semantics into schema+service validation errors. Extraction rule (D7): reconcile the v2 serializer path with the corresponding UI view flows (dojo/finding/views.py— edit, close, risk-acceptance) before finalizing each service function. List every behavioral divergence found (notifications, JIRA sync, status side-effects) in §12 with the chosen canonical behavior — these services are the future home for the UI views too (CONV2), so API-only semantics must not get baked in unnoticed. - Users: read + self; admin-only writes (mirror v2
UserViewSetpermission behavior). - Tests: per-resource
test_apiv3_<resource>.py(CRUD happy path, RBAC 404/403, filters, expand where applicable);test_apiv3_finding_writes.pyasserting service side-effects (JIRA mocked, risk acceptance, vuln ids) match v2 behavior for the same payloads.
Per §4.14: dojo/location/api_v3/, finding/product location sub-resource routes, locations_count already wired in OS1. Read-only. Tests: edge-status shape, URL-only registry behavior, flag-off ⇒ whole /api/v3/ absent (404), RBAC inheritance from parent.
dojo/api_v3/subresources.py factories (§4.12) attached to all seven resources. Reference semantics: the v2 @action implementations (dojo/finding/api/views.py:412-586) and note-privacy/permission classes. Tests: test_apiv3_subresources.py — matrix over (resource × notes/tags/files × CRUD), privacy rules, parent-authorization inheritance, multipart upload + download.
- Port the intent of
unittests/test_apiv2_prefetch_rbac.pyto expand:test_apiv3_expand_rbac.py— a user must never see an expanded/included object outside their authorized querysets. - Repeat query-count + latency benchmark v2 (
?prefetch=) vs v3 (?expand=), 100-row findings list on the largest fixture; record numbers in the PR description. - Docs-site page (
docs/content/...): v3 alpha overview, contract summary, v2→v3 mapping table for the seven objects, "one way to do notes/tags/files", import consolidation, alpha disclaimer. api_v3_examples.md(repo root): real captured request/response pairs (not hand-written) for developer/architect review — findings (detail, list with filtering+pagination+expand+include=counts, notes sub-resource, locations sub-resource, import) and products (the simple-entity contrast: detail, list, CRUD). Captured by executing real requests through the in-process test client against fixture data; bodies verbatim.- Docs UI: built-in Swagger UI already serves
/api/v3-alpha/docs(try-it works with session auth). Evaluate swapping to Scalar (modern OpenAPI reference UI; self-hostable single JS asset pointing at/openapi.json— must be vendored locally like the v2 spectacular sidecar, no CDN). Keep Swagger if vendoring is awkward — this is polish, not contract. - Final §5 invariant checklist pass; final ruff/lint pass; ensure v2 test suite untouched and green.
Deliberately out of the alpha, to be scheduled in later phases. Each is enabled by the service layer and the closed contract, so all are additive:
- Bulk operations (
bulk_update/bulk_delete/… with per-item skip reasons — checklist #5) - Workflow actions (
close,request_review,mark_duplicate,unaccept_risk, … — checklist #6) - Delete-impact preview (
GET /<resource>/{id}/delete-preview— checklist #14) - XLSX export from the filter contract (checklist #15 — CSV shipped in the alpha, §4.15; XLSX is the remaining format)
- delete-time
push_to_jiraparam,configuration_permissionson user writes, v3 self-profile endpoint, filter vocabularies for the location-edge sub-resources (PUT full replace was pulled forward out of this bundle and shipped — §12, 2026-07-20) - Accept a
cweslist on finding writes — DONE: finding create/update/replace now accept acweslist (flatlist[int], primary-first) symmetric with the read shape and parallel tovulnerability_ids(§12, 2026-07-21). - Port the v2 endpoint-level test corpora to v3 (architect-requested). Priority order:
- [x] (1) the import/reimport corpus (
test_import_reimport.py's mixin,test_apiv2_scan_import_options.py,test_importers_closeold.py) — DONE via the dual-endpoint adapterunittests/api_v3/import_corpus_shim.py(§12, 2026-07-21). Zero v2 test files modified. - [x] (2) JIRA push flows (test_jira_import_and_pushing_api.pyintent) against v3 finding writes + import — DONE viaunittests/api_v3/test_apiv3_jira_push.py(targeted ports, mocked JIRA layer) + thepush_to_jiraImportForm field / routepush_all_issuesOR (§12, 2026-07-21). - (3) evaluate the rest oftest_apiv2_*per scope — DONE: the full disposition is the §9.1 table. The remaining PORT-LATER items are the only open checkboxes: - [x]TestDetailmatching-policy read fields (deduplication_algorithm+hash_code_fields, rejected-on-write) — ported viaunittests/api_v3/test_apiv3_test_dedupe_policy.py(§12, 2026-07-21) - [ ]authorized_usersmember-management on asset writes gated byProduct_Manage_Members— portstest_product_authorized_users_api_authz.py- (configuration_permissionson user writes + v3 self-profile endpoint are already tracked in the bundle above; they also closeConfigurationPermissionTest/UserProfileTest.)
- One feature branch, one PR for OS1–OS6. Rationale: the deliverable is additive-only (D1), so blast radius is contained; a single PR deploys/tests as one unit; reviewers get one coherent contract instead of six partial views. Keep one merge-worthy commit per phase so the history doubles as the phase log.
- The OS1 framework gate happens on the branch (draft PR): benchmark + exit-criteria results posted as a PR comment before OS2 work proceeds.
- Post-alpha convergence track (separate PRs, not part of alpha; ordered by risk):
- CONV1 — v2 delegates to services. Refactor
FindingSerializer.update()andImportScanSerializer(then other resources) to call the services. Small diffs; the extensivetest_apiv2_*suite pins behavior. Zero contract change. - CONV2 — UI views delegate to services. View-by-view (never big-bang), starting with the flows the services already reconciled (finding edit/close, risk acceptance, import). UI view test coverage is thinner than the API's, so each flow gets behavior-pinning tests before its refactor. Zero template/UX change.
- CONV3 — delete dead duplicates. Remove the now-unused logic bodies (serializer
update()internals, per-resource notes/files action implementations the views duplicate). Only after CONV1+2 prove the services in production.
- CONV1 — v2 delegates to services. Refactor
8. Downstream adoption phases (outline only — do not start; the §5 invariants are what make these pure additions)
- DS1 — Consume: mount OS v3 in the downstream distribution; verify session auth end-to-end; adapt the SPA's list controller to the v3 envelope/filter grammar (both offset-based — mechanical); migrate 1–2 read-heavy tables (findings first) behind a feature flag.
- DS2 — Extend: subclass schemas (EPSS, priority, …), extend filter specs, remount via router factories (I4/I5); replace restricted picker endpoints with
?fields=. - DS3 — Write parity: bulk operations (partial-success result shape:
{updated, skipped: [{id, reason}]}) and workflow actions (close,request_review,mark_duplicate, …) as routers over the OS service layer; delete-impact preview; CSV export from the filter contract. - DS4 — Aggregation: dashboard and metrics needs served by a small set of generic aggregation primitives (group-by, time-bucket, top-N) taking the identical filter params (D6) and returning chart-neutral data — never per-chart-type endpoints whose envelopes couple the API to one charting library; chart shaping moves to the client. New
include=members where cheap. - DS5 — App services: saved table preferences (backed by the structured filter representation, D6), permission map, global search. Any bootstrap-style mega-endpoint is split into small cacheable resources when it moves — do not port a monolith.
- DS6 — Retirement: legacy private API surfaces retired domain-by-domain; v2 deprecation window last.
| # | Capability | Disposition |
|---|---|---|
| 1 | {count,next,previous,results} + limit/offset (max 250) |
OS1 (D4) |
| 2 | Filtered-list totals in meta |
OS1 include=counts (D6) |
| 3 | Relation inlining (v2 ?prefetch=, 30+ SPA call sites) |
OS1 refs+expand — strictly better |
| 4 | In-band memoized nested objects | Rejected (D3) — no mainstream precedent; gzip; client machinery gets deleted, not ported |
| 5 | Bulk ops w/ per-item skip reasons | DS3 (services designed for it: I6) |
| 6 | Workflow actions (close/review/duplicate/…) | DS3 (same service extractions as OS3) |
| 7 | Dashboard aggregation engine | DS4 (enabled by D6) — as generic group-by/time-bucket/top-N primitives, not per-chart-type endpoints |
| 8 | Metrics/insights rollups | DS4 |
| 9 | Per-object count endpoints | Covered by include=counts pattern |
| 10 | Saved views / table preferences | DS5 |
| 11 | django-filter vocabulary + multi-sort + match modes | OS1/OS2 filter contract (D6) |
| 12 | Restricted/picker list variants | Covered by ?fields= + RBAC querysets |
| 13 | Long-running job status polling (import / report / celery) | Intentionally removed from v3 scope by architect directive (§12, 2026-07-20) — v3 imports are synchronous; no job resource is reserved |
| 14 | Delete-impact preview | DS3 (GET /<resource>/{id}/delete-preview fits contract) |
| 15 | CSV/XLSX export | CSV shipped in the alpha (§4.15, GET /<resource>/export.csv, same filter contract — D6); XLSX deferred |
| 16 | Global search + meta.by_type |
DS5 |
| 17 | Auth bootstrap + permission map | DS5 (session auth ready since OS1: D8) |
| 18 | Unix-ms datetimes / null-stripping / preformatted users | Rejected (D9) — client-side adaptations |
| 19 | Error contract for field errors + retry markers | OS1 problem+json with fields extension (D9) |
Inventory + classification of every remaining v2 API-shaped test suite (the ones the §6 backlog
item names, plus every test_apiv2_*, plus files discovered by an APIClient/rest_framework/-list
scan). Legend: PORTED = covered by the v3 suite today · PORT-LATER = maps to a v3
surface not yet built (a remaining checkbox) · SKIP = covers endpoints/features v3 deliberately
lacks in the alpha (§2) · SUPERSEDED = v3 ships a stronger equivalent. test_rest_framework.py
(the 4241-line master suite) is split by area.
| v2 suite / area | Disposition | Covered-by / needs / reason |
|---|---|---|
test_apiv2_scan_import_options.py (ScanImportOptions) |
PORTED | test_apiv3_import_options_corpus.py (2 scenarios ported via import_zap_scan override) + test_apiv3_import_corpus.py (priority 1). |
test_rest_framework.py → ImportScanTest / ReimportScanTest |
PORTED | test_apiv3_import_corpus.py (71 cases) + test_apiv3_import.py (DB-state equivalence). |
test_jira_import_and_pushing_api.py (import/reimport push, push_all, keep-in-sync, PATCH/PUT push, verified-enforcement — non-group) |
PORTED | test_apiv3_jira_push.py (priority 2, mocked JIRA layer) + test_apiv3_finding_writes.py (PATCH keep-in-sync + push-failure-400). Coverage map in §12 (2026-07-21). |
test_rest_framework.py → Findings/Products(Assets)/ProductType(Organization)/Engagements/Tests/Users CRUD + schema + prefetch + RBAC |
PORTED | per-resource test_apiv3_{findings,assets,organizations,engagements,tests,users}.py (+ finding writes). |
test_rest_framework.py → Location / URL / LocationFindingReference / LocationProductReference (reads + edge shape) |
PORTED | test_apiv3_locations.py (read-only + edge-status shape, URL-only registry). Edge/reference writes → SKIP (v3 locations read-only in alpha, D5). |
test_rest_framework.py → Notes / Files |
PORTED | test_apiv3_subresources.py (notes/tags/files matrix incl. note-create side-effects). |
test_rest_framework.py → FindingCreateUpdateMitigatedFields |
PORTED | test_apiv3_finding_writes.py (EDITABLE_MITIGATED_DATA both states). |
test_apiv2_prefetch_rbac.py (PrefetchRBAC gate) |
SUPERSEDED | test_apiv3_expand_rbac.py — prefetch→expand RBAC leakage sweep (OS6). |
test_apiv2_methods_and_endpoints.py (method/endpoint smoke) |
SUPERSEDED | test_apiv3_openapi.py (path/tag guard) + test_apiv3_authz_sweep.py + test_apiv3_authz_static.py + per-resource query/RBAC sweeps. |
test_rest_framework.py → SchemaChecker (per-endpoint OpenAPI validation) |
SUPERSEDED | test_apiv3_openapi.py (schema renders + path/tag presence); pydantic DTOs generate the schema (D3). |
test_fk_reassignment_authorization.py (FK-reassign authz) |
SUPERSEDED | v3 write RBAC re-checks add on a reassigned parent (engagement→product, test→api_scan_config) — test_apiv3_engagements.py/test_apiv3_tests.py/test_apiv3_assets.py (§12 OS3b). |
test_api_finding_filter_no_multiply.py (to-many filters don't multiply rows) |
SUPERSEDED (for v3's vocabulary) | v3 tags__in uses distinct=True + the assertNumQueries list guards row multiplication. The specific finding_group/found_by/reviewers/risk_acceptance filters → SKIP (not in the v3 alpha finding vocabulary, §4.9). |
test_apiv2_user.py (User CRUD + perms) |
PORTED (core) | test_apiv3_users.py (CRUD, RBAC, password guards). configuration_permissions on user writes → PORT-LATER (already a §6 backlog item). |
test_apiv2_test_dedupe_policy.py (Test API exposes matching policy read-only) |
PORTED | test_apiv3_test_dedupe_policy.py — TestDetail exposes deduplication_algorithm + hash_code_fields as read-only computed fields (reusing the v2 Test.deduplication_algorithm/Test.hash_code_fields model properties verbatim). v3 rejects writes to them (400 unknown-field, extra="forbid") where v2 silently ignores them — deliberate hardening (§12, 2026-07-21). |
test_product_authorized_users_api_authz.py (authorized_users gated by Product_Manage_Members) |
PORT-LATER | v3 AssetWrite does not yet expose authorized_users (member-management is a distinct op). When added, needs the Product_Manage_Members gate + these authz cases. |
test_rest_framework.py → UserProfileTest |
PORT-LATER | v3 self-profile endpoint (already a §6 backlog item). |
test_rest_framework.py → ConfigurationPermissionTest |
PORT-LATER | configuration_permissions (same §6 backlog item as the user sliver). |
test_rest_framework.py → FindingClose / FindingVerify / EngagementCloseReopen / FindingActionAuthz |
SKIP | workflow actions — §2 non-goal (DS3). Status fields are writable and tested; the action endpoints are not. |
test_bulk_risk_acceptance_api.py |
SKIP | bulk operations — §2 non-goal (DS3). |
test_risk_acceptance_api.py + test_rest_framework.py → RiskAcceptanceTest |
SKIP (endpoint) | the Risk_Acceptance CRUD/workflow resource is not a v3 alpha surface. The finding-level risk_accepted invariant (simple_risk_accept / unaccept) is PORTED in test_apiv3_finding_writes.py. |
test_apiv2_endpoint.py + test_rest_framework.py → Endpoint / EndpointStatus / V3EndpointStatus + test_endpoint_status_cross_product_authz.py |
SKIP | legacy Endpoint model — v3 exposes Locations instead (D5); /api/v3/ carries no Endpoint path. |
test_endpoint_meta_import.py / test_generic_meta_import.py / test_apiv2_metadata.py (DojoMeta) + test_rest_framework.py → FindingMetadata |
SKIP | metadata + endpoint-meta import endpoints — not a v3 alpha surface (§2). |
test_apiv2_notifications.py + test_rest_framework.py → Notifications / NotificationWebhooks / Announcement |
SKIP | notifications / webhooks / announcements endpoints — §2 non-goal. |
test_rest_framework.py → Jira{Instances,Issues,Project} |
SKIP (config endpoints) | JIRA configuration endpoints not in v3 alpha; JIRA push behavior is covered (priority 2). |
test_rest_framework.py → Sonarqube* / {Product,Asset}_API_Scan_Configuration / ToolConfigurations / ToolProductSettings / ToolTypes / AppAnalysis + test_api_sonarqube_updater.py |
SKIP | tool-config / SCA / scan-configuration endpoints — not a v3 alpha surface. |
test_rest_framework.py → Language / LanguageType / ImportLanguages / NoteTypes / FindingTemplates / BurpRawRequestResponse / ReportGenerateFormat + test_apiv2_limit_reqresp.py |
SKIP | languages / note-types / templates / burp req-resp pairs / report generation — none are v3 alpha surfaces (§2). |
test_rest_framework.py → DevelopmentEnvironment / TestType (standalone CRUD) |
SKIP (as endpoints) | v3 exposes these only as {id,name} refs / expand targets (§4.4), not as CRUD endpoints in the alpha; the ref rendering is PORTED. |
test_importers_importer.py |
SKIP (unit, not endpoint) | importer-internal unit tests; v3 endpoint equivalence is covered by the import corpus. |
test_location_reference_write_authz.py |
SKIP | location-reference write authz — v3 locations are read-only in the alpha (D5). |
Trivial high-value ports done from this table: none at evaluation time. Every PORT-LATER item
required new v3 schema/route surface (matching-policy read fields, configuration_permissions,
self-profile, authorized_users member-management) that exceeded the <30-min trivial-port bar and
was already tracked as a §6 backlog checkbox — so nothing was implemented during the evaluation
itself (conservative, §10.2). Subsequently closed: the matching-policy read fields were ported in
test_apiv3_test_dedupe_policy.py (§12, 2026-07-21).
⚠ v2-sunset prerequisites (do NOT delete these v2 test files without doing the step first):
test_jira_import_and_pushing_api.pyis the ONLY consumer of the JIRA VCR cassettes. Cassettes are not self-executing — deleting this suite silently orphans the sole coverage of the shared JIRA engine behaviors (issue transitions on mitigation/reactivation,keep_syncreimport flows,enforce_verified_statusgating). The v3 JIRA tests deliberately cover only dispatch/plumbing at the service boundary and inherit engine coverage from this suite. Before removal: record v3-keyed cassettes (or an equivalent live-JIRA integration harness) for the engine scenarios — naturally paired with the post-alpha workflow-actions work, which makes those transitions v3-endpoint-observable.test_import_reimport.pyis imported by the v3 corpus shim.unittests/api_v3/ import_corpus_shim.pysubclassesImportReimportMixinfrom it (the anti-fork design). Before removal: relocate the mixin to a shared module (e.g.unittests/import_corpus_mixin.py) so the v3 corpus keeps running verbatim.
- Never modify v2 (serializers, viewsets, urls, prefetch) or any existing test. v3 is additive. The only shared-file edits allowed:
dojo/urls.py(mount),requirements.txt, settings (new constantsCOUNT_CAP,EXPAND_BUDGET). - Contract is §4; invariants are §5. If the contract is ambiguous or a model field doesn't match §4.5, do not improvise silently — pick the conservative option, implement it, and record the question + choice in §12.
- Tests: all v3 tests live in the
unittests/api_v3/package (base classApiV3TestCase); run only via./run-unittest.sh --test-case unittests.api_v3[.module] 2>&1 | tee /tmp/test_output.log; analyze withgrep -E "PASSED|FAILED|ERROR" /tmp/test_output.log; narrate each iteration (what failed, the fix, then re-run). Neverpytest/manage.py testdirectly. Contract tests go through the Django test client (in-process, full stack);ninja.testing.TestClientonly for kernel-internal units (§6 preamble). - Reuse, don't rewrite: RBAC only through
dojo/<resource>/queries.pyget_authorized_*+dojo.authorizationpermission semantics; import only throughdojo/importers/*wrapped by the new facade; JIRA throughdojo/jira/services.py. Reference implementations to read but not touch:dojo/finding/api/serializer.py:438-560,dojo/api_v2/serializers.py:526-551,dojo/finding/api/views.py:412-586. - Every list endpoint ships with an
assertNumQueriestest proving query count is independent of row count, with and withoutexpand. This is non-negotiable — it is the project's headline claim for v3. - Lint: ruff config is in-repo; run it before finishing a phase. Match surrounding code style; comments only for non-obvious constraints.
- Never commit
CLAUDE.md, anything under.claude/, or scratch/log files. - Phase order is OS1→OS6; do not start OS2 before the OS1 gate outcome is recorded in §12 by the architect — the gate is reviewed, never self-certified. Produce the gate report, then stop and wait.
- Query-count regression suite green (constant queries, ±expand) — the headline claim, tested.
- Import equivalence: v3
POST /import(all three modes) reproduces v2 import/reimport DB state for identical payloads, incl.close_old_findings. - RBAC parity: expand/include never leak unauthorized objects (port of prefetch-RBAC test intent).
- Contract tests: slim shapes (incl. denormalized finding parent refs,
locations_count), expand permutations + cycle guard + budget,fields,include=counts, filter vocabulary snapshot, problem+json on every failure path, both auth modes. - Locations: edge-status shape, URL-only, flag-off ⇒ no
/api/v3/. - OpenAPI generation + client codegen succeed (alpha-tagged).
- Benchmark artifact in PR: v2
?prefetch=vs v3?expand=(query count + latency), 100-row findings list. - Full existing v2 test suite untouched and green.
| Date | Entry |
|---|---|
| 2026-07-19 | Plan created from the design spec + three audits (OS v2 API, downstream consumer usage, Location subsystem). Decisions D1–D10 recorded in §1; supersedes spec where they conflict (notably: finding↔location cardinality, pagination default, memoization rejection, importer-facade-as-new-work). |
| 2026-07-19 | Downstream-anchoring review ("current downstream usage is not written in stone"): four inherited habits replaced with better designs — (1) structured POST /<resource>/query filter representation reserved; bespoke _any/_all suffix params banned (D6); (2) generic /jobs/{id} resource reserved for all async work instead of the three per-feature status mechanisms observed downstream (§4.13); (3) DS4 aggregation as chart-neutral primitives, not chart-library-shaped per-chart endpoints; (4) DS5 splits any bootstrap-style mega-endpoint into cacheable resources. Checked and kept deliberately: offset-default pagination (dual-mode envelope is the hedge if page-jump UX ever goes) and session+CSRF auth (independently correct for first-party SPAs). |
| 2026-07-19 | Duplication across UI views / v2 / v3 addressed via convergence strategy (D7, §7): v3 services are written for all three consumers but only v3 calls them in the alpha PR (stays additive); extraction reconciles serializer + UI-view semantics with divergences logged here; post-alpha CONV1 (v2→services) then CONV2 (UI views→services, behavior-pinning tests first) then CONV3 (delete dead duplicates). |
| 2026-07-19 | Alpha URL carries the stability marker: mount at /api/v3-alpha/, moving to /api/v3/ at beta and staying there through GA (D1, §4.1). Rationale: URL is the only unmissable instability signal (K8s/Google precedent); structuring beta+GA on one URL limits clients to a single migration, timed to the additive-only contract freeze. Prefix is one constant; router factories make the remount a one-liner. |
| 2026-07-19 | Count strategy upgraded from "capped at COUNT_CAP" to hybrid exact→planner-estimate (D4, §4.3): above the cap, count becomes the Postgres EXPLAIN row estimate (PostgREST count=estimated technique), clamped to ≥ CAP+1, flagged count_exact: false. Rationale: real approximate numbers beat a "10000+" floor for pagination UX at ~zero extra cost; pg_class.reltuples rejected (whole-table only, useless under filters/RBAC). |
| 2026-07-19 | OS1 GATE: GO on django-ninja — architect released all OS phases. All 7 exit criteria met; benchmark shows v3 query count constant (5 no-expand / 7 with expand=test.engagement) at both 10 and 100 rows vs v2 scaling 91→271 (no prefetch) / 222→762 (?prefetch=test). Evidence in .claude/os1-gate-report.md (gate report reviewed by coordinator). Coordinator review found+fixed one pre-commit bug: locations_count needed Count(..., distinct=True) against tags__in joins. Latency benchmark deferred to OS6. |
| 2026-07-19 | Trailing slash decided: NO trailing slash on route paths (/findings, /findings/{id}, /import) — ninja-idiomatic, matches GitHub/Stripe convention. §4.3 example updated. next/previous are opaque (D4) so this binds only documentation and client codegen. |
| 2026-07-20 | RESOLVED (2026-07-20, architect sign-off): exact django-ninja/pydantic pins + license/supply-chain sign-off — pins accepted as-is. OS1 implementer note: pinned django-ninja==1.6.2 (latest stable 1.x at implementation; MIT). It requires pydantic>=2,<3; pydantic==2.13.4 (+pydantic-core==2.46.4, MIT) is already present in the image as a transitive dependency, so no new top-level pin was added for pydantic. Both are widely used, actively maintained. |
| 2026-07-20 | RESOLVED (2026-07-20, architect sign-off): COUNT_CAP/EXPAND_BUDGET defaults (10 000 / 10 proposed) — defaults accepted as-is. OS1 implementer note: implemented as DD_API_V3_COUNT_CAP=10000 and DD_API_V3_EXPAND_BUDGET=10 env settings (API_V3_COUNT_CAP/API_V3_EXPAND_BUDGET), settings-overridable per §4.3/§4.6; tests exercise a lowered cap. |
| 2026-07-19 | OS1 impl — CSRF wiring (§4.2 conservative choice): §4.2 specifies NinjaAPI(csrf=True), but django-ninja 1.6.x removed the csrf parameter from NinjaAPI.__init__. Conservative choice: rely on ninja's SessionAuth (ninja.security.django_auth), which defaults csrf=True and enforces CSRF on unsafe methods for cookie/session auth, while TokenAuth (an APIKeyHeader) requires no CSRF. This achieves the D8 contract exactly (token bypasses CSRF; session enforces it) — verified by test_apiv3_auth. No csrf= kwarg is passed to NinjaAPI. |
| 2026-07-19 | OS1 impl — query-count fidelity (beyond §6 literal base-queryset): §6 OS1 lists the findings base queryset as select_related(...) + annotate(locations_count). To keep the headline query count constant, the route also prefetch_related("tags") (FindingSlim exposes tags[], §4.5) and the expand planner additionally pre-loads each expand-target schema's own SELECT_RELATED/PREFETCH_RELATED (e.g. test__tags) so serializing expanded objects issues no per-row queries. Purely additive to the spec; required by criterion 1. |
| 2026-07-19 | OS1 impl — expand-target slim schemas placement: §6 OS1 lists only FindingSlim/FindingDetail in dojo/finding/api_v3/schemas.py, but ?expand=test.engagement needs slim schemas for the targets. TestSlim, EngagementSlim, ProductSlim, ProductTypeSlim, UserSlim, TestTypeSlim, EnvironmentSlim are defined there for OS1 (findings-only). OS3 relocates the canonical copies to their resource modules (dojo/<resource>/api_v3/schemas.py); these will re-export or move. |
| 2026-07-19 | OS1 impl — login-redirect exemption (settings): DefectDojo's LoginRequiredMiddleware redirects unauthenticated requests to /login unless the path matches LOGIN_EXEMPT_URLS. v3 was appended to that list (mirroring ^api/v2/) so anonymous v3 requests get a 401 problem+json instead of a UI redirect. Uses the single-source API_V3_URL_PREFIX constant. |
| 2026-07-19 | OS1 impl — route path trailing slash (§4.3 example vs implementation): routes are registered without a trailing slash (/findings, /findings/{id}, /import) — django-ninja's idiomatic convention and it resolves cleanly under APPEND_SLASH. The §4.3 example shows .../findings/?... with a trailing slash; this does not bind clients because next/previous are opaque URLs (D4) built from the actual request path. Flagged for architect confirmation; switching to trailing slashes is a one-line change per route if preferred (would then need the v2-style POST-trailing-slash middleware handling). |
| 2026-07-19 | OS1 impl — response rendering: routes return a pre-built JsonResponse (custom encoder gives ISO-8601 Z datetimes + X-API-Status header) which ninja passes through unchanged, so ?expand=/?fields= can reshape results dynamically while the declared response= schema still documents the base shape in OpenAPI. Base slim serialization is schema-driven (Schema.model_validate(obj) runs ninja resolvers), so a subclassed schema serializes its new fields automatically (I4). |
| 2026-07-19 | OS2 impl — severity ordering direction (§4.9 ambiguity, conservative choice): o=severity sorts by rank (Critical first) and o=-severity reverses it (Info first). §4.9 names the rank order "(Critical>High>Medium>Low>Info)" but does not say which sign is "ascending"; resolved so plain o=severity yields exactly that listed order (also DefectDojo's canonical display order and Finding.Meta.ordering). Implemented in filtering.py as a query-time Case/When (SEVERITY_RANK Critical=0…Info=4) exposed via FilterSpec.order_expressions, mirroring v2's numerical_severity ordering but computed at query time so it never depends on the denormalized numerical_severity column being populated. |
| 2026-07-19 | OS2 impl — unknown filter param → 400 (§6 OS2 item 2, conservative choice): django-filter silently ignores query params it doesn't declare; §6 OS2 says "keep unknown-filter 400 behavior". Resolved by making apply_filters reject any query param that is neither reserved (RESERVED_PARAMS) nor a declared filter with a 400 filter problem+json, strengthening the single filter contract (D6). Caveat: cache-buster/tracking params (e.g. jQuery's _=) would 400 — acceptable for an alpha API and consistent with a strict typed contract. |
| 2026-07-19 | OS2 impl — ?fields= gets its own error type (I9): unknown ?fields= values previously reused the expand problem type; OS2 adds a dedicated fields type URI (.../errors/fields) so the closed error contract has one type per error kind (I9). Pure add; no shape change. |
| 2026-07-19 | OS2 impl — expand=locations edge-row shape (§4.6/§4.14 reconciliation): expand=locations on a finding renders [{location: {id,name,type}, status, audit_time}] and replaces locations_count in the output (§4.6 "swaps"), driven by prefetch_related("locations__location") (query count stays constant, verified by test). §4.14's additional auditor{ref} on finding-location rows is deferred to the OS4 /findings/{id}/locations sub-resource (OS2 delivers only the expand= projection per the OS2 task's explicit shape). Implemented generically via ExpandRel.special/prefetch_paths/replaces so any resource can declare a computed, prefetch-backed, count-replacing expansion. |
| 2026-07-19 | OS2 impl — filter-contract snapshot storage (read-only test FS): unittests/api_v3/snapshots/filters.json is checked in and compared on every run; the auto-create-on-missing branch of test_apiv3_filter_contract only works on a writable FS (the docker test bind-mount is read-only), so the baseline is generated on the host. Deliberate regeneration: DD_API_V3_UPDATE_SNAPSHOTS=1 (host run). |
| 2026-07-19 | OS2 impl — I5 dependency direction, verified: the seven reusable kernel machinery modules (auth, errors, expand, filtering, include, pagination, refs) import zero resource/service modules. The only resource import under dojo/api_v3/* is a deferred import inside api.build_api() (the composition root/mount that wires the OS router factories) and import_routes.py's use of dojo.importers.* (services, explicitly allowed). Reusable machinery stays resource-agnostic, keeping I5 honest; whether api.py (the mount) belongs in the kernel package or a separate mount package is left as an OS-cleanup question for the architect. |
| 2026-07-19 | OS3a scope + no PUT (§6 OS3, decision recorded per task): CRUD for the three simpler resources product_type, product, user shipped (dojo/<resource>/api_v3/{schemas,routes}.py + build_<resource>_router() factories, mounted in build_api()). Each exposes GET list, GET /{id}, POST, PATCH /{id}, DELETE /{id}, no trailing slash. No PUT in alpha — PATCH-only partial update (pydantic model_dump(exclude_unset=True)); a full-replace PUT is an additive, non-breaking addition available later if a consumer needs it. Engagement/test/finding writes are explicitly OS3b, not delivered here. |
| 2026-07-19 | OS3a impl — canonical slim relocation (fulfils the OS1 §12 placement note): ProductTypeSlim/ProductSlim/UserSlim moved from dojo/finding/api_v3/schemas.py to their resource modules (dojo/<resource>/api_v3/schemas.py); the finding module now imports (re-exports) them so there is exactly one canonical class per model — verified at runtime that finding.api_v3.schemas.ProductSlim is product.api_v3.schemas.ProductSlim (and likewise product_type/user), so finding expand= targets and the resource endpoints serialize through the same schema (I4). No duplicate class. TestSlim/EngagementSlim/TestTypeSlim/EnvironmentSlim stay in the finding module (their resources are OS3b). Import direction finding → product → product_type, finding → user (no cycle). All 65 OS1/OS2 tests stay green. |
| 2026-07-19 | OS3a v2-semantics finding — deletion (mirrored exactly): ProductTypeViewSet.destroy / ProductViewSet.destroy both do: async_delete().delete(instance) when get_setting("ASYNC_OBJECT_DELETE") is truthy, else a synchronous instance.delete() wrapped in with Endpoint.allow_endpoint_init(): (the Endpoint context is required while V3_FEATURE_LOCATIONS is on, because Endpoint.__init__ otherwise raises during the delete cascade). v3 DELETE reproduces this verbatim (_destroy() in each routes module). UsersViewSet.destroy is different: no async, no Endpoint context — a plain instance.delete(), except a user may not delete themselves (v2 returns 400 "Users may not delete themselves"); v3 mirrors this (400 problem+json, self left intact). |
| 2026-07-19 | OS3a v2-semantics finding — user visibility (v2-parity view_user gate + guaranteed self-read; coordinator-corrected): §6 OS3 says "Users: read + self; admin-only writes". v2 UsersViewSet gates all access behind UserHasConfigurationPermissionSuperuser (DjangoModelPermissions: auth.view_user for GET) and returns User.objects.all(). v3 _base_queryset mirrors this (conservative per §10.2, never widening PII/email exposure beyond v2): holders of auth.view_user (superusers/staff via the OS bypass; a non-staff holder via explicit grant) get the RBAC-scoped get_authorized_users("view", user=request.user) queryset — for a non-staff holder that is the collaborator subset (co-members of their authorized products/types, plus superusers), i.e. ≤ v2's "all users" exposure; everyone else sees only themselves (Dojo_User.objects.filter(pk=request.user.pk)), so a plain user lists exactly their own record and GET /users/{own id} always resolves (the previous collaborator-for-all design was rejected: it exposed email to every authenticated user and 404'd a plain user's own record). Writes stay admin/superuser-only via the auth.add_user/change_user/delete_user configuration permissions (user_has_configuration_permission; superusers pass automatically). UserSerializer.validate() rules ported verbatim into the route: only superusers may add/edit superusers or staff; password is write-only and cannot be changed via PATCH (400); a password is required on create when REQUIRE_PASSWORD_ON_USER (400 otherwise); password run through Django validate_password. configuration_permissions (a v2 write field) is intentionally out of the alpha write surface (not in the read shape; M2M member-management concern) — additive later. |
| 2026-07-19 | OS3a v2-semantics finding — write permission checks (mirror the v2 permission classes exactly): product_type (UserHasProductTypePermission): create = user_has_global_permission(user, "add") (the literal string "add" is kept, not the Permissions enum, because the dojo.add_product_type config-permission carve-out keys on permission == "add"); update = object edit; delete = object delete (⇒ staff-only for a non-staff member under the legacy model, so a member can view/edit but a member DELETE is 403 — used as the "authorized-to-see-but-not-modify" 403 test). product (UserHasProductPermission): create = add on the target prod_type in the payload (404 if that product type doesn't exist, 403 if unauthorized — mirrors check_post_permission's get_object_or_404); update = object edit plus add on a reassigned prod_type (re-checked only when the FK actually changes, mirroring check_update_permission); delete = object delete. Reads resolve the object through the authorized view queryset first → 404 for unknown-or-unauthorized, 403 only when visible-but-not-modifiable. permission_to_action was verified to map the Permissions enum and the action string identically, so either form is safe for the object checks. |
| 2026-07-19 | OS3a impl — write-schema strictness + shapes: create/update schemas are ninja Schemas with model_config = {"extra": "forbid"}, so an unknown body field is a 400 problem+json — consistent with the kernel's strict query contract (unknown filter/fields/expand params also 400; I9). Required-vs-optional mirrors the v2 serializers: ProductType requires name; Product requires name+description+prod_type (description is a non-null model field; sla_configuration falls back to the model default when omitted); User requires username+email (+password per REQUIRE_PASSWORD_ON_USER). Relations by integer id (§4.11). §4.5 does not enumerate detail fields for these resources, so the conservative additions are: ProductDetail += business_criticality, platform, origin, external_audience, internet_accessible + product_manager/technical_contact/team_manager refs; UserDetail += is_staff, date_joined; ProductTypeDetail == slim (no heavier fields). Filter vocabularies follow §4.9's minimal template (id__in, name__icontains, parent FK where present, date ranges; orderings id/name/created/updated) and are captured in the regenerated snapshots/filters.json (diff = exactly the three new resources, 73 insertions / 0 deletions, finding section untouched). Per-list-endpoint assertNumQueries-style constant-query tests pass (product_types 2, users 2, products 3 queries, unchanged at 10 vs 100 rows; products constant with expand=product_type too). |
| 2026-07-19 | OS3a impl — no service layer for the three simple resources (I6 note): unlike finding writes (which own JIRA/risk/notification side-effects and get a dojo/finding/services.py in OS3b), product_type/product/user writes are plain ORM CRUD with no cross-cutting orchestration, so the (thin) create/update/delete logic lives directly in the route factories. The one real side-effect — the delete strategy (async vs sync-inside-Endpoint.allow_endpoint_init()) — is a per-route _destroy() helper mirroring v2 exactly. 204 No Content responses return an empty body via HttpResponse(status=204) (not JSON null) while still carrying X-API-Status (§4.1); the kernel errors.py was not extended (only api.py mount was touched, per §10). |
| 2026-07-19 | OS3b scope + canonical-slim relocation: engagement + test CRUD (dojo/engagement/api_v3/, dojo/test/api_v3/) and the finding write path shipped. EngagementSlim relocated to dojo/engagement/api_v3/schemas.py; TestSlim/TestTypeSlim/EnvironmentSlim relocated to dojo/test/api_v3/schemas.py; the finding module now re-exports all four so there is exactly one canonical class per model (is-identity asserted in test_apiv3_engagements.TestApiV3EngagementsRelocation and test_apiv3_tests.TestApiV3TestsRelocation, mirroring the OS3a pattern). Import direction stays acyclic: finding → {engagement, test}, test → engagement, engagement → {product, product_type, user}. No PUT (PATCH-only, per the OS3a decision). Snapshot regenerated additively: filters.json = +engagement +test (66 insertions / 1 formatting deletion; finding/product/product_type/user sections unchanged, verified by the order-independent contract test). All 197 unittests.api_v3 tests green. |
| 2026-07-19 | OS3b v2-semantics finding — engagement/test write permissions (mirror the v2 permission classes exactly): engagement (UserHasEngagementPermission): create = add on the target product in the payload (404 if the product doesn't exist, 403 if unauthorized — mirrors check_post_permission(request, Product, "product", "add")); update = object edit plus add on a reassigned product (re-checked only when the FK actually changes, mirroring check_update_permission); delete = object delete (⇒ staff-only for a non-staff member under the legacy model). test (UserHasTestPermission): create = add on the target engagement (404/403) plus view on api_scan_configuration when present (mirrors the required=False sibling check_post_permission); update = object edit plus view on a reassigned api_scan_configuration (mirrors check_update_permission(request, obj, "view", "api_scan_configuration")) — note the update path does not re-authorize any engagement reassignment because engagement is editable=False and therefore not writable on update (mirrors v2, where the update serializer treats it read-only); delete = object delete (staff-only). Reads resolve through the authorized view queryset first → 404 for unknown-or-unauthorized, 403 only when visible-but-not-modifiable. Permissions.Engagement_Add/Test_Add/Finding_Add map to Action.Add identically to the literal string "add" (verified via permission_to_action), so either form is safe. |
| 2026-07-19 | OS3b v2-semantics finding — engagement/test deletion (mirrored exactly; differs from product/product_type): EngagementViewSet.destroy and TestsViewSet.destroy both do async_delete().delete(instance) when ASYNC_OBJECT_DELETE is truthy, else a plain synchronous instance.delete() — crucially with no Endpoint.allow_endpoint_init() wrapper (unlike ProductViewSet/ProductTypeViewSet destroy, which do wrap it). v3 _destroy() in each routes module reproduces this verbatim: no Endpoint context for engagement/test. (Cascade to findings under V3_FEATURE_LOCATIONS goes through LocationFindingReference, not the legacy Endpoint, so the wrapper is unnecessary here — and mirroring v2 exactly is the mandate.) |
| 2026-07-19 | OS3b impl — engagement/test write-schema shapes (editable subset, conservative FK set): required-vs-optional mirrors the v2 ModelSerializer requiredness rule (required = not (blank or has_default or null)): Engagement requires product+target_start+target_end; Test requires engagement+test_type+target_start+target_end (environment is null=True ⇒ optional). EngagementSerializer.validate (POST: target_start > target_end → 400) ported to _validate_dates, applied on create and on any update that changes either date. Conservative FK subset in the write surface (additive later, per §10.2): engagement exposes lead only (dropped preset/report_type/requester/build_server/scm_server/orchestration_engine tool-config FKs — niche, and each needs its own resolve+validate); test exposes test_type/environment/lead (resolved by id → 400 on bad pk, mirroring DRF PrimaryKeyRelatedField) and api_scan_configuration (permission-gated: 404 if absent, 403 without view). EngagementDetail/TestDetail add conservative heavier read fields (§4.5 doesn't enumerate them): engagement += description/version/first_contacted/reason/tracker/test_strategy/threat_model/api_test/pen_test/check_list/build_id/commit_hash/branch_tag/source_code_management_uri/deduplication_on_engagement; test += description/scan_type/version/build_id/commit_hash/branch_tag (api_scan_configuration left out of the read shape — its model has no Ref label registry entry; additive later). Per-list assertNumQueries-style constant-query tests pass, including with expand. |
| 2026-07-19 | OS3b — finding write service (D7 flagship extraction) reconciliation + divergences: dojo/finding/services.py provides create_finding/update_finding/delete_finding (I6: keyword-only, explicit user, no HTTP context; validation raises rest_framework.exceptions.ValidationError — the same exception the reference serializer raises — which the kernel's DRF boundary adapter maps to 400 problem+json, §12 OS1). It reconciles the v2 serializer path (FindingSerializer/FindingCreateSerializer .update/.create/.validate, dojo/finding/api/serializer.py:328-744) with the UI flow (EditFinding in dojo/finding/ui/views.py:718-1053). The serializer path is the chosen canonical for the API (the plan names it as the reference). UI-only side-effects are consciously deferred to the convergence track (CONV2), not baked into the API. Full divergence table in .claude/os3b-report.md. The material divergences and chosen canonical behavior: (1) risk-acceptance — serializer's process_risk_acceptance (simple_risk_accept/risk_unaccept, gated on enable_simple_risk_acceptance, run in validate) is canonical; the UI's perform_save=False + explicit finding.save() variant is deferred. (2) JIRA push — canonical = serializer semantics: update pushes synchronously with force_sync=True and raises on failure (push_to_jira or is_keep_in_sync); create pushes without force_sync; the route OR-s push_to_jira with jira_project.push_all_issues (mirrors perform_update). UI-only jira link/unlink/change-key and finding-group push are deferred. (3) status side-effects — the UI's process_mitigated_data (auto-is_mitigated+location-edge mitigation when active flips off), process_false_positive_history (retroactive FP reactivation), last_reviewed/last_reviewed_by stamping, burp req/resp, finding-group handling, and github are all UI-only and deferred; the API mirrors only the serializer's field-level updates. (4) notifications — create dispatches the finding_added notification exactly as the serializer does; update dispatches none (matches the serializer). (5) numerical_severity — not set in the service (the model's save() computes it; the UI sets it explicitly, redundantly). |
| 2026-07-19 | OS3b impl — finding write specifics (conscious choices): (a) cve mirror — the v3 write schema exposes a flat vulnerability_ids: list[str] (§4.11), not the v2 nested vulnerability_id_set; create_finding writes vulnerability_ids[0] into Finding.cve before the initial save() (mirroring the serializer, which sets validated_data["cve"] pre-save) because save_vulnerability_ids only sets finding.cve in memory; update_finding calls save_vulnerability_ids before finding.save() so the mirror persists. Empty/absent vulnerability_ids is a no-op (mirrors the serializer's truthy guard). (b) CWE — v3 exposes a scalar cwe (not the v2 nested cwes list). Create always persists the primary Finding.cwe as a Finding_CWE row (save_cwes, mirrors serializer). On update, save_cwes is called whenever the scalar cwe is in the change set (the closest analog to the v2 finding_cwe_set-provided guard) so Finding.cwe and its Finding_CWE rows stay consistent — this resyncs to just the primary CWE, wiping any extra imported Finding_CWE rows (documented divergence: v2's scalar-cwe update alone does not touch Finding_CWE). (c) delete_finding — mirrors FindingViewSet.destroy's dedup/grading hooks via the model's Finding.delete() (finding_helper.finding_delete + perform_product_grading); the plan-mandated signature delete_finding(finding, *, user) omits v2 destroy's push_to_jira query-param (niche delete-time JIRA closure push) — additive later. (d) mitigated-edit rules — can_edit_mitigated_data (requires EDITABLE_MITIGATED_DATA and superuser) gates any attempt to set non-null mitigated/mitigated_by, on both create and update (ported verbatim from both serializers). |
| 2026-07-19 | OS3b — RBAC 403 test shape under OS legacy auth (view-but-not-edit not expressible): the task asks for a "403 view-but-not-edit" finding test, but the OS legacy authorization model (user_has_permission) grants View/Edit/Add equally to any authorized_users member (only Delete is staff-gated). So a member who can see a finding/engagement/test can also edit it — a view-but-not-edit 403 is not expressible without a granular RBAC role model. Conservative resolution (mirrors the OS3a product RBAC test): the "authorized-to-see-but-not-modify" 403 is demonstrated on delete (member GET 200, member DELETE 403 staff-only), and the 404-for-unauthorized case is tested with a non-member (limited) user. Recorded so the API contract test is not mistaken for asserting an edit/view permission split that OS legacy does not implement. |
| 2026-07-19 | OS4 scope + sub-resource placement: read-only Locations resource (GET /locations, GET /locations/{id}) + the GET /findings/{id}/locations and GET /products/{id}/locations edge sub-resources shipped in a new dojo/location/api_v3/{schemas,routes}.py. All three routers are factories (I5) mounted in build_api(). Placement choice: the finding/product location sub-resources live in the location module (not the finding/product route factories) so all location code stays cohesive and the finding/product factories are untouched (lower regression risk); each is an independent build_<x>_router() and the /findings/{id}/locations path resolves cleanly alongside /findings/{id} (distinct URL patterns). Read-only: locations are import-driven (§4.14); no write routes. All 229 unittests.api_v3 tests green (198 prior + 31 new). |
| 2026-07-19 | OS4 v2-semantics finding — Location RBAC (verified superuser-only; mirrored): the v2 LocationViewSet (dojo/location/api/views.py) is a ReadOnlyModelViewSet with permission_classes = (IsSuperUser, DjangoModelPermissions) — AND-ed, and IsSuperUser.has_permission = request.user.is_superuser, so the resource is superuser-only (confirmed). v3 /locations mirrors this exactly: the router gates the whole resource behind request.user.is_superuser → 403 problem+json for any non-superuser (both list and detail), and draws rows from get_authorized_locations("view", user=...) (I8, forward-compatible — a downstream distribution can scope the queryset without a route change; in OS that helper returns all locations, matching v2's Location.objects.order_by_id()). The sub-resources are not superuser-gated: they use parent-inherited authorization — the parent finding/product is resolved through get_authorized_findings/get_authorized_products and an unknown or unauthorized parent is a 404 (never leak existence, §4.10); edges are then read from the parent's own reverse manager (finding.locations / product.locations), matching the v2 LocationFindingReference/LocationProductReference viewsets' RBAC-scoped querysets. |
| 2026-07-19 | OS4 — product edge shape reconciliation (§4.14 shorthand vs model): §4.14 lists both finding- and product-location edge rows as {location, status, audit_time}, but LocationProductReference has no audit_time/auditor columns (only status, relationship, relationship_data). Resolved per the OS4 task's explicit item-2 shapes and the model: finding edge = `{location: {id,name,type}, status, audit_time, auditor: {id,name} |
| 2026-07-19 | OS4 — auditor added to expand=locations edge rows (supersedes the OS2 deferral): OS2 (§12) deferred the auditor{ref} on finding-location rows to OS4; the OS4 task adds it to both the /findings/{id}/locations sub-resource and the existing expand=locations projection. _finding_location_edges now emits {location, status, audit_time, auditor} with prefetch_paths=("locations__location", "locations__auditor") so the query count stays constant (verified). The pre-existing test_apiv3_findings.test_expand_locations_swaps_count_for_edge_rows edge-key assertion was updated to include auditor (a v3 test reflecting the new intended contract, not a v2 test). |
| 2026-07-19 | OS4 — ?fields= / ?expand= interplay (OS2 open question, coordinator decision): the ?fields= allowlist is now schema fields ∪ the schema's registered EXPANDABLE keys, implemented as one kernel helper dojo.api_v3.expand.allowed_field_names(schema) (the single permitted kernel edit besides the mount) and applied at all resource list/detail parse_fields call sites (findings/products/product_types/users/engagements/tests/locations) so the contract is uniform. Effect: ?expand=locations&fields=id,title,locations works; a genuinely unknown name still 400s (dedicated fields type URI, I9); naming an expand key in fields= without expand= is accepted (no 400) but renders nothing (apply_fields keeps only keys present in the serialized dict). Purely widens the allowlist, so no existing 200 becomes a 400. |
| 2026-07-19 | OS4 — flag-off behaviour tested in-process (D5): V3_FEATURE_LOCATIONS=False ⇒ the whole /api/v3-alpha/ tree is absent (the mount in dojo/urls.py is import-time conditional). Made cleanly testable via a URLconf reload: test_apiv3_locations.TestApiV3LocationsFlagOff reloads dojo.urls under override_settings(V3_FEATURE_LOCATIONS=False) (+clear_url_caches()), asserts every v3 path (locations/findings/products/import) raises Resolver404 against the reloaded urlconf, then restores the real (flag-on) urlconf in a finally (reload + clear_url_caches()) so the rest of the suite is unaffected. Verified: the test passes and the other 228 tests stay green in the same run, so the reload does not pollute the shared test process. |
| 2026-07-19 | OS4 — URL-subtype detail fields + query-count fidelity: LocationDetail adds protocol/host/port/path/query/fragment read from the URL subtype via the reverse one-to-one (location.url, related_name="%(class)s"), loaded with select_related("url") on the detail fetch so the six resolvers issue no extra query; a non-URL location (none exist in alpha, D5) renders those fields null (the reverse-O2O accessor's RelatedObjectDoesNotExist is caught). Per-list assertNumQueries-style constant-query tests pass for /locations, /findings/{id}/locations and /products/{id}/locations (sub-resources use select_related("location"[, "auditor"]) — constant regardless of edge count). Filter vocabulary (type, name__icontains, product via products__product; orderings id/name) captured in snapshots/filters.json as an additive-only diff (+location block, all other sections byte-identical; verified by the passing snapshot contract test). Whole-surface query sweep extended with the four new GET endpoints (with location-edge fan-out so per-row queries can't hide) and stays green. |
| 2026-07-19 | OS5 scope + storage support matrix (coordinator scope correction: attach only where the MODEL stores it, verified against dojo/*/models.py; §4.12's "all seven resources" is wrong): three generic factories in dojo/api_v3/subresources.py (build_notes_router/build_files_router/build_tags_router). Model storage: notes (Notes M2M) — engagement, test, finding; files (FileUpload M2M) — engagement, test, finding; tags (TagField) — product, engagement, test, finding, location. product_type/user have none. Attachment: notes+files on finding/engagement/test; tags on finding/engagement/test/product. Location tags NOT attached despite the TagField: location is a read-only, superuser-only resource (OS4) with no v2 tag-mutation endpoint to mirror, and its tags are already surfaced via LocationSlim.tags[] (§4.5) — a tag sub-resource would be a redundant read + an invented write (conservative, §10.2). Unsupported combos return 404 (path not registered); tested. Total surface: 13 new GET paths + POST/PUT/DELETE. All 252 unittests.api_v3 tests green (229 prior + 23 new). |
| 2026-07-19 | OS5 — note privacy = v2 parity (return all; private is not a per-user read filter): verified v2's notes @action GET returns parent.notes.all() (no privacy filter) and the UI comments.html shows every note to any viewer; private only excludes a note from generated reports (dojo/reports/ui/views.py finding.notes.filter(private=False)). So v3 mirrors v2 exactly: the notes list returns all notes (private included) to anyone who can view the parent, with private exposed on NoteSchema so clients can label/exclude. Inventing per-user filtering would be a behavior divergence, not a mirror. Tested (test_private_note_visible_to_other_authorized_user_v2_parity). |
| 2026-07-19 | OS5 — NoteSchema/FileSchema field mappings (model has no matching columns; conservative): NoteSchema.created←Notes.date, updated←Notes.edit_time (the model has no created/updated; edit_time defaults to now at creation, and edited distinguishes a real edit). note_type is out of the alpha write/read surface (§4.12 POST body is {entry, private?}; the v2 single-note-type check is moot without it — additive later). FileSchema.created is always null: FileUpload has no creation timestamp column; size←file.size (a storage stat, not a DB query — keeps list query counts flat). |
| 2026-07-19 | OS5 — tags sub-resource semantics (new convenience API; v2 has only remove_tags): body is a JSON {"tags": [...]} list (§4.11), extra="forbid" (unknown field → 400). GET → {"tags":[...]}; PUT replaces, POST appends (order-preserving, case-insensitive dedup), both return 200 {"tags":[...]}; DELETE /tags/{tag} removes one and returns 204, 404 if absent (the OS5 task's explicit contract — differs from v2 remove_tags's 400-on-absent). Write path mirrors v2 remove_tags exactly: parent.tags = <list>; parent.save() — so tagulous force_lowercase normalization and the tag-inheritance signals fire identically (assigning a list, not a rendered string, is safe for tags with spaces/commas). The {tag} path param is lowercased before matching (stored tags are force-lowercase). No FilterSpec (parent-scoped, simple) ⇒ no snapshot regeneration (§6 OS5). |
| 2026-07-19 | OS5 — file upload/download (mirror v2 validation): extension validated via FileUpload.clean() against settings.FILE_UPLOAD_TYPES (mirrors v2 FileSerializer.validate); v2 enforces no size cap, so v3 doesn't either. FileUpload.title is globally unique=True, so a pre-save exists() check returns 400 on duplicate title (mirrors DRF UniqueValidator, avoids an IntegrityError→500). Download streams via dojo.utils.generate_file_response (correct content-type + Content-Disposition: attachment, mirrors v2 download_file); the FileResponse is returned through ninja unchanged with X-API-Status set manually. Multipart parsed with ninja Form(...)/File(...). Roundtrip tested (bytes + disposition). |
| 2026-07-19 | OS5 — parent-inherited authorization + permission values (mirror the v2 related-object permission classes exactly): the parent is resolved through its get_authorized_* view queryset → 404 unknown-or-unauthorized (never leak existence, §4.10); the applicable permission is then checked on the parent via user_has_permission → 403. Permission values mirror v2: notes GET/POST = view (UserHas*NotePermission post_permission is view); tags GET = view, PUT/POST/DELETE = edit (UserHas*RelatedObjectPermission); files GET/download = Product_Tracking_Files_View, POST = Product_Tracking_Files_Add (UserHas*FilePermission). 403-write not naturally expressible for members under OS legacy (view==edit==add for authorized_users members; only Delete is staff-gated, and no sub-resource write maps to Delete — same finding as OS3b §12): the 403 code path is therefore tested by making the permission check fail (mock user_has_permission→False while the parent stays visible); the 404 path is tested with a real non-member. Parent-view resolvers: finding/product take user=request.user, engagement/test use crum's current user (matching their get_authorized_* signatures and the OS3 routes). |
| 2026-07-19 | OS5 — kernel purity + wiring + unique operation ids: subresources.py imports no parent-resource model (finding/product/…); the parent model/queryset/label/permissions all arrive as factory arguments. It imports only its own storage models (Notes/NoteHistory/FileUpload), the generate_file_response helper, and user_has_permission (authorization infra, not a resource/service module — same class as the kernel's existing rest_framework/django imports; I8). Wiring lives in dojo/api_v3/api.py::_mount_subresources (the composition root, alongside the existing deferred router-factory imports — consistent with the §12 OS2 "only build_api()/import_routes.py import resources under dojo/api_v3/*" rule). Inner view functions are registered manually (router.get(path,...)(fn)) after setting a unique fn.__name__ per resource, so the once-per-resource factory calls don't collide into duplicate OpenAPI operationIds. Query sweep extended with all 13 new GET endpoints (notes/files/tags fan-out + a real file per parent for download); capture_request made streaming-aware (the download FileResponse has no .content). |
| 2026-07-19 | OS6 — expand/include/sub-resource RBAC sweep (ports test_apiv2_prefetch_rbac.py intent): unittests/api_v3/test_apiv3_expand_rbac.py (15 tests, all green). Builds a two-product world, authorizes a non-superuser on product A only (legacy authorized_users M2M), and proves no v3 projection discloses product B: the list/?expand=test.engagement,product,product_type rows are all product A; a product-B finding detail (even with ?expand=) is 404; ?product=<B> intersects the authorized queryset → empty; ?expand=reporter never surfaces product B's distinctive reporter (the v2 4a user-enumeration analogue — note expand swaps the ref for UserSlim, keyed by username); ?include=counts totals reflect only product A (B's severities stay 0, admin's total strictly exceeds the member's); and the /findings/{id}/{notes,locations} sub-resources + expand=locations return 404 for an unauthorized parent (parent-inherited authorization). Structural reason it holds: every projection is computed over get_authorized_findings(...), so nothing reachable from an authorized finding can belong to another product (I8). |
| 2026-07-19 | OS6 — latency + query-count benchmark (honest, in-process; deferred from OS1 gate). Harness unittests/api_v3/test_apiv3_benchmark.py, CI-excluded via @skipUnless(DD_API_V3_BENCH=1); run docker compose exec -e DD_API_V3_BENCH=1 uwsgi python manage.py test unittests.api_v3.test_apiv3_benchmark. Seeded 1000 findings on one test, limit=100, N=30 timed in-process GETs. Results (real capture, 1021 findings): v2 ?prefetch=test = 636 queries, median 521 ms / p95 721 ms; v2 no-prefetch = 229 q, 193/424 ms; v3 slim = 5 q, 38/45 ms; v3 ?expand=test.engagement = 7 q, 60/231 ms. CAVEATS (binding on the latency numbers): in-process only — no HTTP/uwsgi/network/gzip layer, single-threaded, no pool warmup, shared test transaction; treat latency as directional and the query counts (5/7 vs 636, constant vs row count) as the load-bearing, environment-independent evidence. The wall-clock-against-uwsgi procedure in .claude/os1-gate-report.md remains the way to quote production latency. Full report: .claude/os6-benchmark.md. |
| 2026-07-19 | OS6 — captured examples + docs page. api_v3_examples.md (repo root) is generated by unittests/api_v3/test_apiv3_examples.py (CI-excluded, DD_API_V3_EXAMPLES=1) making real in-process requests; bodies verbatim, tokens redacted, lists truncated to ~3 rows. Covers findings (detail slim; expand=test.engagement,locations; filtered list + page-2 envelope showing non-null next/previous; include=counts meta; notes POST+GET; locations edge rows; PATCH; POST /import) and products (detail/list/POST/PATCH). Docs-site page: docs/content/automation/api/api-v3-alpha-docs.md (Doks/thulite theme, weight: 3, next to the v2 api-v2-docs.md — not the asset_modelling pages). Written in plain markdown only (blockquote instead of an {{% alert %}} shortcode) because the theme is a not-locally-installed Hugo module and an unknown shortcode would break the build; covers overview, alpha disclaimer (URL→/api/v3/ at beta), token+session auth, envelope/refs/expand/fields/include/filter summary, the v2→v3 mapping table for the seven objects + import, one-way notes/tags/files, Known alpha gaps, and the /api/v3-alpha/docs link. |
| 2026-07-19 | OS6 — Docs UI (Scalar) evaluation — coordinator decision: DEFER; alpha keeps ninja's built-in Swagger at /api/v3-alpha/docs. Rationale: swapping to Scalar means vendoring an unreviewed third-party JS bundle into a security-product repo, which needs its own supply-chain review — the same class of decision as adding django-ninja (D2), not "polish". What the swap would take when approved: one template view (Hugo shortcode or a small Django TemplateView) rendering the Scalar <script> initialiser pointed at /api/v3-alpha/openapi.json, plus the Scalar JS vendored locally (mirroring how drf-spectacular-sidecar bundles Swagger/Redoc assets for v2) so no CDN is hit and the artifact CSP stays clean. Try-it already works against session auth on the built-in Swagger. No asset was downloaded (per §10 no-downloads). |
| 2026-07-19 | OS6 — §5 invariant checklist I1–I10: ALL PASS (verdicts in .claude/os6-report.md). Verified against the OS6 deliverables + code: I1 envelope closed (pagination.py builds exactly {count,next,previous,results,meta?}, asserted by tests + examples); I2 filter snapshot in CI (snapshots/filters.json, contract test green); I3 ref shape closed (refs.py, {id,name} + type for locations only); I4 named subclassable schemas (OS1 gate crit 7); I5 all resources are build_*_router() factories called by the mount; I6 finding/import writes go through dojo/finding/services.py / dojo/importers/services.py; I7 auth=[TokenAuth(), django_auth]; I8 RBAC only via get_authorized_* — proven for expand/include/sub-resources by OS6's new test (the two is_superuser gates that exist — /locations and user-write validation — are documented faithful mirrors of the v2 IsSuperUser permission class and UserSerializer.validate(), §12 OS4/OS3a, not ad-hoc drift, and object rows still come from authorized querysets); I9 closed problem+json error contract with per-kind type URIs; I10 no UI-flavored formatting (raw username, ISO-8601 Z). |
| 2026-07-19 | OS6 — release-readiness sweep (green). Full unittests.api_v3 = 267 executed + 2 CI-excluded (skipped), 0 failures (/tmp/apiv3_os6_full.log); the 2 skipped are the benchmark + examples harnesses (env-gated so CI never pays). manage.py check clean (4 silenced). v2 regression sample untouched and green: test_rest_framework 879 run / 377 skipped OK, test_apiv2_prefetch_rbac 10 OK. Ruff clean on all new files. Additive-only confirmed: git diff --name-only HEAD = no tracked file modified (all OS6 deliverables are new files; API_V3_PLAN.md §12 is the only tracked edit). This is the final OS phase — the alpha is feature-complete per §2. |
| 2026-07-19 | ALPHA GAP CLOSURE — note-create side-effects now fire in the alpha (architect directive rejecting the OS5 deferral). OS5 (§12) had shipped notes create as pure persist+link, leaving v2's note-create side-effects for the post-alpha convergence track. The architect rejected that deferral: v3 note creation must fire the same side-effects as v2 in the alpha. Verified v2 side-effect matrix (read, not modified): finding notes @action (dojo/finding/api/views.py:445-462) — (a) last_reviewed/last_reviewed_by stamping via finding.save(update_fields=["last_reviewed","last_reviewed_by","updated"]), (b) @mention notifications via process_tag_notifications(...) (dojo/notifications/helper.py:891) with parent_url=view_finding, (c) JIRA comment sync if finding.has_jira_issue: jira_services.add_comment(finding, note) elif finding.has_jira_group_issue: jira_services.add_comment(finding.finding_group, note); engagement notes @action (dojo/engagement/api/views.py:185-194) — process_tag_notifications only (view_engagement); NO last_reviewed, NO JIRA; test notes @action (dojo/test/api/views.py:162-171) — process_tag_notifications only (view_test); NO last_reviewed, NO JIRA. (product notes/files are not a v2 or v3 surface — OS5 matrix; product_type/user have no notes.) Implementation (keeps I5/I6): the kernel notes factory build_notes_router gained an optional on_note_created(parent, note, *, user) callback invoked after persist+link — the kernel imports zero JIRA/notification machinery. New resource services carry the side-effects: dojo/finding/services.py::process_note_added (all three side-effects), dojo/engagement/services.py::process_note_added and dojo/test/services.py::process_note_added (mentions only). Wired in the composition root dojo/api_v3/api.py::_mount_subresources. Conservative choice (§10.2): process_tag_notifications is the exact v2 mention parser/dispatcher and needs the request to build the absolute mention URL; the services read it from crum (get_current_request(), set for the whole request by CurrentRequestUserMiddleware — the same context bridge delete_finding already relies on), and if no request is present (a pure non-HTTP call, per I6) mentions are skipped while the other side-effects still fire. last_reviewed is a Finding-only field (Engagement/Test lack it), so a mis-wired finding callback on engagement/test would 500 — the passing 201 is itself parity evidence. Tests: 8 added to unittests/api_v3/test_apiv3_subresources.py (TestApiV3NoteSideEffectsFinding ×5: last_reviewed stamping, JIRA comment when linked / not-called when unlinked / finding-group branch, @mention→user_mentioned notification; TestApiV3NoteSideEffectsEngagementTest ×3: engagement+test mention parity, no-JIRA guard). Full suite 275 executed + 2 skipped = 277, 0 failures (/tmp/apiv3_notes_full.log); ruff clean. No v2 code/UI/test modified. |
| 2026-07-19 | D11 EXECUTED — v3 wire surface renamed product_type→organization, product→asset (all-or-nothing). Renamed every wire-visible occurrence while leaving Django models (Product/Product_Type), DB tables and dojo/product*/ module paths untouched (the DTO layer decouples wire from model). Touched ~30 files: 14 source modules (dojo/{product,product_type,finding,engagement,test,location}/api_v3/{schemas,routes}.py + dojo/api_v3/{api.py,import_routes.py}); 12 test modules (2 renamed via create+rm: test_apiv3_products.py→test_apiv3_assets.py, test_apiv3_product_types.py→test_apiv3_organizations.py; 10 updated: findings/engagements/tests/import/locations/subresources/expand_rbac/query_report/examples/openapi); regenerated snapshots/filters.json and api_v3_examples.md; docs page api-v3-alpha-docs.md. Wire changes: paths /product_types→/organizations, /products→/assets (+/assets/{id}/locations, /assets/{id}/tags); url_names + OpenAPI operationIds; schema classes Product*→Asset*, ProductType*→Organization* (+ list/edge-response schemas + finding-module re-exports); ref keys finding.asset/finding.organization/engagement.asset/engagement.organization/test.asset/test.organization/asset.organization; ?expand= keys (expand=asset/organization); filter params (asset/asset__in/organization) + registry names (register_filter_spec("asset"/"organization"/...)); write FKs (AssetWrite.organization→model prod_type; EngagementWrite.asset→model product); import form fields product_name→asset_name, product_type_name→organization_name (mapped internally to the AutoCreateContextManager/dojo/importers/services.py product_name/product_type_name keys, which are NOT wire and stay); router factories build_assets_router/build_organizations_router/build_asset_locations_router; tags ["assets"]/["organizations"]; 404/validation messages ("Asset N not found", "Organization N not found"). Django-internal names kept (not wire): ORM field paths inside filter_field(...)/select_related/ExpandRel (e.g. prod_type, test__engagement__product, products__product), get_authorized_products/get_authorized_product_types, Permissions.Product_*, LocationProductReference, refs.py _LABELERS keys (keyed on type(obj).__name__), and the separate model Product_API_Scan_Configuration (not in D11 scope). Conservative residuals (§10.2), documented, deliberately NOT renamed: the scalar organization flags critical_product/key_product (model columns the UI relabel itself retains, e.g. org.critical_product_label; §4.5 lists them verbatim; not entity references). The one clean-renameable role ref was renamed: product_manager→asset_manager on the wire (relabel term "Asset Manager"), mapping to model product_manager. include.py verified: severity counts carry no product-named keys. Unconditional: the API naming does NOT follow DD_ENABLE_V3_ORGANIZATION_ASSET_RELABEL (documented in the docs page's new "Naming: organizations and assets" section). Verification: OpenAPI render shows zero product tokens in paths/tags/component-schema-names; the only component property residual is critical_product/key_product on Organization* (above). api_v3_examples.md has zero product wire keys (only unavoidable fixture data value "description": "test product" and the enum word "production" remain as substrings). Filter snapshot diff = exactly the rename (34 ins / 34 del; model names preserved). Full unittests.api_v3 = 276 executed + 2 skipped, 0 failures (/tmp/apiv3_d11_full2.log) — same count as the pre-rename baseline (the D11 OpenAPI token-absence guard was folded into the existing test_apiv3_openapi methods, adding no test). Ruff clean. No v2 code/UI/v2-test modified. |
| 2026-07-19 | AUTHZ GUARDS — two deny-by-default RBAC tripwires added enforcing I8 structurally (tests only; no production/route/v2 change). (1) Behavioural whole-surface sweep (unittests/api_v3/test_apiv3_authz_sweep.py): walks every operation in api_v3.get_openapi_schema() (all methods × paths = 66 operations) with a completeness gate identical in spirit to test_apiv3_query_report.py — a schema operation without a registered probe fails the test ("add a representative request + expected outcome"), so the sweep extends to future endpoints by construction. Two probes each: anonymous → 401 (ninja auth runs in _run_checks before body parsing, so no payload needed) and a zero-permission user (freshly created, no memberships/roles/config perms) → never receives data: GET lists → 200 count==0/results==[] (findings also checked ?include=counts totals all 0); GET detail / sub-resource GET → 404; writes → 403/404 with a per-operation minimal-valid-payload registry (JSON for most; multipart for /import with a tiny scan file + an existing-but-unauthorized engagement, and for file POSTs) so every write reaches the authz gate — a 400/422 on a write probe is treated as a sweep FAILURE (0 such probes). (2) Structural source-scan tripwire (unittests/api_v3/test_apiv3_authz_static.py): over dojo/*/api_v3/routes.py + dojo/api_v3/{import_routes,subresources}.py — Rule A bans any .objects. line unless its (file, Model.objects.method) pair is in an audited OBJECTS_ALLOWLIST (stale entries also fail); Rule B parses each file's AST and requires every route operation (decorated @router.<verb> OR manually registered via router.<verb>(...)(fn)) to reference an authorization primitive (get_authorized_ / user_has_ / _require / is_superuser) in its own body or one same-module helper deep. (3) Filter-contract rationale expanded in test_apiv3_filter_contract.py docstring + drift message (why the vocabulary is a locked contract: silent-typo failure mode, one-vocabulary-many-projections D6, and the DD_API_V3_UPDATE_SNAPSHOTS=1 reviewable-diff workflow) — no test behaviour changed. HEADLINE FINDING: NO authorization gaps found — every one of the 66 operations denies by default; no production code was changed. .objects. allowlist (5 patterns / 3 files, all justified): LocationFindingReference.objects.filter + LocationProductReference.objects.filter (edges read from an already-get_authorized_*-resolved parent), NoteHistory.objects.create + FileUpload.objects.filter (owned-subobject write / title-uniqueness check after the parent authz+permission gate), Dojo_User.objects.filter (the RBAC scoping itself — self-only fallback for users lacking auth.view_user, plus the post-add_user-gate username-uniqueness check). Two documented non-leak deviations the sweep surfaces honestly: /users list+detail self-visibility (a zero-perm user sees exactly their own record, never another's — §12 OS3a), and /locations superuser-gate 403 (§12 OS4). Full unittests.api_v3 = 284 executed + 2 skipped, 0 failures (/tmp/apiv3_authz_full.log; +8 tests over the 276 baseline). Ruff clean. No v2 code/UI/v2-test modified. |
| 2026-07-20 | ?fields= LIST opt-up + defer optimization (§4.7 updated; architect-approved contract enhancement). Two parts, applied uniformly to every list route (findings/assets/organizations/engagements/tests/users/locations). Part A — a list's ?fields= allowlist widened from Slim fields ∪ EXPANDABLE to Detail fields ∪ EXPANDABLE; the default (no ?fields=) list stays the Slim shape byte-for-byte, and requesting any beyond-Slim field opts the row up into the Detail projection (verified byte-identical to apply_fields(serialize(obj, Detail), requested)). Unknown names still 400 (dedicated fields type, I9); no existing 200 becomes a 400 (allowlist only widened). Part B — LIST querysets defer() the detail-only concrete own-model columns (model._meta.concrete_fields, not is_relation) that were not requested; defer_columns = detail-only-concrete − requested. Default lists stop fetching the heavy TEXT columns entirely; ?fields=impact un-defers exactly impact. Detail-only relation refs add a fixed select_related join (declared per-schema via DETAIL_SELECT_RELATED so D11 wire↔model naming and the location reverse-O2O resolve), applied only when requested — never per-row. Detail GET /{id} routes are not deferred (single-object fetch). defer() not only(): only() requires enumerating every ref column across all select_related paths and silently drops a join when one is missed; defer() composes safely with the existing expand-driven select_related/prefetch/annotations, and any accidental deferred-column access shows up as per-row queries that the assertNumQueries tests + the whole-surface N+1 sweep catch. Kernel: resource-agnostic plan_list_fields() / serialize_list_row() in dojo/api_v3/expand.py; serialize_list_row serializes the Slim base (only ever reads loaded columns) then splices the requested detail fields (concrete via getattr, refs via the detail resolver), so no deferred column is read during serialization. Defer column sets (computed at runtime): finding {description, file_path, impact, line, mitigated, mitigation, references, severity_justification, steps_to_reproduce}; engagement {api_test, branch_tag, build_id, check_list, commit_hash, deduplication_on_engagement, description, first_contacted, pen_test, reason, source_code_management_uri, test_strategy, threat_model, tracker, version}; test {branch_tag, build_id, commit_hash, description, scan_type, version}; asset {business_criticality, external_audience, internet_accessible, origin, platform}; user {date_joined, is_staff}; organization {} (Detail == Slim); location {} (URL-subtype fields live on the url reverse-O2O, not own-model columns — non-deferrable). No un-defer exceptions were needed — the full suite (query-count, N+1 sweep, authz sweep) stayed green with no per-row regression, so no Slim resolver/model property touches a deferred column. Out of scope (stated per task): the location-edge sub-resource lists (/findings/{id}/locations, /assets/{id}/locations) serialize fixed edge dicts (no slim/detail schema) — no opt-up/defer applied. Tests: findings ?fields=id,title,impact,references opt-up + absence-from-default + value-parity + mitigated_by ref + slim/detail/expandable mix + defer-proof SQL assertions; constant-query assertions for default-with-defer and beyond-Slim (incl. mitigated_by fixed join); engagement description opt-up (uniformity). Full unittests.api_v3 = 296 executed + 2 skipped, 0 failures (/tmp/apiv3_fields_full.log; +12 over the 284 baseline). Ruff clean. Default list response bodies byte-identical to before (no contract change without ?fields=). No v2 code/UI/v2-test modified. |
| 2026-07-20 | IMPORT MADE SYNCHRONOUS-ONLY — architect directive removed the background import parameter and the reserved job resource from v3. The reserved deferred-execution import path was removed from the v3 contract, code, and docs. Code: dojo/api_v3/import_routes.py dropped the background: bool = False field from ImportForm, the 400 "not available" reject branch in import_endpoint, and the module-docstring reservation line (ProblemDetail import retained — still used by the except ProblemDetail: raise passthrough). Plan: §4.13's reserved-processing bullet replaced with an explicit "v3 imports are synchronous; no job resource / no deferred-execution path" statement; §6 backlog checkbox for the reserved job resource + flag removed; §9 checklist row 13 disposition reworded to "intentionally removed from v3 scope by architect directive — v3 imports are synchronous; no job resource is reserved" and its capability cell reworded to drop the reserved-flag phrasing. Docs: api-v3-alpha-docs.md known-gaps bullet dropped the reserved-job wording; the cursor-pagination reservation there is kept (separate task is implementing cursor). Tests: none changed — the v3 suite never asserted the reject branch (grep-verified); the authz-sweep import probe and the examples harness carry no reference to the removed field; api_v3_examples.md unchanged (its curated import body never listed the field; the removal changes the OpenAPI schema, not the captured example). OpenAPI guard (test_apiv3_openapi) still green (it asserts only path/tag presence, never the form field). The append-only 2026-07-19 anchoring entry that first reserved the job resource is left intact as historical record. Full unittests.api_v3 = 296 executed + 2 skipped, 0 failures (/tmp/apiv3_t5_full.log; unchanged from the 296 baseline — no test removed). Ruff clean. No v2 code/UI/v2-test modified. |
| 2026-07-20 | PUT (full replace) SHIPPED — reverses the OS3a/OS3b "no PUT in alpha" decision (architect directive). OS3a (§12, 2026-07-19) shipped PATCH-only and deferred PUT to the post-alpha backlog bundle as "an additive, non-breaking addition available later"; this pulls it forward for the six resources that have PATCH today — findings, assets, organizations, engagements, tests, users. Locations stay read-only (no PUT); the rest of the deferred bundle (delete-time push_to_jira, configuration_permissions on user writes, v3 self-profile, location-edge filter vocab) stays deferred. Semantics (§4.11): PUT /{resource}/{id} validates against a create-shaped schema (required fields enforced, extra="forbid") and applies payload.dict() without exclude_unset, so an omitted optional resets to its schema default — a true full replace mirroring v2's DRF update(partial=False). Response is 200 with the detail shape, identical to PATCH's response path; permission ladder is identical to PATCH (authorized-queryset resolve → 404, then object edit permission → 403); server-managed fields stay non-writable; finding PUT goes through dojo/finding/services.py::update_finding with a full changes dict exactly like PATCH (no business logic in routes, I6); user PUT keeps the UserSerializer-ported guards (superuser/staff gating, password never settable via update — a supplied password is a 400, mirroring the PATCH route). Per-resource judgment calls (schema used + why): (a) reuse the existing Write schema for organization (OrganizationWrite) and user (UserWrite) — both already declare their non-null/defaulted columns with the correct model defaults (critical_product/key_product→False; first_name/last_name→"", is_active→True, is_staff/is_superuser→False) and have no immutable parent, so full-replace resets land on valid values. (b) dedicated *Replace schema for finding (FindingReplace), test (TestReplace), asset (AssetReplace), engagement (EngagementReplace) — the Write schema genuinely can't serve, for one or both of these reasons: (1) immutable required parent — FindingWrite requires test and TestWrite requires engagement, but both are editable=False and not reassignable on update (PATCH omits them, mirroring v2's read-only-on-update serializers); so the Replace schema drops the parent and PUT, like PATCH, never reassigns it. (2) non-null / non-blank-with-default columns whose create-appropriate None default would 500 on replace — the create schemas declare these `X |
| 2026-07-20 | CURSOR PAGINATION SHIPPED — reverses the D4/§4.3 "reserved only" alpha scope (architect directive). The reserved ?pagination=cursor param is now implemented as forward-only keyset pagination (GitLab precedent), leaving the offset default byte-for-byte unchanged. Kernel-central (I5/I6): dojo/api_v3/pagination.py owns the whole mode branch inside paginate(); paginate() gained one optional kwarg `filter_spec: FilterSpec |
| 2026-07-20 | CSV EXPORT SHIPPED — the second projection of the D6 filter contract (§4.15 added; leaves XLSX as the only remaining export TODO). GET /<resource>/export.csv on the seven top-level list resources (findings/assets/organizations/engagements/tests/users/locations — locations keeps its superuser gate). New kernel dojo/api_v3/csv_export.py (resource-agnostic; no resource imports): a generic column flattener + streaming-response builder + guard logic. Each of the seven router factories gained one thin export.csv route (authorize → apply_filters → plan_list_fields → build page_qs exactly like its list route, minus pagination/expand, then delegate to the kernel — I5/I6). Contract: identical filter contract as the list (filters, o= full offset ordering vocabulary, q=, ?fields= incl. the Detail opt-up); no pagination — the export is the whole filtered, authorized set; expand/include/limit/offset/pagination/cursor are not applicable → 400 problem+json under a new type=.../errors/export URI (I9: new type URI, closed shape — raised via the existing ProblemDetail, so errors.py was not modified). Cap: API_V3_EXPORT_MAX_ROWS (env DD_API_V3_EXPORT_MAX_ROWS, default 100 000; settings.dist.py — the only shared-file edit) enforced via the existing capped-count helper (compute_count(count_qs, cap) returns count_exact=False iff the count exceeded the cap → 400 "narrow the filter", never silent truncation). Response: StreamingHttpResponse, text/csv; charset=utf-8, Content-Disposition: attachment; filename="<resource>-export.csv", X-API-Status: alpha; streamed by a csv.writer over queryset.iterator(chunk_size=2000) (chunked prefetch), memory-bounded and query-count-independent of row count (pinned by assertNumQueries: equal at 10 vs 100 findings). Column-flattening spec (one generic kernel, derived from the serialization schema — never a per-resource column list): columns = the serialized row's keys in schema order restricted to the ?fields= projection; a Ref ({id,name}) → two columns <key>_id/<key>_name; a LocationRef ({id,name,type}) → three columns <key>_id/<key>_name/<key>_type (none of the seven slim schemas carry a LocationRef today, but the kernel handles it generically); a list field (tags) → one semicolon-joined column; datetimes ISO-8601 Z, dates YYYY-MM-DD, booleans lowercase true/false, null → empty cell; a zero-row export still emits the full header row. Row values are reused from serialize_list_row (same shaping as the list — the CSV never diverges from the JSON) then flattened. CSV-injection hardening (security product): any cell whose text starts with =/+/-/@/TAB is prefixed with a single quote (standard spreadsheet formula-injection defense); tested. Deviation, documented: the /users export mirrors the list's self-visibility scope (a zero-perm user exports exactly their own record, never another's — §12 OS3a), so it is not header-only-empty; /locations export inherits the superuser gate (403 for non-superusers, before streaming — §12 OS4). Completeness gates extended (both green): the authz sweep gained 7 export.csv GET probes (anonymous 401; zero-perm 200 header-only CSV for the 5 legacy-authz resources, 200 self-record CSV for users, 403 for locations) with a new CSV-parsing assertion branch — operations 72 → 79; the query-report sweep gained 7 representative requests (streaming already handled by capture_request, which captures the synchronous auth+capped-count without consuming the streamed body — no N+1 flag; per-row independence pinned by the dedicated assertNumQueries test). New unittests/api_v3/test_apiv3_csv_export.py (27 tests): findings happy path (flattened header + all rows, ref columns, tags join, ISO-Z), filters/o=/q=, ?fields= narrowing + detail opt-up (impact present when requested, absent by default), over-cap 400 (lowered setting), reserved-param 400s (expand/include/limit/offset/pagination/cursor), injection prefix (kernel unit + endpoint), RBAC (zero-perm header-only + member sees only its product's rows), constant-query, assets uniformity, and the Content-Type/Content-Disposition/X-API-Status headers + no-collision-with-{int:id} check. Docs page: new "CSV export" section + Known-gaps reworded to "no XLSX export (CSV is available)"; §4.15 added; §8 backlog checkbox reworded to XLSX-only; §9 checklist row 15 disposition = "CSV shipped, XLSX deferred". Full unittests.api_v3 = 382 executed + 2 skipped, 0 failures (/tmp/apiv3_t4_full.log; +27 over the 355 baseline). Ruff clean. No v2 code/UI/v2-test modified (settings.dist.py is the only shared-file edit, as allowed). |
| 2026-07-21 | V2 IMPORT/REIMPORT TEST CORPUS PORTED TO v3 (§10 backlog #1; dual-endpoint adapter, no copy). The v2 import corpus now proves the same DB behaviour against the consolidated POST /api/v3-alpha/import. Adapter (tests only): unittests/api_v3/import_corpus_shim.py::ApiV3ImportShim overrides ONLY the two v2 helper methods import_scan_with_params / reimport_scan_with_params — mapping v2 wire→v3 wire (product_name→asset_name, product_type_name→organization_name, import/reimport→one POST with `mode=import |
| 2026-07-21 | JIRA PUSH FLOWS PORTED TO v3 (§10 backlog #2). v3 wiring (sanctioned, all in dojo/api_v3/import_routes.py — v3 code): ImportForm.push_to_jira: bool = False added and routed through the common dict to the dojo/importers/services.py facade (which already accepted push_to_jira; §4.13 shared CommonImportScan). The route now OR-s payload.push_to_jira with the resolved JIRA project's push_all_issues when JIRA is enabled — mirroring the v2 import/reimport viewsets (dojo/api_v2/views.py:391-404/517-535) and the v3 finding route's identical OR (D7 canonical). The OR uses the resolved driver per mode (engagement / test / test or engagement or product for auto); _check_auto_permission now returns that driver. Port strategy: the v2 corpus (test_jira_import_and_pushing_api.py) is a VCR/cassette suite entangled with v2-only surfaces (finding groups, epics, webhooks, UI /finding/bulk) — not economical to shim, so targeted v3 ports in unittests/api_v3/test_apiv3_jira_push.py (10 tests) mock JIRA at dojo.jira.services.push (import path — what default_importer + finding.helper.post_process_findings_batch both call) and dojo.finding.services.jira_services.push (write path, as test_apiv3_finding_writes does); block_execution=True forces the post-processing sync so the push is observable. Coverage map (corpus → v3): test_import_with_push_to_jira→test_import_with_push_to_jira_pushes_each_finding (set-equality: every finding in the new test is pushed); test_import_no_push_to_jira/..._is_false→test_import_without_push_to_jira_does_not_push/..._false_does_not_push; test_import_no_push_to_jira_but_push_all→test_import_push_all_issues_forces_push; test_import_no_push_to_jira_reimport_with_push_to_jira→test_reimport_with_push_to_jira_pushes; test_import_no_push_to_jira_reimport_no_push_to_jira→test_reimport_without_push_to_jira_does_not_push; PATCH/PUT push subtests of test_create_edit_update_finding→ existing test_apiv3_finding_writes (create push, keep-in-sync force_sync, push-failure-400) + new TestApiV3FindingWriteJiraProjectSetting (the push_all_issues OR-on-write gap: PATCH/PUT force push, negative when not push_all). Import push-failure divergence (documented): test_import_jira_push_failure_does_not_fail_import — a failed import-time push is logged, not raised, so the import still returns 200 (mirrors v2; the importer never propagated JIRA errors). This differs from finding PATCH/PUT, which surface a push failure as a 400 (test_apiv3_finding_writes.test_update_jira_push_failure_is_400). Skipped with reasons (out of v3 alpha scope): all group_by/finding-group scenarios (~half the corpus; no group_by on the v3 form, groups are §2 non-goal), test_engagement_epic_* (epics), *bulk_edit* (UI /finding/bulk, DS3), test_import_with_push_to_jira_add_comment/add_tags (note/tag→JIRA sync — the note-create JIRA add_comment side-effect is already covered by test_apiv3_subresources note side-effects), the enforce_verified_status[_jira] matrix (JIRA-layer verified-status gating, independent of the v3 wire surface, covered by v2), the keep_sync reimport status/priority-transition asserts and risk-acceptance reopen (require real JIRA issue status via cassettes — not reproducible with a mocked push), and *_auto_create_context_fetches_all_objects_for_push_to_jira (asserts v2 import_settings["push_to_jira"], a v2 Test_Import audit-record shape, not a v3 response field). Full unittests.api_v3 re-run green (see the T8b row). Ruff clean. No v2/UI/v2-test code modified. |
| 2026-07-21 | BACKLOG PRIORITY 3 — v2 API test-corpus evaluation delivered as the §9.1 disposition table (no code, per the task). Every remaining test_apiv2_* + the named API-adjacent suites + an APIClient/rest_framework/-list scan were inventoried and classified PORTED / PORT-LATER / SKIP / SUPERSEDED, with test_rest_framework.py split by area. Two open PORT-LATER checkboxes remain (both new §6 checkboxes): (a) TestDetail matching-policy read fields (ports test_apiv2_test_dedupe_policy.py), (b) authorized_users member-management on asset writes gated by Product_Manage_Members (ports test_product_authorized_users_api_authz.py); configuration_permissions-on-user and the v3 self-profile endpoint (which also close ConfigurationPermissionTest/UserProfileTest) were already tracked in the pre-existing backlog bundle. Notable SUPERSEDED mappings: prefetch-RBAC→test_apiv3_expand_rbac, methods-and-endpoints smoke→OpenAPI guard + authz/query sweeps, per-endpoint SchemaChecker→OpenAPI guard, FK-reassign authz→v3 write RBAC re-check. No trivial ports were implemented — every PORT-LATER item needs new schema/route surface beyond the <30-min bar (§10.2 conservative). |
| 2026-07-21 | FINDINGSLIM EXTENDED WITH vulnerability_ids + cwes (architect directive — identity-critical fields available on lists without per-row detail fetches). Two resolver-backed READ fields added to FindingSlim (so FindingDetail inherits them): vulnerability_ids: list[str] — flat strings (["CVE-2020-1234", ...]), NOT v2's object list; read directly from the vulnerability_id_set reverse relation in storage order (first = the id mirrored into cve), symmetric with what FindingWrite already accepts. cwes: list[int] — the numeric part of each finding_cwe_set row parsed via dojo.finding.cwe.cwe_number. Shape choice list[int] (not the stored strings): Finding_CWE.cwe is a varchar(11) that save_cwes always writes as the canonical CWE-<n> label (via finding_cwe_labels/cwe_label), so the stored format is reliably CWE-<int>; parsing to int is consistent with the slim's scalar `cwe: int |
| 2026-07-21 | TESTDETAIL MATCHING-POLICY READ FIELDS PORTED (§9.1 PORT-LATER→PORTED; ports test_apiv2_test_dedupe_policy.py). TestDetail gains two READ-ONLY computed fields exposing the effective finding-matching policy: deduplication_algorithm: str (always non-null; legacy fallback) and hash_code_fields: list[str] | None (None when the scan type has no per-scanner config). Helper reused verbatim (never duplicated): both resolvers return obj.deduplication_algorithm / obj.hash_code_fields — the v2 Test model properties (dojo/test/models.py:156-193), the exact settings-driven per-scanner lookup the v2 TestSerializer reads (DEDUPLICATION_ALGORITHM_PER_PARSER / HASHCODE_FIELDS_PER_SCANNER, keyed test_type.name then scan_type, DEDUPE_ALGO_LEGACY default). Slim unchanged (§4.5): these are config lookups, not identity fields, so TestSlim stays as-is. Reject-vs-ignore deviation (the deliberate hardening): the fields are not on TestWrite/TestUpdate/TestReplace (all extra="forbid"), so a PATCH or PUT supplying either → 400 unknown-field problem+json; v2's serializer declares them read_only and therefore silently ignores writes to them (200, no change). v3 rejects because silent-ignore is the exact failure mode v3 rejects everywhere (same principle as unknown filter/expand/fields= params — D6/I9); the reject is atomic (a mixed PATCH bundling the policy fields with a real description change alters nothing). Query impact = zero on detail: the resolvers read only test_type (already select_related in TestSlim.SELECT_RELATED) and scan_type (a loaded concrete column — detail routes never defer()), verified with assertNumQueries(0). ?fields= opt-up + defer handling (kernel accommodation, §4.7): on a LIST these detail-only fields opt up via ?fields=; being resolver-backed (not model._meta.concrete_fields) they never enter the defer set (asserted). But their resolvers read the concrete scan_type column, which is deferred by default on tests lists (§12 2026-07-20) — a naive opt-up would lazy-load it per row (N+1). Smallest correct accommodation: a new optional per-schema DETAIL_FIELD_COLUMNS: dict[computed_field, tuple[concrete_column, ...]] honored by the resource-agnostic kernel plan_list_fields (dojo/api_v3/expand.py) — requesting a computed detail field un-defers exactly the columns its resolver reads (analogous to DETAIL_SELECT_RELATED, but for own columns rather than relation joins). TestDetail.DETAIL_FIELD_COLUMNS = {deduplication_algorithm: (scan_type,), hash_code_fields: (scan_type,)}. Kernel change is additive (a schema without DETAIL_FIELD_COLUMNS behaves identically) and covers the CSV route too (it reuses plan_list_fields). GET /tests?fields=id,name,deduplication_algorithm verified constant-query (row-independent). CSV: flows automatically — deduplication_algorithm → one column, hash_code_fields (list) → one semicolon-joined column (empty when null); asserted. Docs: the v3 docs page does not enumerate per-object detail fields — no docs edit needed. Tests: new unittests/api_v3/test_apiv3_test_dedupe_policy.py (16): two scan types with distinct policies (ZAP Scan→hash_code+[title,cwe,severity]; Checkmarx Scan detailed→unique_id_from_tool+None), default-fallback (legacy+None), detail zero-extra-query, PATCH×2/PUT×2 reject + atomic reject-vs-ignore, list not-on-default + opt-up value-parity + defer-set + constant-query, CSV flatten. Full unittests.api_v3 = 492 executed + 6 skipped, 0 failures (/tmp/apiv3_dedupe_full.log; +16 over the 476 baseline). No new endpoints → the authz/query-report/OpenAPI completeness gates are unaffected and stay green. Ruff clean. No v2 code/UI/v2-test modified. |
| 2026-07-21 | cwes LIST ACCEPTED ON FINDING WRITES (architect directive; §6 backlog bullet closed; symmetric with the read side + parallel to vulnerability_ids). Wire: cwes: list[int] | None = None added to FindingWrite, FindingUpdate, FindingReplace (plain CWE numbers, e.g. 79; matches FindingSlim.cwes). Read side unchanged. Precedence (mirrors v2 FindingSerializer.update ~:450-454 and the vulnerability_ids[0]→cve mirror): an explicit non-None scalar cwe wins as the primary; when only cwes is supplied its first entry is mirrored into cwe before the initial/save() so the primary persists (like cve). Persisted Finding_CWE rows = primary first, then the rest, via the existing save_cwes (finding_cwe_labels(finding.cwe, finding.unsaved_cwes) dedups with the primary first). When BOTH a scalar cwe and a non-None cwes are present in one request, save_cwes runs once through the cwes branch (not the scalar-only resync elif) — list drives the rows, scalar stays primary; no double-persist. unsaved_cwes shape discovered: save_cwes→finding_cwe_labels→cwe_label accepts plain ints OR CWE-<n>/"79" strings and canonicalizes+dedups, so the wire list[int] is placed onto finding.unsaved_cwes verbatim (no pre-formatting) — the full list, letting save_cwes order primary-first. Omission (no-reset-on-omit, extends the vulnerability_ids/reporter record to cwes): the service runs the list-driven mutation only when cwes is non-None; omitted (None) on PATCH and PUT ⇒ no list mutation, rows untouched except the pre-existing scalar-cwe-change resync (§12 2026-07-19 OS3b) — on PATCH that resync fires only if the scalar cwe changed; on PUT cwe is always in the full-replace set (as its reset-to-None default) so the scalar resync always fires and the rows follow the (possibly reset) scalar. Because PUT always carries cwe, a key-presence test would wrongly suppress the mirror, so the mirror keys off an explicit non-None scalar (cwe_explicit = "cwe" in changes and changes["cwe"] is not None; create uses the equivalent "cwe" not in scalars, since None is dropped from scalars). Empty-list decision (conservative, recorded): an explicit cwes: [] is a statement (unlike omission) → it clears the extra rows and resyncs to just the scalar-derived primary (unsaved_cwes=[], save_cwes ⇒ rows = {primary} when a scalar exists, else empty); on PUT [] and omission converge (both leave just the scalar primary, because the PUT scalar resync always runs). Read-back symmetry (rows persisted primary-first, resolve_cwes reads storage order): write cwes=[89,79] (no scalar) → read [89,79]; write scalar 79 + cwes=[89,100] → read [79,89,100]. Files: dojo/finding/api_v3/schemas.py (field + docstrings on the three write schemas), dojo/finding/api_v3/routes.py (pop cwes at create/PATCH/PUT, pass as service kwarg), dojo/finding/services.py (cwes kwarg on create_finding/update_finding, "cwes" added to _SPECIAL_KEYS, scalar mirror before save + rows resync). Docs page unchanged (it documents write conventions, not per-object write field lists — verified). Tests (+9) in unittests/api_v3/test_apiv3_finding_writes.py: create cwes-only (mirror+rows+read-back), create scalar+list (scalar wins), PATCH cwes replaces rows, PATCH scalar-only still resyncs (existing behavior), PATCH [] clears extras keeps primary, PATCH omit-both leaves rows untouched, PATCH scalar+list (scalar wins), PUT with cwes, PUT omitting cwes follows the scalar resync. Module 30→39. Full unittests.api_v3 = 504 executed + 6 skipped, 0 failures (/tmp/apiv3_cwes_full.log); the +12 over the 492 dedupe-policy baseline is this change's +9 plus concurrent-workstream tests (TEST dedupe-policy / Scalar docs) present in the tree — all green. Ruff clean. No v2 code/UI/v2-test modified. |
| 2026-07-21 | SCALAR MOVED FROM CDN+SRI TO NPM-AT-IMAGE-BUILD (architect directive — supersedes the CDN+SRI row above). @scalar/api-reference is now an exact-pinned dependency in components/package.json (1.63.0, integrity via components/yarn.lock), installed by the EXISTING yarn step in Dockerfile.nginx-alpine — zero Dockerfile changes because STATICFILES_DIRS already includes components/node_modules (settings.dist.py:467), so the bundle is served as a normal static file (dojo/api_v3/reference_docs.py → static("@scalar/api-reference/dist/browser/standalone.js")). Rationale: identical trust model to every other yarn-managed frontend dependency; asset fetched once at image build (lockfile hashes) instead of from every user's browser at runtime; air-gapped deployments get a working reference page; no usage metadata leaks to jsDelivr. SRI attribute dropped (same-origin, image-baked). Engine note: Scalar 1.63.0 declares node>=22; the image base ships node 22 (verified v22.23.0 on python:3.14.5-alpine3.22), so the image build passes engines — local lockfile regeneration on older node needs --ignore-engines. Tests updated: static-URL assertion, no-CDN/no-SRI assertions, exact-pin guard reading components/package.json. Docs page air-gap wording flipped to works-everywhere. |
{ "count": 4321, // exact below COUNT_CAP, else planner estimate (see below) "next": "https://.../api/v3/findings?limit=25&offset=50&severity=High", "previous": null, "results": [ ... ], "meta": { } // present only when ?include= adds content, plus // "count_exact": false when count was capped }