refactor(reorg): extract every type of class into modules [10/10] (DefectDojo#14987)

* refactor(product_type): extract Product_Type model into dojo/product_type/

Phase 1 of module reorg per AGENTS.md. Move Product_Type class + admin
registration into dojo/product_type/{models,admin}.py with backward-compat
re-export in dojo/models.py. No migration change (app_label unchanged).

* refactor(product_type): move forms + UI filter into dojo/product_type/ui/ [Phase 3,4]

Phase 3+4 of module reorg per AGENTS.md. Move Product_TypeForm,
Delete_Product_TypeForm, Add_Product_Type_AuthorizedUsersForm into ui/forms.py
(re-export from dojo/forms.py) and ProductTypeFilter into ui/filters.py. The filter
keeps its DojoFilter base; its only consumer is the product_type view, so no
dojo/filters.py re-export is kept (matches the url module) — avoids the
extracted-filter<->dojo.filters circular import.

* refactor(product_type): move views into dojo/product_type/ui/views.py [Phase 5]

Phase 5 of module reorg per AGENTS.md. Move dojo/product_type/views.py to
dojo/product_type/ui/views.py and update its two importers (dojo/organization/urls.py
and the counts unit test). product_type has no urls.py (routes live in
dojo/organization/urls.py), so only the views move.

* refactor(product_type): extract API layer into dojo/product_type/api/ [Phase 6,8,9]

Phase 6/8/9 of module reorg per AGENTS.md (Phase 7 N/A - no product_type API filter).
Move ProductTypeSerializer into api/serializer.py (re-exported from api_v2/serializers.py,
still used by ReportGenerateSerializer) and ProductTypeViewSet into api/views.py. Add
api/urls.py with add_product_type_urls() preserving route 'product_types' + basename
'product_type'; wire it into dojo/urls.py. Viewset re-export omitted (would cycle
api_v2.views<->product_type.api.views; only consumer was a test, now imports new path).

* docs(agents): fold Phase 1 reorg lessons into the playbook

Update AGENTS.md with conventions learned doing the Phase 1 model extractions:
string FK refs to break circular imports, ruff noqa conventions (PLC0415/E402/F401),
consolidated re-export placement, single-sourced constants, load-bearing side-effect
imports, and corrected docker-based verify commands (manage.py shell, run-unittest.sh).

* docs(agents): add Phase 2-9 lessons from the product_type full reorg

Conditional services phase; keep filter base class + drop re-export when sole
consumer is the module's own view (circular-import fix); per-symbol re-export
decisions by actual consumers (incl. multi-line imports); preserve DRF route+basename;
run real tests after dropping a re-export.

* docs(agents): add API serializer/viewset cycle-break + class-copy lessons from engagement reorg

* docs(agents): note prefetcher full-reexport + extend_schema_field cycle-break (Phase 6)

* docs(agents): add Phase 10 peripheral-module 10-PR stack plan

Self-contained brief for finishing the reorg: 5 new draft PRs (DefectDojo#6-10)
stacked on top of the finding PR, plus CWE+BurpRawRequestResponse folded
into the existing finding module PR. Bundles, line ranges, stack/cascade
mechanics, and module-specific gotchas. Marks the 5 core modules Complete.

* refactor(test): extract Test/Test_Type/Test_Import models into dojo/test/

Phase 1 of module reorg per AGENTS.md. Move Test, Test_Type, Test_Import,
Test_Import_Finding_Action + admin registrations into dojo/test/{models,admin}.py.
Cross-module FKs use string refs to avoid circular imports; IMPORT_* action
constants single-sourced in dojo/test/models.py with re-export in dojo/models.py.
No migration change.

* refactor(test): extract copy_test workflow into services.py [test Phase 2]

Move the test copy workflow (clone into engagement + product grade recalc +
notification) out of the copy_test view into an HTTP-free copy_test(test,
engagement, user) service, mirroring copy_engagement. View thinned to call it.
Add a unit test for the service (was previously untested). Notification URL uses
relative reverse() per codebase convention.

* refactor(test): move forms + UI filters into dojo/test/ui/ [test Phase 3,4]

Move TestForm/DeleteTestForm/CopyTestForm into ui/forms.py (TestForm re-exported
from dojo/forms.py since engagement views use it; the other two have no external
consumers). Move TestImportFilter/TestImportFindingActionFilter/TestTypeFilter into
ui/filters.py keeping DojoFilter bases; re-exports omitted (would cycle) and
consumers (test/finding/test_type views) updated to the new path. EngagementTest*
filters left in dojo/filters.py (engagement domain); Api* filters left for Phase 7.

* refactor(test): move views + urls into dojo/test/ui/ [test Phase 5]

Move dojo/test/{views,urls}.py to dojo/test/ui/ and update consumers
(dojo/urls.py include + test_apply_finding_template test). urls.py now imports
views from dojo.test.ui.

* refactor(test): extract API layer into dojo/test/api/ [test Phase 6,7,8,9]

Move 8 Test-domain serializers into api/serializer.py (TestSerializer re-exported
from api_v2/serializers.py for ReportGenerateSerializer; rest omitted, sole consumers
were the moved viewsets), ApiTestFilter/TestImportAPIFilter into api/filters.py, and
TestsViewSet/TestTypesViewSet/TestImportViewSet into api/views.py. api/urls.py adds
add_test_urls() preserving routes tests/test_types/test_imports + basenames
test/test_type/test_imports. Viewset re-exports omitted (would cycle); dojo/urls.py +
test_rest_framework updated. Finding* serializers left in place.
Full rest_framework suite green (871 tests).

* refactor(engagement): extract Engagement/Engagement_Presets models into dojo/engagement/

Phase 1 of module reorg per AGENTS.md. Move Engagement, Engagement_Presets +
admin registrations into dojo/engagement/{models,admin}.py. Cross-module FKs use
string refs to avoid circular imports; ENGAGEMENT_STATUS_CHOICES single-sourced
with re-export. No migration change.

* refactor(engagement): extract copy_engagement workflow into services.py [Phase 2 pilot]

Phase 2 of module reorg per AGENTS.md. Move the engagement copy workflow (copy +
product grade recalc + notification) out of the copy_engagement view into an
HTTP-free copy_engagement(engagement, user) service, so both UI and (future) API
can reuse it. The inline notification carried a TODO asking for exactly this.
View is thinned to call the service. Add a unit test for the service (the workflow
was previously untested). Notification URL uses relative reverse() to match the
codebase convention.

* refactor(engagement): move forms + UI filters into dojo/engagement/ui/ [engagement Phase 3,4]

* refactor(engagement): move views + urls into dojo/engagement/ui/ [engagement Phase 5]

* refactor(engagement): extract API layer into dojo/engagement/api/ [engagement Phase 6,7,8,9]

* refactor(product): extract Product/Product_Line/Product_API_Scan_Configuration into dojo/product/

Phase 1 of module reorg per AGENTS.md. Move Product, Product_Line,
Product_API_Scan_Configuration + admin registrations into dojo/product/{models,admin}.py.
Cross-module FKs use string refs to avoid circular imports. Product_Type re-export
now pure backward-compat (F401). No migration change.

* refactor(product): move forms + UI filters into dojo/product/ui/ [product Phase 3,4]

* refactor(product): move views into dojo/product/ui/ [product Phase 5]

* refactor(product): extract API layer into dojo/product/api/ [product Phase 6,7,8,9]

* refactor(finding): extract Finding/Vulnerability_Id/Finding_Group/Finding_Template into dojo/finding/

Phase 1 of module reorg per AGENTS.md. Move Finding (+ custom FindingAdmin),
Vulnerability_Id, Finding_Group, Finding_Template + admin registrations into
dojo/finding/{models,admin}.py. Cross-module FKs use string refs; date/util field
defaults imported from dojo.models to preserve migration serialization path; restore
load-bearing parse_cvss_data re-export for dojo.location side-effect registration.
No migration change.

* refactor(finding): move forms + UI filters into dojo/finding/ui/ [finding Phase 3,4]

* refactor(finding): move views + urls into dojo/finding/ui/ [finding Phase 5]

* refactor(finding): extract API layer into dojo/finding/api/ [finding Phase 6,7,8,9]

* refactor(finding): fold CWE + BurpRawRequestResponse into dojo/finding/ [finding Phase 1,6,8,9]

* refactor(user): extract user module into dojo/user/ [user Phase 1,3,4,5,6,7,8,9]

* refactor(system_settings): extract System_Settings into dojo/system_settings/ [system_settings Phase 1,3,5,6,8,9]

* refactor(tools): extract tool_type, tool_config, tool_product into dojo/<module>/ [Phase 1,3,5,6,8,9]

* refactor(endpoint): extract endpoint module into dojo/endpoint/ [endpoint Phase 1,3,4,5,6,7,8,9]

* refactor(survey,benchmark): extract survey + benchmark modules into dojo/<module>/ [Phase 1,3,4,5]

* refactor(notes,files): extract note_type, notes, file_uploads, reports into dojo/<module>/ [Phase 1,3,5,6,8,9]

* refactor(risk_acceptance): extract Risk_Acceptance into dojo/risk_acceptance/ [risk_acceptance Phase 1,3,6,7,8,9]

* refactor(misc): extract regulations, banner, announcement, development_environment, object [Phase 1,3,4,5,6,8,9]

* docs(agents): mark Phase 10 peripheral modules Complete + refresh monolith line counts

* refactor(reorg): add backward-compat re-exports for moved symbols

External consumers (notably the dojo-pro plugin, a separate repo) still
import several symbols from their pre-reorg locations. Add the missing
backward-compat re-exports following the patterns already used on this
branch:

- dojo.forms: user forms (UserContactInfoForm, DojoUserForm, ...) + FindingForm
- dojo.api_v2.serializers: TestType/TestCreate/EngagementCheckList serializers
- dojo.api_v2.views: ViewSets moved into per-module api packages, exposed
  lazily via PEP 562 __getattr__ to avoid entry-order circular imports
- dojo.filters: filters moved into per-module ui/api packages, exposed
  lazily via __getattr__ (they import dojo.filters base classes), plus
  BooleanFilter and Product_API_Scan_Configuration
- dojo.auditlog: TAG_MODEL_MAPPING added to the lazy export table
- dojo/<module>/views.py: pure re-export shims to dojo/<module>/ui/views.py
  (matching the existing dojo/api_v2/permissions.py / pghistory_* shims)

* fix(test): retarget finding_added dispatch patch to moved serializer module

The reorg moved FindingCreateSerializer (and its dojo_dispatch_task call
for the finding_added notification) from dojo/api_v2/serializers.py to
dojo/finding/api/serializer.py. Three TestNotificationTriggersApi tests
still patched dojo.api_v2.serializers.dojo_dispatch_task, which no longer
has that attribute -> AttributeError at patch time. Point them at the
module where the dispatch now resolves.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* fix(reorg): restore disabled prefetch schema on DojoMetaViewSet

The DojoMetaViewSet extraction re-added an active
@extend_schema_view(**schema_with_prefetch()) above the intentionally
commented-out original. This re-enabled prefetch schema generation for
the DojoMeta endpoint, which emits a $ref to a Location component that
is not registered when V3_FEATURE_LOCATIONS=False, breaking the
openapi-generator schema validation integration test
(PaginatedMetaList.Location is not of type schema).

* fix(api): restore prefetch schema + serializer re-export for ProductAPIScanConfiguration

Two omissions from the API-layer extraction (commit 7ba6cf2), flagged in
review:
- Restore @extend_schema_view(**schema_with_prefetch()) on
  ProductAPIScanConfigurationViewSet (lost when extracted from the monolith) so
  the prefetch query params reappear in the OpenAPI docs.
- Re-export ProductAPIScanConfigurationSerializer from dojo/api_v2/serializers.py
  so the prefetcher can discover it, enabling prefetch of
  Product_API_Scan_Configuration on the /tests/ endpoint.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* refactor(object): import Product from dojo.product.models at module top

Address review feedback on DefectDojo#14987: replace the four repeated in-function
'from dojo.models import Product' lazy imports in dojo/object/ui/views.py with a
single top-level 'from dojo.product.models import Product'. dojo.product.models
does not import the object module, so this avoids the circular dependency
(no longer routing through the dojo.models hub) without the per-function repetition.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>

Author: valentijnscholten

AGENTS.md

@@ -0,0 +1 @@
+import dojo.announcement.admin  # noqa: F401

dojo/announcement/__init__.py

@@ -0,0 +1,6 @@
+from django.contrib import admin
+
+from dojo.announcement.models import Announcement, UserAnnouncement
+
+admin.site.register(Announcement)
+admin.site.register(UserAnnouncement)

dojo/announcement/admin.py

@@ -0,0 +1 @@
+path = "announcements"  # noqa: RUF067

dojo/announcement/api/__init__.py

@@ -0,0 +1,21 @@
+from django.db import IntegrityError
+from rest_framework import serializers
+
+from dojo.announcement.models import Announcement
+
+
+class AnnouncementSerializer(serializers.ModelSerializer):
+
+    class Meta:
+        model = Announcement
+        fields = "__all__"
+
+    def create(self, validated_data):
+        validated_data["id"] = 1
+        try:
+            return super().create(validated_data)
+        except IntegrityError as e:
+            if 'duplicate key value violates unique constraint "dojo_announcement_pkey"' in str(e):
+                msg = "No more than one Announcement is allowed"
+                raise serializers.ValidationError(msg)
+            raise

dojo/announcement/api/serializer.py

@@ -0,0 +1,7 @@
+from dojo.announcement.api import path
+from dojo.announcement.api.views import AnnouncementViewSet
+
+
+def add_announcement_urls(router):
+    router.register(path, AnnouncementViewSet, basename="announcement")
+    return router

dojo/announcement/api/urls.py

@@ -0,0 +1,20 @@
+from django_filters.rest_framework import DjangoFilterBackend
+
+from dojo.announcement.api.serializer import AnnouncementSerializer
+from dojo.announcement.models import Announcement
+from dojo.api_v2.views import DojoModelViewSet
+from dojo.authorization import api_permissions as permissions
+
+
+# Authorization: configuration
+class AnnouncementViewSet(
+    DojoModelViewSet,
+):
+    serializer_class = AnnouncementSerializer
+    queryset = Announcement.objects.none()
+    filter_backends = (DjangoFilterBackend,)
+    filterset_fields = "__all__"
+    permission_classes = (permissions.UserHasConfigurationPermissionStaff,)
+
+    def get_queryset(self):
+        return Announcement.objects.all().order_by("id")

dojo/announcement/api/views.py

@@ -0,0 +1,28 @@
+from django.db import models
+from django.utils.translation import gettext as _
+
+ANNOUNCEMENT_STYLE_CHOICES = (
+    ("info", "Info"),
+    ("success", "Success"),
+    ("warning", "Warning"),
+    ("danger", "Danger"),
+)
+
+
+class Announcement(models.Model):
+    message = models.CharField(max_length=500,
+                                help_text=_("This dismissable message will be displayed on all pages for authenticated users. It can contain basic html tags, for example <a href='https://www.fred.com' style='color: #337ab7;' target='_blank'>https://example.com</a>"),
+                                default="")
+    style = models.CharField(max_length=64, choices=ANNOUNCEMENT_STYLE_CHOICES, default="info",
+                            help_text=_("The style of banner to display. (info, success, warning, danger)"))
+    dismissable = models.BooleanField(default=False,
+                                      null=False,
+                                      blank=True,
+                                      verbose_name=_("Dismissable?"),
+                                      help_text=_("Ticking this box allows users to dismiss the current announcement"),
+                                      )
+
+
+class UserAnnouncement(models.Model):
+    announcement = models.ForeignKey("dojo.Announcement", null=True, editable=False, on_delete=models.CASCADE, related_name="user_announcement")
+    user = models.ForeignKey("dojo.Dojo_User", null=True, editable=False, on_delete=models.CASCADE)

dojo/announcement/models.py

@@ -1,7 +1,8 @@
 from django.db.models.signals import post_save
 from django.dispatch import receiver
 
-from dojo.models import Announcement, Dojo_User, UserAnnouncement
+from dojo.announcement.models import Announcement, UserAnnouncement
+from dojo.user.models import Dojo_User
 
 
 @receiver(post_save, sender=Dojo_User)