Skip to content

Commit 9e935c4

Browse files
rossopsrossops
authored
Merge pull request #14705 from Maffooch/chore/module-reorganization-playbook
docs: add CLAUDE.md with module reorganization playbook
2 parents a61ceeb + 56c84a7 commit 9e935c4

File tree

1 file changed

+249
-0
lines changed

1 file changed

+249
-0
lines changed

‎CLAUDE.md‎

Lines changed: 249 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,249 @@
1+
# DefectDojo Development Guide
2+
3+
## Project Overview
4+
5+
DefectDojo is a Django application (`dojo` app) for vulnerability management. The codebase is undergoing a modular reorganization to move from monolithic files toward self-contained domain modules.
6+
7+
## Module Reorganization
8+
9+
### Reference Pattern: `dojo/url/`
10+
11+
All domain modules should match the structure of `dojo/url/`. This is the canonical example of a fully reorganized module.
12+
13+
```
14+
dojo/{module}/
15+
├── __init__.py # import dojo.{module}.admin # noqa: F401
16+
├── models.py # Domain models, constants, factory methods
17+
├── admin.py # @admin.register() for the module's models
18+
├── services.py # Business logic (no HTTP concerns)
19+
├── queries.py # Complex DB aggregations/annotations
20+
├── signals.py # Django signal handlers
21+
├── [manager.py] # Custom QuerySet/Manager if needed
22+
├── [validators.py] # Field-level validators if needed
23+
├── [helpers.py] # Async task wrappers, tag propagation, etc.
24+
├── ui/
25+
│ ├── __init__.py # Empty
26+
│ ├── forms.py # Django ModelForms
27+
│ ├── filters.py # UI-layer django-filter classes
28+
│ ├── views.py # Thin view functions — delegates to services.py
29+
│ └── urls.py # URL routing
30+
└── api/
31+
├── __init__.py # path = "{module}"
32+
├── serializer.py # DRF serializers
33+
├── views.py # API ViewSets — delegates to services.py
34+
├── filters.py # API-layer filters
35+
└── urls.py # add_{module}_urls(router) registration
36+
```
37+
38+
### Architecture Principles
39+
40+
41+
**services.py is the critical layer**: Both `ui/views.py` and `api/views.py` call `services.py` for business logic. Services accept domain objects and primitives — never request/response objects, forms, or serializers.
42+
43+
**Backward-compatible re-exports**: When moving code out of monolithic files (`dojo/models.py`, `dojo/forms.py`, `dojo/filters.py`, `dojo/api_v2/serializers.py`, `dojo/api_v2/views.py`), always leave a re-export at the original location:
44+
```python
45+
from dojo.{module}.models import {Model} # noqa: F401 -- backward compat
46+
```
47+
Never remove re-exports until all consumers are updated in a dedicated cleanup pass.
48+
49+
### Current State
50+
51+
Modules in various stages of reorganization:
52+
53+
| Module | models.py | services.py | ui/ | api/ | Status |
54+
|--------|-----------|-------------|-----|------|--------|
55+
| **url** | In module | N/A | Done | Done | **Complete** |
56+
| **location** | In module | N/A | N/A | Done | **Complete** |
57+
| **product_type** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work |
58+
| **test** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work |
59+
| **engagement** | In dojo/models.py | Partial (32 lines) | Partial (views at root) | In dojo/api_v2/ | Needs work |
60+
| **product** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work |
61+
| **finding** | In dojo/models.py | Missing | Partial (views at root) | In dojo/api_v2/ | Needs work |
62+
63+
### Monolithic Files Being Decomposed
64+
65+
These files still contain code for multiple modules. Extract code to the target module's subdirectory and leave a re-export stub.
66+
67+
- `dojo/models.py` (4,973 lines) — All model definitions
68+
- `dojo/forms.py` (4,127 lines) — All Django forms
69+
- `dojo/filters.py` (4,016 lines) — All UI and API filter classes
70+
- `dojo/api_v2/serializers.py` (3,387 lines) — All DRF serializers
71+
- `dojo/api_v2/views.py` (3,519 lines) — All API viewsets
72+
73+
---
74+
75+
## Reorganization Playbook
76+
77+
When asked to reorganize a module, follow these phases in order. Each phase should be independently verifiable.
78+
79+
### Phase 0: Pre-Flight (Read-Only)
80+
81+
Before any changes, identify all code to extract:
82+
83+
```bash
84+
# 1. Model classes and line ranges in dojo/models.py
85+
grep -n "class {Model}" dojo/models.py
86+
87+
# 2. Form classes in dojo/forms.py
88+
grep -n "class.*{Module}" dojo/forms.py
89+
grep -n "model = {Model}" dojo/forms.py
90+
91+
# 3. Filter classes in dojo/filters.py
92+
grep -n "class.*{Module}\|class.*{Model}" dojo/filters.py
93+
94+
# 4. Serializer classes
95+
grep -n "class.*{Model}" dojo/api_v2/serializers.py
96+
97+
# 5. ViewSet classes
98+
grep -n "class.*{Model}\|class.*{Module}" dojo/api_v2/views.py
99+
100+
# 6. Admin registrations
101+
grep -n "admin.site.register({Model}" dojo/models.py
102+
103+
# 7. All import sites (to verify backward compat)
104+
grep -rn "from dojo.models import.*{Model}" dojo/ unittests/
105+
106+
# 8. Business logic in current views
107+
# Scan dojo/{module}/views.py for: .save(), .delete(), create_notification(),
108+
# jira_helper.*, dojo_dispatch_task(), multi-model workflows
109+
```
110+
111+
### Phase 1: Extract Models
112+
113+
1. Create `dojo/{module}/models.py` with the model class(es) and associated constants
114+
2. Create `dojo/{module}/admin.py` with `admin.site.register()` calls (remove from `dojo/models.py`)
115+
3. Update `dojo/{module}/__init__.py` to `import dojo.{module}.admin # noqa: F401`
116+
4. Add re-exports in `dojo/models.py`
117+
5. Remove original model code (keep re-export line)
118+
119+
**Import rules for models.py:**
120+
- Upward FKs (e.g., Test -> Engagement): import from `dojo.models` if not yet extracted, or `dojo.{module}.models` if already extracted
121+
- Downward references (e.g., Product_Type querying Finding): use lazy imports inside method bodies
122+
- Shared utilities (`copy_model_util`, `_manage_inherited_tags`, `get_current_date`, etc.): import from `dojo.models`
123+
- Do NOT set `app_label` in Meta — all models inherit `dojo` app_label automatically
124+
125+
**Verify:**
126+
```bash
127+
python manage.py check
128+
python manage.py makemigrations --check
129+
python -c "from dojo.{module}.models import {Model}"
130+
python -c "from dojo.models import {Model}"
131+
```
132+
133+
### Phase 2: Extract Services
134+
135+
Create `dojo/{module}/services.py` with business logic extracted from UI views.
136+
137+
**What belongs in services.py:**
138+
- State transitions (close, reopen, status changes)
139+
- Multi-step creation/update workflows
140+
- External integration calls (JIRA, GitHub)
141+
- Notification dispatching
142+
- Copy/clone operations
143+
- Bulk operations
144+
- Merge operations
145+
146+
**What stays in views:**
147+
- HTTP request/response handling
148+
- Form instantiation and validation
149+
- Serialization/deserialization
150+
- Authorization checks (`@user_is_authorized`, `user_has_permission_or_403`)
151+
- Template rendering, redirects
152+
- Pagination, breadcrumbs
153+
154+
**Service function pattern:**
155+
```python
156+
def close_engagement(engagement: Engagement, user: User) -> Engagement:
157+
engagement.active = False
158+
engagement.status = "Completed"
159+
engagement.save()
160+
if jira_helper.get_jira_project(engagement):
161+
dojo_dispatch_task(jira_helper.close_epic, engagement.id, push_to_jira=True)
162+
return engagement
163+
```
164+
165+
Update UI views and API viewsets to call the service instead of containing logic inline.
166+
167+
### Phase 3: Extract Forms to `ui/forms.py`
168+
169+
1. Create `dojo/{module}/ui/__init__.py` (empty)
170+
2. Create `dojo/{module}/ui/forms.py` — move form classes from `dojo/forms.py`
171+
3. Add re-exports in `dojo/forms.py`
172+
173+
### Phase 4: Extract UI Filters to `ui/filters.py`
174+
175+
1. Create `dojo/{module}/ui/filters.py` — move module-specific filters from `dojo/filters.py`
176+
2. Shared base classes (`DojoFilter`, `DateRangeFilter`, `ReportBooleanFilter`) stay in `dojo/filters.py`
177+
3. Add re-exports in `dojo/filters.py`
178+
179+
### Phase 5: Move UI Views/URLs into `ui/`
180+
181+
1. Move `dojo/{module}/views.py` -> `dojo/{module}/ui/views.py`
182+
2. Move `dojo/{module}/urls.py` -> `dojo/{module}/ui/urls.py`
183+
3. Update URL imports:
184+
- product: update `dojo/asset/urls.py`
185+
- product_type: update `dojo/organization/urls.py`
186+
- others: update the include in `dojo/urls.py`
187+
188+
### Phase 6: Extract API Serializers to `api/serializer.py`
189+
190+
1. Create `dojo/{module}/api/__init__.py` with `path = "{module}"`
191+
2. Create `dojo/{module}/api/serializer.py` — move from `dojo/api_v2/serializers.py`
192+
3. Add re-exports in `dojo/api_v2/serializers.py`
193+
194+
### Phase 7: Extract API Filters to `api/filters.py`
195+
196+
1. Create `dojo/{module}/api/filters.py` — move `Api{Model}Filter` from `dojo/filters.py`
197+
2. Add re-exports
198+
199+
### Phase 8: Extract API ViewSets to `api/views.py`
200+
201+
1. Create `dojo/{module}/api/views.py` — move from `dojo/api_v2/views.py`
202+
2. Add re-exports in `dojo/api_v2/views.py`
203+
204+
### Phase 9: Extract API URL Registration
205+
206+
1. Create `dojo/{module}/api/urls.py`:
207+
```python
208+
from dojo.{module}.api import path
209+
from dojo.{module}.api.views import {ViewSet}
210+
211+
def add_{module}_urls(router):
212+
router.register(path, {ViewSet}, path)
213+
return router
214+
```
215+
2. Update `dojo/urls.py` — replace `v2_api.register(...)` with `add_{module}_urls(v2_api)`
216+
217+
### After Each Phase: Verify
218+
219+
```bash
220+
python manage.py check
221+
python manage.py makemigrations --check
222+
python -m pytest unittests/ -x --timeout=120
223+
```
224+
225+
---
226+
227+
## Cross-Module Dependencies
228+
229+
The model hierarchy is: Product_Type -> Product -> Engagement -> Test -> Finding
230+
231+
Extract in this order (top to bottom) so that upward FKs can import from already-extracted modules. The recommended order is: product_type, test, engagement, product, finding.
232+
233+
For downward references (e.g., Product_Type's cached properties querying Finding), always use lazy imports:
234+
```python
235+
@cached_property
236+
def critical_present(self):
237+
from dojo.models import Finding # lazy import
238+
return Finding.objects.filter(test__engagement__product__prod_type=self, severity="Critical").exists()
239+
```
240+
241+
---
242+
243+
## Key Technical Details
244+
245+
- **Single Django app**: Everything is under `app_label = "dojo"`. Moving models to subdirectories does NOT require migration changes.
246+
- **Model discovery**: Triggered by `__init__.py` importing `admin.py`, which imports `models.py`. This is the same chain `dojo/url/` uses.
247+
- **Signal registration**: Handled in `dojo/apps.py` via `import dojo.{module}.signals`. Already set up for test, engagement, product, product_type.
248+
- **Watson search**: Uses `self.get_model("Product")` in `apps.py` — works via Django's model registry regardless of file location.
249+
- **Admin registration**: Currently at the bottom of `dojo/models.py` (lines 4888-4973). Must be moved to `{module}/admin.py` and removed from `dojo/models.py` to avoid `AlreadyRegistered` errors.

0 commit comments

Comments
 (0)