Skip to content

refactor: consolidate audit-log code into dojo/auditlog/ package#14763

Merged
Maffooch merged 3 commits intoDefectDojo:devfrom
Maffooch:auditlog-cleanup
May 1, 2026
Merged

refactor: consolidate audit-log code into dojo/auditlog/ package#14763
Maffooch merged 3 commits intoDefectDojo:devfrom
Maffooch:auditlog-cleanup

Conversation

@Maffooch
Copy link
Copy Markdown
Contributor

@Maffooch Maffooch commented Apr 27, 2026

Summary

  • Moves dojo/auditlog.py (1,083 lines), dojo/pghistory_models.py, and dojo/pghistory_utils.py into a self-contained dojo/auditlog/ package that mirrors the dojo/url/ canonical layout from CLAUDE.md.
  • Also pulls the audit-log view (action_history), its template, and LogEntryFilter/PgHistoryFilter into the package — they previously lived in dojo/views.py, dojo/templates/, and dojo/filters.py.
  • New dojo/auditlog/settings.py becomes the source of truth for the audit-log env-var schema and pghistory field defaults; settings.dist.py imports from it.
  • Backward-compatible re-exports stay at the original locations per the playbook, so the 21 existing call sites keep working without churn.

Layout

dojo/auditlog/
├── __init__.py        # PEP 562 lazy re-exports
├── models.py          # DojoEvents (app_label auto-inferred to "dojo")
├── services.py        # registration + cleanup + config
├── helpers.py         # display formatting + Celery context wrappers
├── backfill.py        # backfill subsystem
├── filters.py         # LogEntryFilter, PgHistoryFilter
├── settings.py        # env schema + pghistory field defaults
├── ui/
│   ├── __init__.py
│   ├── views.py       # action_history
│   └── urls.py        # action_history route
└── templates/dojo/action_history.html

Notable details

  • __init__.py uses PEP 562 lazy attribute resolution. settings.dist.py imports dojo.auditlog.settings at Django settings-load time, before django.conf.settings is built; eager re-exports would circularly pull in django-dependent submodules.
  • DojoEvents drops the explicit app_label = "dojo" per CLAUDE.md ("DO NOT set app_label in Meta — auto-inferred"). makemigrations --check reports zero new migrations.
  • Defensive try/except blocks in services and backfill are intentionally dropped — let pgtrigger / COPY failures surface loudly rather than silently swallowing them.
  • dojo/settings/settings.dist.py becomes a consumer of dojo.auditlog.settings: imports the env-schema dict (splatted into environ.Env via **AUDITLOG_ENV_SCHEMA), the three PGHISTORY_*_FIELD constants, and AUDITLOG_DISABLE_ON_RAW_SAVE; adds dojo/auditlog/templates to TEMPLATES[0][\"DIRS\"] since dojo.auditlog isn't a separate Django app.
  • unittests/test_auditlog.py: two @patch targets updated from dojo.auditlog.call_command to dojo.auditlog.services.call_command, reflecting where the import now lives.

Out of scope (deferred per playbook)

  • Updating the 21 stub-using call sites (apps.py, tasks.py, signals, management commands, migrations, two test files) to import from new paths — explicitly a separate cleanup pass.
  • Moving per-domain audit-log signal handlers (dojo/test/signals.py etc.) into dojo/auditlog/signals.py — they're domain-coupled by design.
  • Removing the legacy django-auditlog library; LogEntryFilter still references it for the page's "legacy entries" pane.

Test plan

  • python manage.py check — no issues
  • python manage.py makemigrations --checkNo changes detected (the critical migration gate)
  • unittests.test_auditlog — 7/7 pass
  • unittests.test_importers_performance — 10/10 pass
  • reverse('action_history', cid=1, oid=2) — resolves to /history/1/2 (route name preserved)
  • End-to-end: edit a Finding in the UI; visit /history/<cid>/<oid>; confirm template renders, filters work (event type, user, IP, diff search), and a pghistory event is recorded with user/url/remote_addr/source/scan_type context fields populated.

🤖 Generated with Claude Code

@Maffooch Maffooch requested a review from mtesauro as a code owner April 27, 2026 21:53
@github-actions github-actions Bot added settings_changes Needs changes to settings.py based on changes in settings.dist.py included in this PR unittests labels Apr 27, 2026
Move dojo/auditlog.py (1,083 lines), dojo/pghistory_models.py, and
dojo/pghistory_utils.py into a self-contained dojo/auditlog/ package
that mirrors the dojo/url/ canonical layout from CLAUDE.md. Also pulls
in the action_history view, its template, and the LogEntryFilter /
PgHistoryFilter that previously lived in dojo/views.py, dojo/templates/,
and dojo/filters.py.

Layout:
- models.py        DojoEvents proxy model (app_label dropped, auto-inferred)
- services.py      pghistory registration, retention/cleanup, config
- helpers.py       display formatting + Celery context wrappers
- backfill.py      backfill subsystem (used by migrations + commands)
- filters.py       LogEntryFilter, PgHistoryFilter
- settings.py      env-var schema + pghistory field defaults (source of truth)
- ui/views.py      action_history view
- ui/urls.py       action_history route (name preserved)
- templates/dojo/action_history.html

Backward-compatible re-exports stay at dojo/pghistory_models.py,
dojo/pghistory_utils.py, and dojo/filters.py per the playbook, so the
21 existing call sites keep working without churn. dojo/auditlog.py and
the original template are deleted (the package replaces the file).

Notable details:
- __init__.py uses PEP 562 lazy attribute resolution. settings.dist.py
  imports dojo.auditlog.settings at Django settings-load time, before
  django.conf.settings is built; eager re-exports would pull in
  django-dependent submodules and circularly import settings.
- DojoEvents drops the explicit app_label = "dojo" per CLAUDE.md
  ("DO NOT set app_label in Meta — auto-inferred"). makemigrations
  --check reports zero new migrations.
- Defensive try/except blocks in services and backfill are dropped per
  the user's directive — let pgtrigger / COPY failures surface loudly
  rather than silently swallowing them.
- settings.dist.py becomes a consumer of dojo.auditlog.settings:
  imports the env-schema dict (splatted into environ.Env via
  **AUDITLOG_ENV_SCHEMA), the three PGHISTORY_*_FIELD constants, and
  AUDITLOG_DISABLE_ON_RAW_SAVE; adds dojo/auditlog/templates to
  TEMPLATES[0]["DIRS"] since auditlog isn't a separate Django app.
- unittests/test_auditlog.py: two @patch targets updated from
  dojo.auditlog.call_command to dojo.auditlog.services.call_command,
  reflecting where the import now lives.

Verified end-to-end against the running stack:
- python manage.py check                       -> no issues
- python manage.py makemigrations --check      -> No changes detected
- unittests.test_auditlog                      -> 7/7 pass
- unittests.test_importers_performance         -> 10/10 pass
- reverse('action_history', cid=1, oid=2)      -> /history/1/2

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@Maffooch Maffooch added this to the 2.58.0 milestone Apr 27, 2026
Copy link
Copy Markdown
Member

@valentijnscholten valentijnscholten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Defensive try/except blocks in services and backfill are intentionally dropped — let pgtrigger / COPY failures surface loudly rather than silently swallowing them.

Why are these being dropped? In case of some unexpected error this would block users from starting their instances after an upgrade.

@Maffooch
Copy link
Copy Markdown
Contributor Author

I went back and looked at each spot to make sure the upgrade path wouldn't blow up, and I think we are actually safe here. For configure_pghistory_triggers has the same try/catch that was just an error log then a raise. The exception already propagated, so removing the wrapper only loses the friendlier log line ahead of the traceback, but the startup blocking behavior is identical.

For the outer try/catch around process_model_backfill, the only callers on the upgrade path are the backfill migrations, and both already wrap the call in their own try/catch. A backfill failure during migration wont abort the migration or block startup after an upgrade.

The one real behavior change is in the manual pghistory_backfill_fast. it now exits on model failure instead of silently moving on. If you'd like the friendlier log message ahead of the traceback, happy to add the error log back and let it continue if there was an error

Restore the seven try/except blocks that were dropped in the auditlog/
reorg: pgtrigger enable/disable wrappers in configure_pghistory_triggers,
the outer + per-batch (consecutive_failures) wrappers and the id-column
ValueError fallback in process_model_backfill, and the LookupError +
per-event Exception fallbacks in process_events_for_display.

Behavior now matches dev: a single bad pghistory event no longer 500s
the action-history page, a single bad batch is logged and skipped (up
to 3 in a row before bailing on the model), a model-level failure
returns (0, 0.0) so pghistory_backfill_fast continues with the next
model, and pgtrigger enable/disable surface a friendly diagnostic log
line ahead of the traceback before re-raising.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@dryrunsecurity
Copy link
Copy Markdown

dryrunsecurity Bot commented Apr 29, 2026

DryRun Security

This pull request contains a high-severity SQL injection risk in dojo/auditlog/backfill.py, where multiple queries and a COPY statement are built with f-string interpolation of table names and SQL fragments. Although the values currently come from internal mappings, the raw SQL pattern could become exploitable if any of those inputs are ever influenced by attacker-controlled data.

🟠 Potential SQL Injection in dojo/auditlog/backfill.py (drs_6ece4114)
Vulnerability Potential SQL Injection
Description The code uses raw SQL execution with dynamically formatted table names and SQL fragments via cursor.execute(f"SELECT COUNT(*) FROM {table_name}"), cursor.execute(f"SELECT COUNT(*) FROM {event_table_name} ..."), cursor.execute(f"SELECT t.id FROM {table_name} t ..."), and copy_sql = f"COPY {event_table_name} (...) ...". Even though the table names come from internal mappings rather than direct request input, this is still raw SQL built via string interpolation, which is the kind of pattern that can lead to SQL injection if any of those values become attacker-controlled or are modified unsafely.

cursor.execute(f"SELECT COUNT(*) FROM {table_name}")
total_count = cursor.fetchone()[0]
if total_count == 0:


Comment to provide feedback on these findings.

Report false positive: @dryrunsecurity fp [FINDING ID] [FEEDBACK]
Report low-impact: @dryrunsecurity nit [FINDING ID] [FEEDBACK]

Example: @dryrunsecurity fp drs_90eda195 This code is not user-facing

All finding details can be found in the DryRun Security Dashboard.

@valentijnscholten
Copy link
Copy Markdown
Member

I went back and looked at each spot to make sure the upgrade path wouldn't blow up, and I think we are actually safe here. For configure_pghistory_triggers has the same try/catch that was just an error log then a raise. The exception already propagated, so removing the wrapper only loses the friendlier log line ahead of the traceback, but the startup blocking behavior is identical.

For the outer try/catch around process_model_backfill, the only callers on the upgrade path are the backfill migrations, and both already wrap the call in their own try/catch. A backfill failure during migration wont abort the migration or block startup after an upgrade.

The one real behavior change is in the manual pghistory_backfill_fast. it now exits on model failure instead of silently moving on. If you'd like the friendlier log message ahead of the traceback, happy to add the error log back and let it continue if there was an error

ok sounds like it is ok as-is

@github-actions
Copy link
Copy Markdown
Contributor

This pull request has conflicts, please resolve those before we can evaluate the pull request.

Copy link
Copy Markdown
Contributor

@mtesauro mtesauro left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved

# Conflicts:
#	dojo/settings/settings.dist.py
@github-actions
Copy link
Copy Markdown
Contributor

Conflicts have been resolved. A maintainer will review the pull request shortly.

@Maffooch Maffooch merged commit 29fb41e into DefectDojo:dev May 1, 2026
157 checks passed
@Maffooch Maffooch deleted the auditlog-cleanup branch May 1, 2026 19:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

settings_changes Needs changes to settings.py based on changes in settings.dist.py included in this PR unittests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants