Skip to content

Commit c08db32

Browse files
skywalke34skywalke34
authored
feat(parsers): add Alert Logic CSV parser (#14930)
* feat(parser): scaffold Alert Logic parser package Empty __init__.py + stub parser.py with the 4 required methods returning placeholder values. Sets up the package for TDD tests to import against before the real implementation in Task 8. Authored by T. Walker - DefectDojo * test(parser): add synthetic Alert Logic CSV fixtures Three fixtures matching the 26-column Alert Logic vulnerability export shape (UTF-8 BOM, embedded CRLF in multi-line fields): - no_vuln.csv — header only, 0 data rows - one_vuln.csv — single Medium finding (HTTP/2 Rapid Reset) - many_vulns.csv — 7 rows covering Info / Low / Medium / High / Critical, with/without CVE, single & multi-IP (IPv4+IPv6), CISA Known Exploited Yes/No, multi-line Description and Resolution, a >500-char title for truncation test, empty CVSS and empty Operating System edge cases. All asset names, IPs, deployment names, and the customer account are synthetic (reserved doc IP ranges 192.0.2.x / 198.51.100.x / 203.0.113.x; .example.com hostnames; fictional AcmeCorp account). CVE identifiers and their associated descriptions/resolutions are from public sources. Authored by T. Walker - DefectDojo * test(parser): add failing TDD scaffold for Alert Logic parser Skeleton with 4 tests: get_scan_types, parse_no_findings, parse_one_finding, parse_many_findings. The one/many assertions fail against the Task 3 stub (which returns []) — that's the intended TDD red state. Full field-validation tests will be appended in Task 9 after the parser implementation lands in Task 8. Authored by T. Walker - DefectDojo * feat(parser): implement Alert Logic CSV parser Parses Alert Logic vulnerability scan CSV exports (26 columns, UTF-8 with BOM, multi-line quoted fields). Single-format, monolithic implementation following the IriusRisk skeleton. Field mapping: - Vulnerability → title (truncated at 500 chars with ellipsis) - Severity → severity (direct 1:1 Info/Low/Medium/High/Critical) - CVSS Score → cvssv3_score (float, None if empty) - Asset Name → component_name - IP Address → unsaved_endpoints (comma-split IPv4/IPv6) - Protocol/Port → endpoint protocol + port (port 0 → omitted) - CVE → unsaved_vulnerability_ids - Resolution → mitigation - Vulnerability ID → unique_id_from_tool (stable native ID) - Description, Evidence, OS, Vuln Span ID, Vuln Key, Asset Key/Type, Service, Category, VPC/Network, Deployment Name, Customer Account, First Seen, Last Scanned, Published Date, Age (days), CISA KEV → description (markdown table) - CISA Known Exploited = Yes → unsaved_tags: ["cisa-known-exploited"] static_finding=True, dynamic_finding=False (infrastructure vulnerability scanner pattern, matches Qualys VMDR). All 7 fixture findings parse cleanly with correct severities, multi-IP endpoint extraction (IPv4+IPv6), title truncation, CVE list, CVSS score, and tags. endpoint.clean() passes on all 10 endpoints generated from the many_vulns fixture. Authored by T. Walker - DefectDojo * test(parser): add field-validation tests for Alert Logic parser Adds 28 new tests on top of the TDD scaffold, bringing total coverage to 32 tests. Categories covered: - Scan-type metadata: get_label, get_description - Basic fields: title, severity, component_name, unique_id_from_tool, cvssv3_score, static/dynamic flags, mitigation content, description structure - Severity mapping: one test per source level (Info/Low/Medium/High/Critical) - Title truncation: long (>500) gets [:497] + "...", short stays as-is - unique_id_from_tool: distinct values per finding, matches source - Endpoints: single IPv4, multi-IP (IPv4+IPv6), IPv6-only, port=0 omission, endpoint.clean() on every endpoint - CVE handling: present and absent - CISA Known Exploited tag: added on "Yes", absent on "No" - CVSS score: parsed when present, None when empty - BOM handling: title resolves correctly (proves UTF-8 BOM is stripped) - Multi-line field preservation in description All 32 tests pass against the parser implementation from the previous commit. Authored by T. Walker - DefectDojo * docs(parser): add Alert Logic parser documentation Documents the Alert Logic CSV parser including: - File-export workflow from the Alert Logic console - Default deduplication strategy (unique_id_from_tool + hashcode fallback) - Complete 26-column field mapping table (expandable) - Additional Finding field settings (static/dynamic flags, active default) - Special processing notes covering severity conversion, title truncation, description construction, endpoint multi-IP / IPv6 / port-zero handling, deduplication algorithm, CVE handling, CISA Known Exploited tagging, and UTF-8 BOM + multi-line field handling Authored by T. Walker - DefectDojo * feat(parser): register Alert Logic deduplication configuration Adds Alert Logic Scan entries to: - HASHCODE_FIELDS_PER_SCANNER with ["title", "component_name", "vuln_id_from_tool"] (fallback when Vulnerability ID is missing on a row) - DEDUPLICATION_ALGORITHM_PER_PARSER as DEDUPE_ALGO_UNIQUE_ID_FROM_TOOL_OR_HASH_CODE (uses Vulnerability ID as the stable native identifier with hashcode fallback) Mirrors the Qualys VMDR dedup pattern (same field set, same algorithm). Authored by T. Walker - DefectDojo * fix(parser): support V3_FEATURE_LOCATIONS in Alert Logic parser The Endpoint model is deprecated and raises NotImplementedError when V3_FEATURE_LOCATIONS is enabled. Build LocationData URL locations in that mode and fall back to Endpoint otherwise, matching the established parser migration pattern (e.g. Qualys VMDR). Endpoint tests now read via the get_unsaved_locations helper so they pass under both settings. Authored by T. Walker - DefectDojo
1 parent 98e801a commit c08db32

8 files changed

Lines changed: 547 additions & 0 deletions

File tree

‎docs/content/supported_tools/parsers/file/alertlogic.md‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
---
2+
title: "Alert Logic"
3+
toc_hide: true
4+
---
5+
6+
The [Alert Logic](https://www.alertlogic.com/) parser for DefectDojo supports imports from CSV format. This document details the parsing of Alert Logic vulnerability scan exports into DefectDojo field mappings, unmapped fields, and transformation notes for easier troubleshooting and analysis.
7+
8+
## Supported File Types
9+
10+
The Alert Logic parser accepts CSV file format. To generate this file from Alert Logic:
11+
12+
1. Log into the Alert Logic console
13+
2. Navigate to **Validate → Vulnerabilities** (or the equivalent vulnerability listing view)
14+
3. Apply the filters you want included in the export
15+
4. Export the filtered vulnerability list as CSV
16+
5. Save the file with a `.csv` extension
17+
6. Upload to DefectDojo using the "Alert Logic Scan" scan type
18+
19+
The parser handles UTF-8 with byte-order mark (BOM) and multi-line quoted fields commonly present in Description, Evidence, and Resolution columns.
20+
21+
## Default Deduplication Hashcode Fields
22+
23+
Alert Logic provides a stable native vulnerability identifier in the `Vulnerability ID` column. DefectDojo uses it as `unique_id_from_tool` with hashcode fields as a fallback:
24+
25+
- title
26+
- component_name
27+
- vuln_id_from_tool
28+
29+
### Sample Scan Data
30+
31+
Sample Alert Logic scans can be found in the [sample scan data folder](https://github.com/DefectDojo/django-DefectDojo/tree/master/unittests/scans/alertlogic).
32+
33+
## Link To Tool
34+
35+
- [Alert Logic](https://www.alertlogic.com/)
36+
- [Alert Logic Documentation](https://docs.alertlogic.com/)
37+
38+
## CSV Format
39+
40+
### Total Fields in CSV
41+
42+
- Total data fields: 26
43+
- Total data fields parsed: 26
44+
- Total data fields NOT parsed: 0
45+
46+
### CSV Format Field Mapping Details
47+
48+
<details>
49+
<summary>Click to expand Field Mapping Table</summary>
50+
51+
| Source Field | DefectDojo Field | Notes |
52+
| ----------------------- | --------------------------- | ------------------------------------------------------------------------------ |
53+
| Vulnerability | title | Truncated to 500 characters with "..." suffix if longer |
54+
| Severity | severity | Direct one-to-one mapping (Info / Low / Medium / High / Critical) |
55+
| CVSS Score | cvssv3_score | Parsed as float; empty values produce no score |
56+
| Asset Name | component_name | The affected host or service from the scan |
57+
| IP Address | unsaved_endpoints | Comma-separated IPv4 / IPv6 list; each value becomes a separate endpoint |
58+
| Protocol/Port | unsaved_endpoints | Parsed as `PROTOCOL/PORT`; a port of 0 is omitted |
59+
| CVE | unsaved_vulnerability_ids | Single CVE identifier when present |
60+
| Resolution | mitigation | Direct copy, including multi-line content |
61+
| Vulnerability ID | unique_id_from_tool | Alert Logic's stable native vulnerability identifier (used for deduplication) |
62+
| Description | description | Included in structured description block |
63+
| Evidence | description | Included in structured description block |
64+
| Operating System | description | Included in structured description block (CPE strings preserved) |
65+
| Vulnerability Span ID | description | Included in structured description block |
66+
| Vulnerability Key | description | Included in structured description block |
67+
| Asset Key | description | Included in structured description block |
68+
| Asset Type | description | Included in structured description block |
69+
| Service | description | Included in structured description block |
70+
| Category | description | Included in structured description block |
71+
| VPC/Network | description | Included in structured description block |
72+
| Deployment Name | description | Included in structured description block |
73+
| Customer Account | description | Included in structured description block |
74+
| First Seen | description | Included in structured description block |
75+
| Last Scanned | description | Included in structured description block |
76+
| Published Date | description | Included in structured description block |
77+
| Age (days) | description | Included in structured description block |
78+
| CISA Known Exploited | description, unsaved_tags | Added as `cisa-known-exploited` tag when value is "Yes" |
79+
80+
</details>
81+
82+
### Additional Finding Field Settings (CSV Format)
83+
84+
<details>
85+
<summary>Click to expand Additional Settings Table</summary>
86+
87+
| Finding Field | Default Value | Notes |
88+
| ---------------- | ------------- | ----------------------------------------------------------- |
89+
| static_finding | True | Alert Logic is an infrastructure vulnerability scanner |
90+
| dynamic_finding | False | Alert Logic is an infrastructure vulnerability scanner |
91+
| active | True | Alert Logic exports do not carry a mitigation status column |
92+
93+
</details>
94+
95+
## Special Processing Notes
96+
97+
### Severity Conversion
98+
99+
Alert Logic uses a five-level severity scale that aligns one-to-one with DefectDojo severity levels:
100+
101+
- `Critical` → Critical
102+
- `High` → High
103+
- `Medium` → Medium
104+
- `Low` → Low
105+
- `Info` → Info
106+
107+
Any unrecognized severity value defaults to Info.
108+
109+
### Title Format
110+
111+
Finding titles are derived from the "Vulnerability" column. Titles longer than 500 characters are truncated to 497 characters with a "..." suffix appended. Shorter titles are used as-is without modification.
112+
113+
### Description Construction
114+
115+
The parser constructs a structured markdown description containing all relevant CSV fields not already mapped to dedicated Finding columns. Each field is rendered as `**Label:** value` with blank lines between entries. Fields are included only when they contain a non-empty value, so the description stays tight for sparsely populated rows.
116+
117+
### Endpoint Construction
118+
119+
The "IP Address" column may contain one or more comma-separated IP addresses, mixing IPv4 and IPv6 (for example: `198.51.100.30, fe80::250:56ff:fe96:b97`). Each address becomes a separate endpoint. The "Protocol/Port" column is parsed as `PROTOCOL/PORT` (e.g., `TCP/443`); when the port is `0` the value is treated as "no specific port" and omitted from the endpoint. All endpoints are validated via `endpoint.clean()` before being attached to the finding.
120+
121+
### Deduplication
122+
123+
Alert Logic exports include a stable per-vulnerability identifier in the "Vulnerability ID" column. DefectDojo uses this as `unique_id_from_tool` and the deduplication algorithm `DEDUPE_ALGO_UNIQUE_ID_FROM_TOOL_OR_HASH_CODE`. When the ID is missing (some scan exports omit it for non-vulnerability findings), DefectDojo falls back to the hashcode algorithm using `title`, `component_name`, and `vuln_id_from_tool` (the CVE) as the stable fields.
124+
125+
### CVE Handling
126+
127+
The "CVE" column carries a single CVE identifier or is empty. When present it is attached to the finding via `unsaved_vulnerability_ids`; when absent no CVE is set.
128+
129+
### CISA Known Exploited Tagging
130+
131+
When the "CISA Known Exploited" column equals "Yes", the finding receives a `cisa-known-exploited` tag. This makes it straightforward to filter, route, or escalate findings already known to be exploited in the wild.
132+
133+
### BOM and Multi-Line Field Handling
134+
135+
Alert Logic exports start with a UTF-8 byte-order mark (`\xef\xbb\xbf`). The parser uses `utf-8-sig` decoding to strip the BOM transparently. Description, Evidence, and Resolution columns frequently contain multi-line content (separated by `\r\n` inside the quoted field); these newlines are preserved in the resulting `description` and `mitigation` Finding fields.

‎dojo/settings/settings.dist.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1107,6 +1107,7 @@ def generate_url(scheme, double_slashes, user, password, host, port, path, param
11071107
"Orca Security Alerts": ["title", "component_name"],
11081108
"Xygeni SCA Scan": ["vulnerability_ids", "component_name", "component_version"],
11091109
"Qualys VMDR": ["title", "component_name", "vuln_id_from_tool"],
1110+
"Alert Logic Scan": ["title", "component_name", "vuln_id_from_tool"],
11101111
}
11111112

11121113
# Override the hardcoded settings here via the env var
@@ -1381,6 +1382,7 @@ def generate_url(scheme, double_slashes, user, password, host, port, path, param
13811382
"Xygeni SCA Scan": DEDUPE_ALGO_UNIQUE_ID_FROM_TOOL_OR_HASH_CODE,
13821383
"Xygeni Secrets Scan": DEDUPE_ALGO_UNIQUE_ID_FROM_TOOL,
13831384
"Qualys VMDR": DEDUPE_ALGO_UNIQUE_ID_FROM_TOOL_OR_HASH_CODE,
1385+
"Alert Logic Scan": DEDUPE_ALGO_UNIQUE_ID_FROM_TOOL_OR_HASH_CODE,
13841386
}
13851387

13861388
# Override the hardcoded settings here via the env var

‎dojo/tools/alertlogic/__init__.py‎

Whitespace-only changes.

‎dojo/tools/alertlogic/parser.py‎

Lines changed: 172 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,172 @@
1+
import csv
2+
import io
3+
4+
from django.conf import settings
5+
6+
from dojo.models import Endpoint, Finding
7+
from dojo.tools.locations import LocationData
8+
9+
SEVERITY_MAPPING = {
10+
"Info": "Info",
11+
"Low": "Low",
12+
"Medium": "Medium",
13+
"High": "High",
14+
"Critical": "Critical",
15+
}
16+
17+
18+
class AlertlogicParser:
19+
20+
def get_scan_types(self):
21+
return ["Alert Logic Scan"]
22+
23+
def get_label_for_scan_types(self, scan_type):
24+
return scan_type
25+
26+
def get_description_for_scan_types(self, scan_type):
27+
return "Import Alert Logic vulnerability scan findings (CSV)."
28+
29+
def get_findings(self, filename, test):
30+
content = filename.read()
31+
if isinstance(content, bytes):
32+
content = content.decode("utf-8-sig")
33+
elif content.startswith(""):
34+
content = content.lstrip("")
35+
36+
reader = csv.DictReader(io.StringIO(content), delimiter=",", quotechar='"')
37+
findings = []
38+
for row in reader:
39+
vuln = (row.get("Vulnerability") or "").strip()
40+
if not vuln:
41+
continue
42+
43+
severity_raw = (row.get("Severity") or "").strip()
44+
severity = SEVERITY_MAPPING.get(severity_raw, "Info")
45+
46+
title = vuln[:497] + "..." if len(vuln) > 500 else vuln
47+
48+
description = _build_description(row)
49+
mitigation = (row.get("Resolution") or "").strip()
50+
component_name = (row.get("Asset Name") or "").strip() or None
51+
unique_id = (row.get("Vulnerability ID") or "").strip() or None
52+
cve = (row.get("CVE") or "").strip()
53+
54+
finding = Finding(
55+
test=test,
56+
title=title,
57+
severity=severity,
58+
description=description,
59+
mitigation=mitigation,
60+
component_name=component_name,
61+
unique_id_from_tool=unique_id,
62+
static_finding=True,
63+
dynamic_finding=False,
64+
)
65+
66+
cvssv3_score = _parse_cvss(row.get("CVSS Score"))
67+
if cvssv3_score is not None:
68+
finding.cvssv3_score = cvssv3_score
69+
70+
if cve:
71+
finding.unsaved_vulnerability_ids = [cve]
72+
73+
_add_locations(
74+
finding,
75+
row.get("IP Address"),
76+
row.get("Protocol/Port"),
77+
)
78+
79+
tags = _build_tags(row)
80+
if tags:
81+
finding.unsaved_tags = tags
82+
83+
findings.append(finding)
84+
85+
return findings
86+
87+
88+
def _build_description(row):
89+
field_order = [
90+
("Description", "Description"),
91+
("Evidence", "Evidence"),
92+
("Operating System", "Operating System"),
93+
("Vulnerability ID", "Vulnerability ID"),
94+
("Vulnerability Span ID", "Vulnerability Span ID"),
95+
("Vulnerability Key", "Vulnerability Key"),
96+
("Asset Key", "Asset Key"),
97+
("Asset Type", "Asset Type"),
98+
("Service", "Service"),
99+
("Category", "Category"),
100+
("VPC/Network", "VPC/Network"),
101+
("Deployment Name", "Deployment Name"),
102+
("Customer Account", "Customer Account"),
103+
("First Seen", "First Seen"),
104+
("Last Scanned", "Last Scanned"),
105+
("Published Date", "Published Date"),
106+
("Age (days)", "Age (days)"),
107+
("CISA Known Exploited", "CISA Known Exploited"),
108+
]
109+
parts = []
110+
for source_field, label in field_order:
111+
value = (row.get(source_field) or "").strip()
112+
if value:
113+
parts.append(f"**{label}:** {value}")
114+
return "\n\n".join(parts)
115+
116+
117+
def _parse_cvss(value):
118+
if value is None:
119+
return None
120+
value = value.strip()
121+
if not value:
122+
return None
123+
try:
124+
return float(value)
125+
except ValueError:
126+
return None
127+
128+
129+
def _add_locations(finding, ip_field, protoport_field):
130+
if not ip_field:
131+
return
132+
protocol, port = _parse_proto_port(protoport_field)
133+
for raw_host in ip_field.split(","):
134+
host = raw_host.strip()
135+
if not host:
136+
continue
137+
if settings.V3_FEATURE_LOCATIONS:
138+
finding.unsaved_locations.append(
139+
LocationData.url(host=host, protocol=protocol or "", port=port),
140+
)
141+
else:
142+
# TODO: Delete this after the move to Locations
143+
kwargs = {"host": host}
144+
if protocol:
145+
kwargs["protocol"] = protocol
146+
if port:
147+
kwargs["port"] = port
148+
finding.unsaved_endpoints.append(Endpoint(**kwargs))
149+
150+
151+
def _parse_proto_port(value):
152+
if not value:
153+
return None, None
154+
value = value.strip()
155+
if "/" not in value:
156+
return None, None
157+
proto, _, port_str = value.partition("/")
158+
proto = proto.strip().lower() or None
159+
try:
160+
port = int(port_str.strip())
161+
except (ValueError, TypeError):
162+
port = None
163+
if port == 0:
164+
port = None
165+
return proto, port
166+
167+
168+
def _build_tags(row):
169+
tags = []
170+
if (row.get("CISA Known Exploited") or "").strip().lower() == "yes":
171+
tags.append("cisa-known-exploited")
172+
return tags

0 commit comments

Comments
 (0)