DIY SIEM

DIY SIEM ingests Windows Security Event logs, parses the XML payloads, and raises alerts through pluggable detectors such as brute-force, log clearing, and remote desktop attack detection.

Features

• Event subscription to the Windows Security log via pywin32.
• Parallel XML parsing and detector evaluation through worker threads.
• YAML-driven detector configuration (config/*.yaml).
• Extensible detector registry for custom analytics.

Prerequisites

• Windows Server with administrative privileges.
• Python 3.13+

Quick Start

1. Clone & Update

   git clone https://github.com/your-org/diy-SIEM.git
   cd diy-SIEM
   git pull

2. Create & Activate Virtual Environment

   py -3 -m venv .venv
   .venv\Scripts\Activate.ps1

3. Install Dependencies

   python -m pip install -r requirements.txt

4. Run the Orchestrator

   python orchestrator.py

Firewall watcher parser

I extracted the firewall event enrichment logic into utils/firewall_watcher.py. This module exposes:

• build_firewall_context(evt, lookup_fn, process_cache, build_ancestry_fn) — builds the incident context dict previously produced inline in watcher._handle_firewall_event.
  • evt should be a parsed event dict (the same shape returned by utils.xml_parser.parse_event_xml).
  • lookup_fn(pid) is called to resolve live process metadata (WMI lookup helper in watcher is _lookup_process_metadata).
  • process_cache is the in-memory 4688 cache (used to correlate short-lived CLI invocations like netsh).
  • build_ancestry_fn(start_pid, max_depth) is used to enrich initiator chains (watcher's _build_process_ancestry).

The module also exposes two helpers useful for testing:

• _find_recent_netsh(process_cache) — returns a recent netsh cache entry (or None).
• _resolve_parent_for_netsh(netsh_entry, lookup_fn, process_cache) — attempts to resolve the parent process for a cached netsh entry.

Example (used by watcher.py):

from utils.firewall_watcher import build_firewall_context

# inside the watcher callback
context = build_firewall_context(evt, _lookup_process_metadata, g_process_cache, _build_process_ancestry)
_report_incident(context, event_type="firewall")

Unit tests for the parser are in tests/test_firewall_watcher.py.

Note: Run Visual Studio Code as Administrator so the process can access the Security Event log.

Configuration

Place YAML files in config/ with at least:

name: User Credential Brute Forcing
start_events:
  - 4624
step_events:
  - 4625
end_events:
  - 4624
thresholds:
  attempts: 5
  window_minutes: 10

Each name must correspond to a key in DETECTOR_MAP inside orchestrator.py. Add new detectors by implementing the interface in detectors/ and updating the map.

Architecture Overview

• orchestrator.py: wires configuration loading, detector instantiation, event subscription, and worker threads.
• subscribers/: handles event callbacks and maintains the in-memory queue.
• utils/xml_parser.py: converts raw XML events into structured dictionaries.
• detectors/: contains individual analytics components.

Development Tips

• Keep config/ YAML minimal and validated; malformed entries are skipped.
• Use unit tests for detectors to safeguard detection logic.
• For troubleshooting, enable verbose logging in detectors and parsers.

Roadmap Ideas

• Add centralized alert delivery (webhooks, email, SIEM forwarders).
• Persist alerts to disk or a database.
• Expand detector library (privilege escalation, lateral movement, etc.).

License

Specify the appropriate license here (e.g., MIT) and include the text in LICENSE.