rewamp htmlreport and bootstrap5 (kevoreilly#2784)

Author: doomedraven

SKILLS.md

@@ -0,0 +1,112 @@
+# CAPE Sandbox Developer Skills & Architecture Guide
+
+This document outlines the architectural structure, core concepts, and development patterns for the CAPE Sandbox (v2). It serves as a guide for extending functionality, debugging, and maintaining the codebase.
+
+## 1. Project Overview
+CAPE (Config And Payload Extraction) is a malware analysis sandbox derived from Cuckoo Sandbox. It focuses on automated malware analysis with a specific emphasis on extracting payloads and configuration from malware.
+
+**Core Tech Stack:**
+- **Language:** Python 3
+- **Web Framework:** Django
+- **Database:** PostgreSQL (SQLAlchemy) for task management, MongoDB/Elasticsearch for results storage.
+- **Virtualization:** KVM/QEMU (preferred), VirtualBox, VMWare.
+- **Frontend:** HTML5, Bootstrap, Jinja2 Templates.
+
+## 2. Directory Structure Key
+| Directory | Purpose |
+| :--- | :--- |
+| `agent/` | Python script (`agent.py`) running *inside* the Guest VM to handle communication. |
+| `analyzer/` | Core analysis components running *inside* the Guest VM (monitor, analyzers). |
+| `conf/` | Configuration files (`cuckoo.conf`, `reporting.conf`, `web.conf`, etc.). |
+| `data/` | Static assets, yara rules, monitor binaries, and HTML templates (`data/html`). |
+| `lib/cuckoo/` | Core logic (Scheduler, Database, Guest Manager, Result Processor). |
+| `modules/` | Pluggable components (Signatures, Processing, Reporting, Auxiliary). |
+| `web/` | Django-based web interface (Views, URLs, Templates). |
+| `utils/` | Standalone CLI utilities (`process.py`, `cleaners.py`, `rooter.py`). |
+
+## 3. Core Workflows
+
+### A. The Analysis Lifecycle
+1.  **Submission:** User submits file/URL via WebUI (`web/submission/`) or API (`web/api/`).
+2.  **Scheduling:** Task is added to SQL DB. `lib/cuckoo/core/scheduler.py` picks it up.
+3.  **Execution:**
+    *   VM is restored/started.
+    *   `analyzer` is uploaded to VM.
+    *   Sample is injected/executed.
+    *   Behavior is monitored via API hooking (CAPE Monitor).
+4.  **Result Collection:** Logs, PCAP, and dropped files are transferred back to Host.
+5.  **Processing:** `modules/processing/` parses raw logs into a structured dictionary.
+6.  **Signatures:** `modules/signatures/` runs logic against the processed data.
+7.  **Reporting:** `modules/reporting/` exports data (JSON, HTML, MongoDB, MAEC).
+
+### B. Web Interface Architecture
+The Web UI is split into two distinct rendering logic paths:
+1.  **Django Views (`web/analysis/views.py`):** Handles URL routing, authentication, and context generation. It fetches data from MongoDB/Elasticsearch.
+2.  **Jinja2 Templates:**
+    *   **Web Templates (`web/templates/`):** Standard Django templates for the UI.
+    *   **Report Templates (`data/html/`):** Standalone Jinja2 templates used by the `reporthtml` module to generate static HTML reports. *Note: Changes here affect the downloadable HTML report, not necessarily the Web UI.*
+
+## 4. Development Guides
+
+### How to Add a Detection Signature
+Signatures live in `modules/signatures/`.
+```python
+from lib.cuckoo.common.abstracts import Signature
+
+class MyMalware(Signature):
+    name = "my_malware_behavior"
+    description = "Detects specific bad behavior"
+    severity = 3
+    categories = ["trojan"]
+    authors = ["You"]
+
+    def on_call(self, call, process):
+        # Inspect individual API calls
+        if call["api"] == "CreateFileW" and "evil.exe" in call["arguments"]["filepath"]:
+            return True
+```
+
+### How to Add a Processing Module
+Processing modules (`modules/processing/`) run after analysis to extract specific data (e.g., Static analysis of a file).
+```python
+from lib.cuckoo.common.abstracts import Processing
+
+class MyExtractor(Processing):
+    def run(self):
+        self.key = "my_data" # Key in the final report JSON
+        result = {}
+        # ... logic ...
+        return result
+```
+
+### How to Modify the Web Report
+1.  **Locate the Template:** Look in `web/templates/analysis/`.
+    *   `overview/index.html`: Main dashboard.
+    *   `overview/_info.html`: General details.
+    *   `overview/_summary.html`: Behavioral summary.
+2.  **Edit:** Use Django template language (`{% if %}`, `{{ variable }}`).
+3.  **Context:** Data is usually passed as `analysis` object. Access fields like `analysis.info.id`, `analysis.network`, `analysis.behavior`.
+
+## 5. Troubleshooting & Debugging
+
+### Common Issues
+*   **"Waiting for container":** Usually a network configuration issue in `conf/cuckoo.conf` or `conf/auxiliary.conf`.
+*   **Report Empty:** Check `reporting.conf`. If using MongoDB, ensure `mongodb` is enabled.
+*   **Template Errors:** Use `{% if variable %}` guards aggressively. Missing keys in MongoDB documents cause Jinja2 crashes.
+
+### Important Commands
+*   `poetry run python cuckoo.py -d`: Run CAPE in debug mode (verbose logs).
+*   `poetry run python utils/process.py -r <task_id>`: Re-run processing and reporting for a specific task without restarting the VM.
+*   `poetry run python utils/cleaners.py --clean`: Wipe all tasks and reset the DB.
+
+### Database Querying (MongoDB)
+CAPE stores unstructured analysis results in the `analysis` collection.
+```bash
+mongo cuckoo
+db.analysis.find({"info.id": 123}, {"behavior.summary": 1}).pretty()
+```
+
+## 6. Best Practices
+1.  **Conditionally Render:** Always check if a dictionary key exists in templates before rendering to avoid UI breaks on different analysis types (Static vs Dynamic).
+2.  **Keep Views Light:** Perform heavy data crunching in `modules/processing`, not in Django views.
+3.  **Modular CSS/JS:** Keep custom styles in `web/static/` rather than inline in templates when possible.

changelog.md

@@ -1,3 +1,15 @@
+### [16.01.2026] CAPE v2.5
+* Bootstrap 5 upgrade and some visual WEBGUI rewamp. Some improvements still might come soon!
+* htmlreport - rewamp!
+* cape2.sh - Libvirt + YARA python libraries install without external scripts.
+* Datatime UTC normalization on tasks/VMs changes.
+* Added check on startup for enable firewall.
+* Volatility3 - more modules added. Test them and let us know if you have any issue.
+* Filedescripts leaks fixed.
+* Stucked VM monitoring and kill. [PR](https://github.com/kevoreilly/CAPEv2/pull/2809)
+
+PS no changes required to CAPA library to support CAPE v2.5 ;)
+
 ### [02.01.2026]
 * CAPE installer:
     *  now support custom destination folder env variable:

conf/default/integrations.conf.default

@@ -14,11 +14,6 @@ hwp = no
 # Number of workers for pool to run them in parallel
 max_workers = 6
 
-[mandiant_intel]
-enabled = no
-api_access =
-api_secret =
-
 # Create your apikey: https://threatfox.abuse.ch/api/#auth_key
 # MalwareBazaar uses this key too
 [abusech]

conf/default/memory.conf.default

@@ -10,80 +10,76 @@ delete_memdump = no
 # Delete memory dump in the event of a volatility exception
 delete_memdump_on_exception = no
 
+# Masks. Data that should not be logged
+# Just get this information from your plain VM Snapshot (without running malware)
+# This will filter out unwanted information in the logs
+[mask]
+enabled = no
+pid_generic =
+
+
 # List of available modules
 # enabled: enable this module
 # filter: use filters to remove benign system data from the logs
 # Filters are defined in the mask section at below
 
 # Scans for hidden/injected code and dlls
-# http://code.google.com/p/volatility/wiki/CommandReferenceMal23#malfind
 [malfind]
 enabled = no
 filter = on
 
 # Lists official processes. Does not detect hidden processes
-# https://github.com/volatilityfoundation/volatility/wiki/Command-Reference#pslist
 [pslist]
 enabled = no
 filter = off
 
 # Process listing in tree form. Does not detect hidden processes (Don't work currently in CAPE)
-# https://github.com/volatilityfoundation/volatility/wiki/Command-Reference#pstree
 [pstree]
 enabled = no
 filter = off
 
 # Lists hidden processes. Enumerate processes in the Kernel memory using pool tag scanning _POOL_HEADER
-# https://github.com/volatilityfoundation/volatility/wiki/Command-Reference#psscan
 [psscan]
 enabled = no
 filter = off
 
 
 # Show callbacks
-# http://code.google.com/p/volatility/wiki/CommandReferenceMal23#callbacks
 [callbacks]
 enabled = no
 filter = off
 
 # Show sids
-# http://code.google.com/p/volatility/wiki/CommandReference23#getsids
 [getsids]
 enabled = no
 filter = off
 
 # Show privileges
-# http://code.google.com/p/volatility/wiki/CommandReference23#privs
 [privs]
 enabled = no
 filter = off
 
 # Display processes' loaded DLLs- Does not display hidden DLLs
-# http://code.google.com/p/volatility/wiki/CommandReference23#dlllist
 [dlllist]
 enabled = no
 filter = on
 
 # List open handles of processes
-# http://code.google.com/p/volatility/wiki/CommandReference23#handles
 [handles]
 enabled = no
 filter = on
 
 # Scan for Mutexes (whole system)
-# http://code.google.com/p/volatility/wiki/CommandReference23#mutantscan
 [mutantscan]
 enabled = no
 filter = on
 
 # Scan for services
-# http://code.google.com/p/volatility/wiki/CommandReferenceMal23#svcscan
 [svcscan]
 enabled = no
 filter = on
 
 # Scan for kernel drivers (includes hidden, unloaded)
-# http://code.google.com/p/volatility/wiki/CommandReference23#modscan
 [modscan]
 enabled = no
 filter = on
@@ -106,9 +102,144 @@ filter = off
 enabled = no
 filter = off
 
-# Masks. Data that should not be logged
-# Just get this information from your plain VM Snapshot (without running malware)
-# This will filter out unwanted information in the logs
-[mask]
+# Not tested module below
+
+[info]
 enabled = no
-pid_generic =
+filter = off
+
+[psxview]
+enabled = no
+filter = off
+
+[ldrmodules]
+enabled = no
+filter = off
+
+[cmdline]
+enabled = no
+filter = off
+
+[envars]
+enabled = no
+filter = off
+
+[modules]
+enabled = no
+filter = off
+
+[driverscan]
+enabled = no
+filter = off
+
+[driverirp]
+enabled = no
+filter = off
+
+[verinfo]
+enabled = no
+filter = off
+
+[filescan]
+enabled = no
+filter = off
+
+[vadinfo]
+enabled = no
+filter = off
+
+[timers]
+enabled = no
+filter = off
+
+[hivelist]
+enabled = no
+filter = off
+
+[hashdump]
+enabled = no
+filter = off
+
+[lsadump]
+enabled = no
+filter = off
+
+[cachedump]
+enabled = no
+filter = off
+
+[symlinkscan]
+enabled = no
+filter = off
+
+[thrdscan]
+enabled = no
+filter = off
+
+[hollowprocesses]
+enabled = no
+filter = off
+
+[processghosting]
+enabled = no
+filter = off
+
+[suspiciousthreads]
+enabled = no
+filter = off
+
+[devicetree]
+enabled = no
+filter = off
+
+[consoles]
+enabled = no
+filter = off
+
+[cmdscan]
+enabled = no
+filter = off
+
+[amcache]
+enabled = no
+filter = off
+
+[shimcache]
+enabled = no
+filter = off
+
+[userassist]
+enabled = no
+filter = off
+
+[unloadedmodules]
+enabled = no
+filter = off
+
+[iat]
+enabled = no
+filter = off
+
+[skeletonkey]
+enabled = no
+filter = off
+
+[unhookedsyscalls]
+enabled = no
+filter = off
+
+[etwpatch]
+enabled = no
+filter = off
+
+[mftscan]
+enabled = no
+filter = off
+
+[svclist]
+enabled = no
+filter = off
+
+[svcdiff]
+enabled = no
+filter = off