Merge pull request #1 from sysangels/dev

plugin malware-scan

Author: mkf-sysangels

malware-scan/GDPR.md

@@ -0,0 +1,389 @@
+# BunkerWeb Malware Scan Plugin
+
+**Version**: 0.8.0 | **Status**: Production Ready | **Release**: 2026-01-27
+
+Multi-layer malware scanning plugin for BunkerWeb with advanced caching, webhook notifications, and modular architecture.
+
+## ✨ Key Features
+
+- **🛡️ Multi-layer scanning**: ClamAV (local) + VirusTotal + SentinelOne (cloud)
+- **⚡ Smart caching**: SHA256-based result caching with differential TTL (clean: 5min, malicious: 24h, max 7d, configurable)
+- **📢 Webhook notifications**: Real-time alerts to Discord and Microsoft Teams
+- **🔧 Modular architecture**: 21 specialized modules for maintainability
+- **🌐 Cluster-ready**: Works correctly on manager and worker nodes
+
+## ⚡ Performance Metrics (v0.8.0)
+
+| Metric | Result | Details |
+|--------|--------|---------|
+| **First scan (cache miss)** | ~4 seconds | Full ClamAV INSTREAM scan |
+| **Repeat scan (cache hit)** | ~1 second | 75% faster - hash calculation + cache lookup |
+| **Code maintainability** | 21 modules | Specialized components for scanning, caching, APIs |
+| **Webhook delivery** | Real-time | Discord & Microsoft Teams notifications |
+| **Module initialization** | 100% | All 21 modules load successfully |
+
+## 🎯 Real-World Examples
+
+### Example 1: First Upload (Cache Miss) - Full Scan
+
+Uploading EICAR test file for the first time:
+
+```log
+[MALWARE-SCAN] file upload detected, scanning for malware
+[MALWARE-SCAN] file SHA256: 2546dcffc5ad854d4ddc64fbf056871cd5a00f2471cb7a5bfd4ac23b6e9eedad
+[CACHE] ClamAV local cache MISS for checksum: 2546dcffc5ad854d...
+
+# Full ClamAV INSTREAM scan performed
+[MALWARE-SCAN] scanning file via INSTREAM
+[MALWARE-SCAN] INSTREAM: Read chunk 1, size: 703
+[MALWARE-SCAN] INSTREAM: All chunks sent - 1 chunks, 703 bytes total
+clamd[922]: instream(127.0.0.1@34668): Eicar-Test-Signature FOUND
+
+# Detection and caching
+[MALWARE-SCAN] MALWARE DETECTED BY CLAMAV
+[MALWARE-SCAN] ClamAV signature: Eicar-Test-Signature
+[CACHE] Added ClamAV result to cache: 2546dcffc5ad854d... = Eicar-Test-Signature (TTL: 7776000s)
+
+# Webhook notification
+[WEBHOOK] Processing Discord webhook
+[WEBHOOK] Cleaned URL: https://discord.co***
+
+# Request blocked
+POST /wp-admin/async-upload.php HTTP/3.0" 403 (4 seconds total)
+```
+
+**Timeline**: 4 seconds (file buffering + ClamAV scan + webhook)
+
+---
+
+### Example 2: Second Upload (Cache Hit) - Instant Block
+
+Uploading the same EICAR file again within 90 days:
+
+```log
+[MALWARE-SCAN] file upload detected, scanning for malware
+[MALWARE-SCAN] file SHA256: 2546dcffc5ad854d4ddc64fbf056871cd5a00f2471cb7a5bfd4ac23b6e9eedad
+[CACHE] ClamAV local cache HIT for checksum: 2546dcffc5ad854d...
+
+# Cache hit - no ClamAV scan needed!
+[CACHE_HIT] ClamAV cache shows malicious - skipping ClamAV scan
+[MALWARE DETECTED BY CLAMAV (cached)]
+[MALWARE-SCAN] ClamAV signature: Eicar-Test-Signature (SHA256: 2546dcffc5ad854d...)
+
+# Webhook notification (still fires)
+[WEBHOOK] Processing Discord webhook
+[WEBHOOK] Cleaned URL: https://discord.co***
+
+# Request blocked instantly
+POST /wp-admin/async-upload.php HTTP/3.0" 403 (1 second total)
+```
+
+**Timeline**: 1 second (hash calculation + cache lookup + webhook) - **75% faster!**
+
+---
+
+### Example 3: Multi-Layer Mode with Cloud Scanners
+
+Configuration: `MALWARE_SCAN_SKIP_OTHERS_ON_CLAMAV_DETECT=no`
+
+```log
+# ClamAV detects malware
+[MALWARE-SCAN] MALWARE DETECTED BY CLAMAV
+[MALWARE-SCAN] ClamAV signature: Eicar-Test-Signature
+
+# Continue to cloud scanners for additional intelligence
+[MALWARE-SCAN] ClamAV detected malware, continuing to VirusTotal for additional analysis
+[MALWARE-SCAN] calculated SHA256: 2546dcffc5ad854d4ddc64fbf056871cd5a00f2471cb7a5bfd4ac23b6e9eedad
+
+# VirusTotal confirmation
+[MALWARE-SCAN] checking file against VirusTotal API
+[MALWARE-SCAN] file checksum found in VT cache: 0 suspicious and 61 malicious
+
+# Dual confirmation
+[MALWARE-SCAN] MALWARE DETECTED BY VIRUSTOTAL - BLOCKING REQUEST
+[MALWARE-SCAN] VirusTotal detection: 61 malicious engines confirmed
+
+# Webhook with multiple detections
+[WEBHOOK] Processing Discord webhook
+Scanner: ClamAV → Eicar-Test-Signature
+Scanner: VirusTotal → 61/61 engines detected malware
+
+POST /wp-admin/async-upload.php HTTP/3.0" 403
+```
+
+**Benefits**:
+- ✅ Dual confirmation from multiple sources
+- ✅ Rich threat intelligence
+- ✅ Lower false positive rate
+- ✅ Better for security research
+
+---
+
+### Comparison: Fast vs Full Mode
+
+| Feature | Fast Mode (default) | Full Multi-Layer Mode |
+|---------|--------------------|-----------------------|
+| **Config** | `SKIP_OTHERS_ON_CLAMAV_DETECT=yes` | `SKIP_OTHERS_ON_CLAMAV_DETECT=no` |
+| **ClamAV scan** | ✅ Always | ✅ Always |
+| **Cache check** | ✅ Before ClamAV | ✅ Before ClamAV |
+| **Cloud scanners** | Only if ClamAV clean | Always (additional intelligence) |
+| **Response time** | ~1s cached, ~4s uncached | ~1s cached, ~5s uncached |
+| **Detection sources** | Single (ClamAV) | Multiple (ClamAV + VT + TF + S1) |
+| **API quota usage** | Lower | Higher |
+| **Best for** | Production speed | Security research |
+
+## 📥 Installation
+
+### Option 1: Docker ClamAV (Recommended for New Deployments)
+
+For production environments, use the automated Docker installation script for ClamAV:
+
+```bash
+# Supports: Debian 11+, Ubuntu 20.04+, CentOS 7+, Rocky 8+, AlmaLinux 8+, Fedora 33+
+sudo ./install-docker-clamav.sh
+
+# Container runs on port 3310 with 4GB memory limit
+# Signature updates: Every 2 hours (12 checks per day)
+# Logging: JSON-file (docker logs clamav)
+```
+
+**Features:**
+- ✅ Automatic OS detection (Debian/CentOS)
+- ✅ Auto-start on reboot (systemd service)
+- ✅ 24/7 availability (concurrent database reload)
+- ✅ Comprehensive troubleshooting guides included
+
+See [install-docker-clamav.md](install-docker-clamav.md) for detailed instructions.
+
+### Option 2: Plugin Installation
+
+```bash
+# Clone the repository
+git clone https://github.com/bunkerity/bunkerweb-plugins.git
+cd bunkerweb-plugins/malware-scan
+
+# Deploy files
+sudo cp *.lua plugin.json /etc/bunkerweb/plugins/malware-scan/
+sudo chmod 644 /etc/bunkerweb/plugins/malware-scan/*
+
+# Restart BunkerWeb
+sudo systemctl restart bunkerweb
+```
+
+### Verification
+
+Check that all modules loaded successfully:
+
+```bash
+journalctl -u bunkerweb -n 50 | grep -i "malware-scan"
+```
+
+Expected output:
+```
+[MALWARE-SCAN] cache module loaded successfully
+[MALWARE-SCAN] file_ops module loaded successfully
+[MALWARE-SCAN] clamav module loaded successfully
+[MALWARE-SCAN] virustotal module loaded successfully
+[MALWARE-SCAN] sentinelone module loaded successfully
+[MALWARE-SCAN] webhook module loaded successfully
+```
+
+### ClamAV Debugging
+
+The install script automatically enables ClamAV debugging for detailed scanning logs. To manually enable or check:
+
+```bash
+# Check current ClamAV configuration
+grep -E "^(LogVerbose|Debug|LogClean)" /etc/clamav/clamd.conf
+
+# Enable debugging (if not already enabled)
+sudo sed -i 's/^LogVerbose false/LogVerbose true/' /etc/clamav/clamd.conf
+sudo sed -i 's/^Debug false/Debug true/' /etc/clamav/clamd.conf
+sudo sed -i 's/^LogClean false/LogClean true/' /etc/clamav/clamd.conf
+sudo systemctl restart clamav-daemon
+
+# View ClamAV logs
+sudo tail -f /var/log/clamav/clamav.log
+
+# View BunkerWeb malware-scan logs
+sudo journalctl -u bunkerweb -f | grep MALWARE
+```
+
+**What you'll see with debugging enabled:**
+- `LogVerbose`: Detailed information about each scan (file paths, scan methods)
+- `Debug`: Daemon-level debugging (connections, protocol details)
+- `LogClean`: Logs for clean files (not just infected ones)
+
+## ⚙️ Configuration
+
+Edit `/etc/bunkerweb/variables.env`:
+
+### Core Settings
+
+```bash
+USE_MALWARE-SCAN=yes                    # Enable malware scanning
+MALWARE_SCAN_DEBUG=no                       # Debug logging (disable in production)
+```
+
+### ClamAV (Local Scanning)
+
+```bash
+MALWARE_SCAN_CLAMAV_HOST=127.0.0.1         # ClamAV daemon host
+MALWARE_SCAN_CLAMAV_PORT=3310              # ClamAV daemon port
+MALWARE_SCAN_CLAMAV_TIMEOUT=10000          # Connection timeout (ms)
+MALWARE_SCAN_CLAMAV_MAX_SIZE=26214400      # Max file size (25MB)
+```
+
+### Multi-Layer Scanning
+
+```bash
+MALWARE_SCAN_SKIP_OTHERS_ON_CLAMAV_DETECT=yes        # Fast mode (skip cloud on ClamAV hit)
+MALWARE_SCAN_RESUME_SCAN_ON_FIRST_HIT=no             # WARNING: DOS risk! Continue scanning after first detection
+MALWARE_SCAN_SCAN_FULL_MULTIPART_BODY=no             # Skip full multipart body scan (recommended - see note below)
+MALWARE_SCAN_HASH_MAX_SIZE=67108864                  # Max file size for hash scanning (64MB)
+```
+
+**Note on SCAN_FULL_MULTIPART_BODY**:
+
+By default (`no`), the plugin skips scanning the entire multipart request body and only scans extracted individual files. This is **recommended** because:
+- HTTP multipart boundaries contain random strings that change on every upload
+- Full body SHA256 is different each time, making caching completely ineffective
+- Individual file extraction provides consistent SHA256s for reliable caching
+- Wastes ClamAV resources scanning the same files repeatedly
+
+Only enable (`yes`) if you suspect malware might be split across multipart boundaries (extremely rare).
+
+**Note on RESUME_SCAN_ON_FIRST_HIT**:
+
+The plugin performs **dual scanning** for comprehensive malware detection:
+1. **Full-body scan** (optional): Scans entire multipart upload - disabled by default (see SCAN_FULL_MULTIPART_BODY above)
+2. **Per-file scan** (when resume enabled): Parses multipart data and scans each uploaded file individually
+
+When `RESUME_SCAN_ON_FIRST_HIT=yes`:
+- All scanners complete even after detection (comprehensive threat intelligence)
+- Continues to next file after detecting malware in one file
+- Aggregates detections from whole-body + all individual files before blocking
+
+When `RESUME_SCAN_ON_FIRST_HIT=no` (default):
+- Blocks immediately on first detection (fast response, minimal resources)
+
+**DOS Risk**: Attackers can force expensive operations (multiple file scans, API calls) even after detection. Only enable for security research or when you need complete visibility into multi-file attacks.
+
+### VirusTotal (70+ AV Engines)
+
+```bash
+MALWARE_SCAN_API_USE_VIRUSTOTAL=yes                            # Enable VirusTotal
+MALWARE_SCAN_API_VIRUSTOTAL_KEY=your_api_key               # API key (required)
+MALWARE_SCAN_API_VIRUSTOTAL_SUSPICIOUS=5                   # Suspicious threshold
+MALWARE_SCAN_API_VIRUSTOTAL_MALICIOUS=3                    # Malicious threshold
+MALWARE_SCAN_API_VIRUSTOTAL_UPLOAD_UNKNOWN=yes             # Upload unknown malware
+MALWARE_SCAN_API_VIRUSTOTAL_AUTO_VOTE=yes                  # Auto-vote on uploads
+MALWARE_SCAN_API_VIRUSTOTAL_MAX_UPLOAD_SIZE=33554432       # Max upload size (32MB)
+```
+
+### SentinelOne (Enterprise)
+
+```bash
+MALWARE_SCAN_API_USE_SENTINELONE=yes                         # Enable SentinelOne
+MALWARE_SCAN_API_SENTINELONE_URL=https://...             # Console URL
+MALWARE_SCAN_API_SENTINELONE_TOKEN=your_token            # API token
+```
+
+### Webhooks (Discord & Teams)
+
+```bash
+MALWARE_SCAN_DISCORD_WEBHOOK_URL=https://discord.com/api/webhooks/...
+MALWARE_SCAN_TEAMS_WEBHOOK_URL=https://outlook.office.com/webhook/...
+```
+
+### False Positive Handling
+
+```bash
+MALWARE_SCAN_ALLOWLIST_HASHES=sha256_hash1,sha256_hash2  # Comma-separated hashes
+MALWARE_SCAN_ALLOWLIST_FILE=/path/to/allowlist.txt       # File with hashes (one per line)
+```
+
+### Cleanup
+
+```bash
+MALWARE_SCAN_CLEANUP_FILES=yes              # Delete temp files after scan
+MALWARE_SCAN_CLEANUP_ORPHANED=yes           # Clean up old orphaned files
+MALWARE_SCAN_CLEANUP_AGE=3600               # Max age for orphaned files (seconds)
+```
+
+## 🧪 Testing
+
+### Test with EICAR File
+
+```bash
+# Download EICAR standard test file
+curl -o eicar.txt https://secure.eicar.org/eicar.com.txt
+
+# Upload through your web application
+# Example: WordPress media uploader
+```
+
+Expected result:
+```
+HTTP 403 Forbidden
+malware detected by ClamAV: Eicar-Test-Signature
+```
+
+### Test Caching
+
+1. Upload EICAR file - should take ~4 seconds (full scan)
+2. Upload same file again - should take ~1 second (cache hit)
+3. Check logs for `[CACHE_HIT]` messages
+
+## 📋 API Rate Limits
+
+### VirusTotal Public API
+- **Rate limits**: 4 requests/minute, 500 requests/day
+- **Free tier**: https://www.virustotal.com/gui/join-us
+- **Premium**: Unlimited requests
+
+### SentinelOne API
+- **Rate limits**: 10 req/sec (50 burst)
+- **License**: Requires SentinelOne subscription
+
+## 📚 Documentation
+
+- **[install-docker-clamav.md](install-docker-clamav.md)** - **NEW!** Automated Docker ClamAV setup (Debian & CentOS)
+- **[install-docker-clamav.sh](install-docker-clamav.sh)** - Deployment script with OS auto-detection
+- **[install.md](install.md)** - Detailed installation guide
+- **[TECHNICAL.md](TECHNICAL.md)** - Architecture, security features, development details
+- **[GDPR.md](GDPR.md)** - GDPR compliance guide with privacy policy template
+- **[SECURITY.md](SECURITY.md)** - Vulnerability reporting and security policy
+- **[all-features-enabled.md](all-features-enabled.md)** - Complete configuration with all scanners
+- **[variables.env.example](variables.env.example)** - Complete configuration reference
+- **[ALLOWLIST.md](ALLOWLIST.md)** - False positive handling with SHA256 allowlists
+- **[GRANULAR_CACHE_CLEANUP.md](GRANULAR_CACHE_CLEANUP.md)** - Per-scanner cache cleanup configuration
+
+## 🔒 Security
+
+### Security Best Practices
+
+- Keep ClamAV signatures updated: `sudo freshclam`
+- Use default settings: `BLOCK_ON_FILENAME_INJECTION_ATTACK=yes`
+- Avoid `RESUME_SCAN_ON_FIRST_HIT=yes` in production (DOS risk)
+- Store API keys in environment variables, never commit to git
+- Review logs regularly for attack patterns
+
+### Reporting Vulnerabilities
+
+See [SECURITY.md](SECURITY.md) for vulnerability disclosure policy and security contacts.
+
+## 💬 Support
+
+- **Documentation**: [docs.bunkerweb.io](https://docs.bunkerweb.io/)
+- **Issues**: Check logs first: `journalctl -u bunkerweb`
+- **ClamAV**: [docs.clamav.net](https://docs.clamav.net/)
+- **Security**: [SECURITY.md](SECURITY.md)
+
+## 📜 License
+
+This plugin is part of the BunkerWeb project.
+
+---
+
+**User-Agent**: API requests include version: `bunkerweb - https://github.com/bunkerity/bunkerweb - malware-scan module v0.8.0`