Skip to content

false-positives.md

Latest commit

 

History

History
391 lines (297 loc) · 13.1 KB

false-positives.md

File metadata and controls

391 lines (297 loc) · 13.1 KB
Raw

🔧 False Positive Generation

Complete guide to generating realistic false positive scenarios for detection rule tuning and SOC training.

📋 Table of Contents

Overview

The false positive generation system creates realistic scenarios that trigger security alerts but represent legitimate business activities. This is essential for:

  • Detection Rule Tuning: Identifying overly broad or sensitive rules
  • SOC Training: Teaching analysts to distinguish between threats and benign activities
  • Alert Fatigue Simulation: Testing analyst workflows under realistic conditions
  • System Performance Testing: Validating alert processing capabilities

Quick Start

Basic False Positive Generation

# Generate alerts with 20% false positives
yarn start generate-alerts --count 100 --false-positive-rate 0.2

# Generate high false positive scenario for training
yarn start generate-alerts --count 50 --false-positive-rate 0.6

# Generate themed false positives
yarn start generate-alerts --theme corporate --count 75 --false-positive-rate 0.3

Campaign-Specific False Positives

# APT campaign with realistic false positive rate
yarn start generate-campaign apt --realistic --false-positive-rate 0.1

# Ransomware with higher false positive rate for training
yarn start generate-campaign ransomware --false-positive-rate 0.4 --count 100

False Positive Types

1. Administrative Activities

Legitimate system administration that triggers security alerts:

Examples:

  • Scheduled Tasks: Automated backup scripts running at odd hours
  • Software Updates: Legitimate software installation and updates
  • System Maintenance: Disk cleanup, registry maintenance, service restarts
  • User Management: Account creation, password resets, permission changes

Generated Scenarios:

{
  "event.category": "process",
  "process.name": "powershell.exe",
  "process.command_line": "PowerShell.exe -ExecutionPolicy Bypass -File C:\\Scripts\\BackupScript.ps1",
  "user.name": "backup_service",
  "host.name": "backup-server-01",
  "tags": ["false_positive", "administrative"]
}

2. Business Applications

Legitimate business software with suspicious characteristics:

Examples:

  • Development Tools: Compilers, debuggers, scripting environments
  • Remote Access: VPN connections, remote desktop, SSH tunnels
  • Data Processing: ETL jobs, database maintenance, file synchronization
  • Monitoring Tools: System monitoring, network scanning, performance analysis

3. User Behavior Anomalies

Normal user activities that appear suspicious:

Examples:

  • Travel Patterns: Legitimate travel causing geographic anomalies
  • Schedule Changes: Working unusual hours for project deadlines
  • New Responsibilities: Accessing new systems due to role changes
  • Training Activities: Learning new tools or accessing training environments

4. Network Activities

Legitimate network traffic that triggers alerts:

Examples:

  • Cloud Services: API calls to cloud providers and SaaS applications
  • Content Delivery: CDN traffic, software downloads, update checks
  • Inter-Office Communication: Site-to-site VPN, branch office connections
  • Partner Integrations: B2B data exchanges, vendor portal access

Configuration Options

False Positive Rate Control

# Low false positive rate (production-like)
--false-positive-rate 0.1    # 10% false positives

# Medium false positive rate (testing)
--false-positive-rate 0.3    # 30% false positives

# High false positive rate (training)
--false-positive-rate 0.6    # 60% false positives

Category Targeting

# Focus on specific false positive categories
yarn start generate-alerts --count 50 --false-positive-categories administrative,business_apps

# Generate comprehensive false positive scenarios
yarn start generate-alerts --count 100 --false-positive-categories all

Time-Based Patterns

# After-hours administrative activities
yarn start generate-alerts --count 30 --false-positive-rate 0.4 --time-pattern off_hours

# Business hours false positives
yarn start generate-alerts --count 80 --false-positive-rate 0.2 --time-pattern business_hours

SOC Training Scenarios

Beginner Training

High false positive rates to teach basic triage skills:

# Clear distinction between real threats and false positives
yarn start generate-alerts --count 40 --false-positive-rate 0.7 --difficulty beginner

# Focus on common false positive types
yarn start generate-alerts --count 30 --false-positive-categories administrative --theme corporate

Training Objectives:

  • Identify obvious false positives
  • Understand business context
  • Practice initial triage procedures
  • Learn escalation criteria

Intermediate Training

Moderate false positive rates with nuanced scenarios:

# Mixed scenarios requiring deeper analysis
yarn start generate-alerts --count 60 --false-positive-rate 0.4 --difficulty intermediate

# Campaign-based training with false positives
yarn start generate-campaign apt --realistic --false-positive-rate 0.3 --count 50

Training Objectives:

  • Analyze ambiguous alerts
  • Correlate multiple events
  • Distinguish sophisticated attacks from complex legitimate activities
  • Practice investigation workflows

Advanced Training

Low false positive rates simulating production environments:

# Production-like false positive rates
yarn start generate-alerts --count 100 --false-positive-rate 0.1 --difficulty advanced

# Complex campaign scenarios
yarn start generate-campaign insider --realistic --false-positive-rate 0.15 --count 75

Training Objectives:

  • Handle high-volume alert environments
  • Identify subtle false positive patterns
  • Optimize investigation efficiency
  • Practice advanced correlation techniques

Rule Tuning Workflows

Detection Rule Testing

Use false positives to validate and tune detection rules:

# Generate test data for rule tuning
yarn start generate-alerts --count 200 --false-positive-rate 0.25 --rule-testing

# Create baseline for false positive analysis
yarn start generate-logs --count 1000 --false-positive-rate 0.2 --baseline

Alert Threshold Optimization

Test different alert thresholds with varying false positive rates:

# High sensitivity testing
yarn start generate-alerts --count 100 --false-positive-rate 0.5 --sensitivity high

# Balanced sensitivity testing
yarn start generate-alerts --count 100 --false-positive-rate 0.3 --sensitivity medium

# Low sensitivity testing
yarn start generate-alerts --count 100 --false-positive-rate 0.1 --sensitivity low

Exception Rule Development

Generate data to develop alert exceptions and filters:

# Administrative activity analysis
yarn start generate-alerts --count 50 --false-positive-categories administrative --exception-analysis

# Business application profiling
yarn start generate-alerts --count 75 --false-positive-categories business_apps --profiling

Integration with Other Features

MITRE ATT&CK False Positives

Generate false positives that mimic MITRE techniques:

# False positives resembling MITRE techniques
yarn start generate-alerts --count 40 --mitre --false-positive-rate 0.4

# Campaign false positives with MITRE context
yarn start generate-campaign apt --mitre --false-positive-rate 0.3 --count 60

AI-Enhanced False Positive Analysis

Use AI to generate contextual false positive scenarios:

# AI-generated false positive contexts
yarn start generate-alerts --count 50 --ai --false-positive-rate 0.3

# Detailed false positive explanations
yarn start generate-alerts --count 30 --ai --false-positive-categories administrative --explain

Multi-Environment False Positives

Generate false positives across different environments:

# Development environment false positives
yarn start generate-alerts --space development --false-positive-rate 0.5 --count 40

# Production environment false positives
yarn start generate-alerts --space production --false-positive-rate 0.1 --count 100

Use Cases

1. SOC Analyst Training

Create comprehensive training scenarios:

# Progressive training curriculum
yarn start generate-alerts --count 30 --false-positive-rate 0.7 --level 1
yarn start generate-alerts --count 50 --false-positive-rate 0.4 --level 2
yarn start generate-alerts --count 75 --false-positive-rate 0.2 --level 3

2. Detection Rule Validation

Test detection rules against realistic false positive scenarios:

# Rule effectiveness testing
yarn start generate-alerts --count 200 --false-positive-rate 0.3 --rule-validation

# Coverage analysis
yarn start generate-alerts --count 150 --false-positive-categories all --coverage-test

3. Performance Testing

Test SOC workflows under realistic alert loads:

# High-volume false positive testing
yarn start generate-alerts --count 500 --false-positive-rate 0.4 --performance-test

# Alert fatigue simulation
yarn start generate-alerts --count 1000 --false-positive-rate 0.6 --fatigue-test

4. Business Context Training

Teach analysts about legitimate business activities:

# Business application training
yarn start generate-alerts --count 40 --false-positive-categories business_apps --business-context

# Administrative process training
yarn start generate-alerts --count 30 --false-positive-categories administrative --process-training

Best Practices

False Positive Design

  1. Realistic Scenarios: Base false positives on actual business activities
  2. Appropriate Timing: Consider business hours and operational schedules
  3. Contextual Accuracy: Ensure false positives match organizational context
  4. Progressive Difficulty: Start simple, increase complexity gradually
  5. Clear Learning Objectives: Define what analysts should learn from each scenario

Training Implementation

  1. Baseline Assessment: Test analyst skills before training
  2. Structured Progression: Move from obvious to subtle false positives
  3. Business Context: Provide organizational background and context
  4. Feedback Loops: Provide immediate feedback on triage decisions
  5. Performance Metrics: Track improvement in false positive identification

Rule Tuning Strategy

  1. Systematic Testing: Test rules against comprehensive false positive sets
  2. Exception Development: Create specific exceptions for known false positives
  3. Threshold Optimization: Adjust sensitivity based on false positive analysis
  4. Continuous Monitoring: Regularly update false positive scenarios
  5. Documentation: Maintain records of false positive patterns and resolutions

Troubleshooting

Common Issues

Unrealistic False Positives

Issue: Generated false positives don't match business environment Solutions:

  • Use appropriate themes (corporate, technical)
  • Customize false positive categories
  • Adjust time patterns to match business hours
  • Review and update false positive templates

Training Effectiveness

Issue: Analysts struggle to identify false positives Solutions:

  • Start with higher false positive rates
  • Provide clearer business context
  • Use progressive difficulty levels
  • Add detailed explanations and feedback

Rule Tuning Challenges

Issue: Difficulty balancing sensitivity and false positive rates Solutions:

  • Test multiple false positive rates
  • Analyze false positive patterns systematically
  • Develop granular exception rules
  • Monitor production false positive rates

Performance Optimization

  • Batch Processing: Generate false positives in appropriate batch sizes
  • Resource Management: Monitor system resources during generation
  • Index Impact: Consider impact on Elasticsearch indices
  • Storage Planning: Plan for additional storage requirements

Advanced Configuration

Custom False Positive Templates

Define organization-specific false positive scenarios:

{
  "false_positive_templates": {
    "organizational": {
      "administrative_tools": ["backup_software.exe", "monitoring_agent.exe"],
      "business_applications": ["erp_system.exe", "crm_client.exe"],
      "development_tools": ["visual_studio.exe", "git.exe"],
      "remote_access": ["vpn_client.exe", "rdp_client.exe"]
    }
  }
}

Adaptive False Positive Generation

Automatically adjust false positive rates based on training effectiveness:

# Adaptive training mode
yarn start generate-alerts --count 50 --adaptive-false-positives --training-mode

# Performance-based adjustment
yarn start generate-alerts --count 75 --false-positive-rate adaptive --analyst-performance-tracking

Ready to improve detection accuracy and analyst skills? Start with yarn start generate-alerts --count 50 --false-positive-rate 0.3 for realistic false positive training scenarios!