Add AI advisory agents and architecture/dataflow analysis

Introduces AI-powered advisory agents for architecture, dataflow, and secret confidence analysis, with opt-in consent and redaction for privacy. Adds new modules for AI engine, agent protocols, and feature-based architecture and dataflow scanning. Updates CLI to support AI features, integrates AI findings into scan results, and improves documentation on ethics and usage. Removes test_secret.py and adds cross-file test cases.

Author: Tuntii

AI_ETHICS.md

@@ -0,0 +1,28 @@
+# 🛡 AI Ethics & Privacy in OpenAuditKit
+
+OpenAuditKit integrates AI capabilities with a "Safety-First" approach. We believe security tools should not compromise the privacy of the code they analyze.
+
+## 1. Opt-In by Default
+AI features are **strictly opt-in**.
+- You must explicitly pass the `--ai` flag to enable them.
+- On the first run, you will be asked to grant consent interactively.
+- For CI/CD, you must explicitly enable consent (e.g., via `openaudit consent --grant`).
+
+## 2. Data Redaction
+Before any code snippet is sent to an LLM (Large Language Model):
+- **Secrets are Redacted**: We use our static analysis engine to detect and mask secrets (API keys, passwords, tokens) with `[REDACTED]`.
+- **Anonymization**: We aim to strip PII where possible, though code context is preserved for analysis.
+
+## 3. Advisory Nature
+AI is non-deterministic.
+- All AI-generated findings are tagged as **Advisory**.
+- They should be reviewed by a human.
+- They do not block builds by default unless configured otherwise.
+
+## 4. Local vs External
+- We support local LLMs (e.g., via Ollama) for users who want zero data egress.
+- External providers (e.g., OpenAI, Anthropic) are optional and require your own API keys. We do not proxy your code through our servers.
+
+## 5. Transparency
+- We explain *why* an AI finding was generated.
+- We show the prompt context (in debug mode) so you know exactly what was sent.

README.md

@@ -7,7 +7,7 @@ OpenAuditKit is an open-source CLI security audit tool designed to scan your cod
 - **Config Scanning**: Identifies misconfigurations in deployment files (e.g., .env, Dockerfile).
 - **Secure**: Secrets are masked in outputs; offline-first design.
 - **Backend Ready**: Feature-based architecture with Pydantic models for easy integration into dashboards or APIs.
-- **Customizable**: Add your own rules! See [Rule Documentation](rules/README.md).
+- **Customizable**: Add your own rules! See [Rule Documentation](openopenaudit/rules/README.md).
 
 ## 🛡️ Why OpenAuditKit?
 
@@ -33,7 +33,7 @@ Often, security tools are either too simple (grep) or too complex (enterprise SA
 pip install openaudit
 
 # Or from source
-git clone https://github.com/StartUp-Agency/OpenAuditKit.git
+git clone https://github.com/neuralforgeone/OpenAuditKit.git
 cd OpenAuditKit
 pip install .
 ```

build/lib/openaudit/__init__.py

@@ -0,0 +1 @@
+__version__ = "0.1.0"

build/lib/openaudit/main.py

@@ -0,0 +1,7 @@
+from openaudit.interface.cli.app import app
+
+def main():
+    app()
+
+if __name__ == "__main__":
+    main()

build/lib/openaudit/rules/config.yaml

@@ -0,0 +1,67 @@
+rules:
+  # .env Rules
+  - id: "CONF_DEBUG_ENABLED"
+    description: "Debug mode enabled in configuration"
+    regex: "(?i)^\\s*DEBUG\\s*=\\s*(true|1|yes)"
+    severity: "high"
+    confidence: "high"
+    category: "config"
+    remediation: "Set DEBUG=False in production environments."
+
+  - id: "CONF_DATABASE_URL_UNENCRYPTED"
+    description: "Plaintext database URL detected"
+    regex: "^\\s*DATABASE_URL\\s*=\\s*(postgres|mysql|mongodb)://"
+    severity: "high"
+    confidence: "high"
+    category: "config"
+    remediation: "Use encrypted secrets management or mask credentials."
+
+  - id: "CONF_ENV_DEV_IN_PROD"
+    description: "Development environment setting detected"
+    regex: "(?i)^\\s*ENV\\s*=\\s*(dev|development)"
+    severity: "medium"
+    confidence: "high"
+    category: "config"
+    remediation: "Ensure this is not a production environment."
+
+  # Dockerfile Rules
+  - id: "DOCKER_USER_ROOT"
+    description: "Container running as root"
+    regex: "^\\s*USER\\s+root"
+    severity: "high"
+    confidence: "high"
+    category: "infrastructure"
+    remediation: "Create and switch to a non-root user."
+
+  - id: "DOCKER_EXPOSE_ALL"
+    description: "Exposing service on all interfaces (0.0.0.0)"
+    regex: "^\\s*EXPOSE\\s+.*0\\.0\\.0\\.0"
+    severity: "medium"
+    confidence: "high"
+    category: "infrastructure"
+    remediation: "Bind to specific interfaces if possible."
+
+  - id: "DOCKER_ADD_COPY_ALL"
+    description: "Broad COPY instruction (COPY . /)"
+    regex: "^\\s*COPY\\s+\\.\\s+/"
+    severity: "low"
+    confidence: "medium"
+    category: "infrastructure"
+    remediation: "Use .dockerignore and copy only necessary files."
+
+  # Docker Compose Rules (Regex approximation for simple detection, can be refined with yaml parsing)
+  - id: "COMPOSE_RESTART_ALWAYS"
+    description: "Restart policy set to always"
+    regex: "restart:\\s*always"
+    severity: "low"
+    confidence: "high"
+    category: "infrastructure"
+    remediation: "Consider 'on-failure' or specific restart policies."
+
+  - id: "COMPOSE_PORT_EXPOSURE"
+    description: "Port exposed to host (broad range)"
+    regex: "\\s*-\\s*[\"']?0\\.0\\.0\\.0:"
+    severity: "medium"
+    confidence: "high"
+    category: "infrastructure"
+    remediation: "Bind ports to localhost (127.0.0.1) if external access is not required."

build/lib/openaudit/rules/secrets.yaml

@@ -0,0 +1,18 @@
+rules:
+  - id: "AWS_ACCESS_KEY_ID"
+    description: "AWS Access Key ID"
+    regex: "(?:A3T[A-Z0-9]|AKIA|AGPA|AIDA|AROA|AIPA|ANPA|ANVA|ASIA)[A-Z0-9]{16}"
+    entropy_check: false
+    severity: "critical"
+    confidence: "high"
+    category: "secret"
+    remediation: "Revoke the key immediately and rotate credentials."
+
+  - id: "GENERIC_API_KEY"
+    description: "Potential High Entropy Key"
+    regex: "api_key['\"]?\\s*[:=]\\s*['\"]?([A-Za-z0-9_\\-]{32,})"
+    entropy_check: true
+    severity: "high"
+    confidence: "medium"
+    category: "secret"
+    remediation: "Verify if this is a real secret and move to environment variables."

openaudit.egg-info/PKG-INFO

@@ -26,7 +26,7 @@ OpenAuditKit is an open-source CLI security audit tool designed to scan your cod
 - **Config Scanning**: Identifies misconfigurations in deployment files (e.g., .env, Dockerfile).
 - **Secure**: Secrets are masked in outputs; offline-first design.
 - **Backend Ready**: Feature-based architecture with Pydantic models for easy integration into dashboards or APIs.
-- **Customizable**: Add your own rules! See [Rule Documentation](rules/README.md).
+- **Customizable**: Add your own rules! See [Rule Documentation](openopenaudit/rules/README.md).
 
 ## 🛡️ Why OpenAuditKit?
 
@@ -48,7 +48,13 @@ Often, security tools are either too simple (grep) or too complex (enterprise SA
 
 ## Installation
 ```bash
-pip install -r requirements.txt
+# From PyPI (Coming Real Soon!)
+pip install openaudit
+
+# Or from source
+git clone https://github.com/neuralforgeone/OpenAuditKit.git
+cd OpenAuditKit
+pip install .
 ```
 
 ## Usage

openaudit/ai/__init__.py

@@ -0,0 +1,5 @@
+from .models import PromptContext, AIResult
+from .protocol import AgentProtocol
+from .ethics import Redactor, ConsentManager
+
+__all__ = ["PromptContext", "AIResult", "AgentProtocol", "Redactor", "ConsentManager"]

openaudit/ai/engine.py

@@ -0,0 +1,30 @@
+from typing import List, Dict
+from openaudit.ai.models import PromptContext, AIResult
+from openaudit.ai.protocol import AgentProtocol
+from openaudit.ai.ethics import ConsentManager
+
+class AIEngine:
+    """
+    Orchestrator for AI Agents.
+    """
+    def __init__(self, offline_only: bool = True):
+        self.offline_only = offline_only
+        self.agents: Dict[str, AgentProtocol] = {}
+
+    def register_agent(self, agent: AgentProtocol):
+        """
+        Register a new agent capability.
+        """
+        self.agents[agent.name] = agent
+
+    def run_agent(self, agent_name: str, context: PromptContext) -> AIResult:
+        """
+        Run a specific agent by name.
+        """
+        if not ConsentManager.has_consented():
+            raise PermissionError("User has not consented to AI usage.")
+            
+        if agent_name not in self.agents:
+            raise ValueError(f"Agent {agent_name} not found.")
+            
+        return self.agents[agent_name].run(context)

openaudit/ai/ethics.py

@@ -0,0 +1,58 @@
+import re
+from typing import List
+from pathlib import Path
+
+# Placeholder for consent storage file
+CONSENT_FILE = Path(".openaudit_consent")
+
+class Redactor:
+    """
+    Utility to redaction secrets from text before sending to an LLM.
+    Uses basic patterns to identify potential secrets.
+    """
+    
+    # Simple regex for common secrets (placeholder, ideally reuse SecretScanner patterns)
+    # This is a safety net; specific scanners should also redact.
+    SENSITIVE_PATTERNS = [
+        r"(?i)(api[_-]?key|secret|token|password|passwd|pwd)['\"]?\s*[:=]\s*['\"]?([a-zA-Z0-9_\-]{8,})['\"]?",
+        r"(?i)private[_-]?key",
+    ]
+
+    @classmethod
+    def redact(cls, text: str) -> str:
+        """
+        Replace sensitive patterns with [REDACTED].
+        """
+        redacted_text = text
+        for pattern in cls.SENSITIVE_PATTERNS:
+            redacted_text = re.sub(pattern, lambda m: m.group(0).replace(m.group(2), "[REDACTED]"), redacted_text)
+        return redacted_text
+
+class ConsentManager:
+    """
+    Manages user consent for AI features.
+    """
+    
+    @staticmethod
+    def has_consented() -> bool:
+        """
+        Check if the user has explicitly consented to AI usage.
+        For now, we check for a specific marker file or env var.
+        """
+        # In a real impl, this might check a global config file in user home
+        return CONSENT_FILE.exists()
+
+    @staticmethod
+    def grant_consent():
+        """
+        Grant consent creates the marker.
+        """
+        CONSENT_FILE.touch()
+
+    @staticmethod
+    def revoke_consent():
+        """
+        Revoke consent removes the marker.
+        """
+        if CONSENT_FILE.exists():
+            CONSENT_FILE.unlink()