dmitriimaksimovdevelop
diff --git a/‎README.md‎
Lines changed: 127 additions & 5 deletions b/‎README.md‎
Lines changed: 127 additions & 5 deletions
diff --git a/‎cmd/melisai/main.go‎
Lines changed: 39 additions & 4 deletions b/‎cmd/melisai/main.go‎
Lines changed: 39 additions & 4 deletions
@@ -31,7 +31,7 @@ Most performance tools give you raw numbers. melisai gives you **a diagnosis**.
 - Runs Brendan Gregg's [USE Method](https://www.brendangregg.com/usemethod.html) automatically
 - Flags anomalies with severity (warning/critical) using field-tested thresholds
 - Computes a single **health score** (0-100) so an AI agent can decide what to do next
-- Generates a context-aware **AI prompt** with 23 known anti-patterns
+- Generates a context-aware **AI prompt** with 27 known anti-patterns
 - Works over **MCP** (Model Context Protocol) so Claude/Cursor can diagnose a server interactively
 
 ---
@@ -82,7 +82,7 @@ melisai mcp   # starts stdio JSON-RPC server
 | `get_health` | Quick 0-100 score + anomalies. Tier 1 only, no root needed | ~1s |
 | `collect_metrics` | Full profile with all BCC/eBPF tools. Args: `profile`, `focus`, `pid` | 10s-60s |
 | `explain_anomaly` | Root causes + recommendations for a specific anomaly ID | instant |
-| `list_anomalies` | All 23 detectable anomaly metric IDs with descriptions | instant |
+| `list_anomalies` | All 29 detectable anomaly metric IDs with descriptions | instant |
 
 ### Typical workflow
 
@@ -108,7 +108,9 @@ Agent                              melisai
 melisai collects metrics at three tiers with automatic fallback:
 
 ```
-Tier 1: /proc, /sys, ss, dmesg          ← always works, no root
+Tier 1: /proc, /sys, ss, ethtool, dmesg  ← always works, no root
+        Includes deep network diagnostics: conntrack, softnet,
+        IRQ distribution, NIC hardware, TCP extended stats
 Tier 2: 67 BCC tools (runqlat, bio...)   ← root + bcc-tools
 Tier 3: native eBPF (cilium/ebpf)       ← root + kernel ≥ 5.8 + BTF
 ```
@@ -126,7 +128,7 @@ The report includes:
 | `summary.resources` | USE metrics per resource (utilization, saturation, errors) |
 | `summary.recommendations[]` | Copy-paste sysctl commands with citations |
 | `categories.*` | Raw data: histograms, events, stack traces per subsystem |
-| `ai_context.prompt` | Dynamic prompt with system context and 23 anti-patterns |
+| `ai_context.prompt` | Dynamic prompt with system context and 27 anti-patterns |
 
 ---
 
@@ -212,7 +214,7 @@ Abbreviated JSON:
 
 ## Anomaly Detection
 
-20 threshold rules based on Gregg's recommended values:
+29 threshold rules based on Gregg's recommended values:
 
 | Metric | Warning | Critical | Source |
 |--------|---------|----------|--------|
@@ -229,10 +231,130 @@ Abbreviated JSON:
 | biolatency_p99_ssd | 5ms | 25ms | BCC histogram |
 | biolatency_p99_hdd | 50ms | 200ms | BCC histogram |
 | cpu_throttling | 100 | 1000 periods | cgroup cpu.stat |
+| conntrack_usage_pct | 70% | 90% | /proc/sys/net/netfilter/ |
+| softnet_dropped | 1 | 10 | /proc/net/softnet_stat |
+| listen_overflows | 1 | 100 | /proc/net/netstat |
+| nic_rx_discards | 100 | 10000 | ethtool -S |
+| tcp_close_wait | 1 | 100 | ss |
+| softnet_time_squeeze | 1 | 100 | /proc/net/softnet_stat |
+| tcp_abort_on_memory | 1 | 10 | /proc/net/netstat |
+| irq_imbalance | 5x ratio | 20x ratio | /proc/softirqs |
+| udp_rcvbuf_errors | 1 | 100 | /proc/net/snmp |
 | ... and 7 more (PSI, cache miss, DNS, container memory, network errors) | | | |
 
 ---
 
+## Manual Usage (without AI)
+
+melisai works perfectly as a standalone CLI tool — no AI agent required.
+
+### Getting Help
+
+```bash
+# General help — all commands and capabilities
+melisai --help
+
+# Detailed help for collect (profiles, flags, examples)
+melisai collect --help
+
+# Help for other commands
+melisai diff --help
+melisai install --help
+melisai mcp --help
+melisai capabilities --help
+```
+
+### Typical Manual Workflow
+
+```bash
+# 1. Quick health check — see if something is obviously wrong
+sudo melisai collect --profile quick -o quick.json
+
+# 2. Read the summary
+cat quick.json | python3 -m json.tool | head -30
+# or use jq:
+jq '.summary' quick.json
+
+# 3. Check health score and anomalies
+jq '.summary.health_score' quick.json                     # 0-100
+jq '.summary.anomalies[]' quick.json                      # what's wrong
+jq '.summary.recommendations[].title' quick.json          # what to fix
+
+# 4. Deep dive into network
+sudo melisai collect --profile standard --focus network -o net.json
+jq '.categories.network[0].data.conntrack' net.json       # conntrack usage
+jq '.categories.network[0].data.softnet_stats' net.json   # per-CPU drops
+jq '.categories.network[0].data.listen_overflows' net.json # accept queue
+
+# 5. Profile a specific process
+sudo melisai collect --profile standard --pid $(pgrep nginx) -o nginx.json
+
+# 6. Compare before/after a change
+sudo melisai collect --profile quick -o before.json
+# ... apply your fix ...
+sudo melisai collect --profile quick -o after.json
+melisai diff before.json after.json                        # human-readable
+melisai diff before.json after.json -o diff.json           # JSON diff
+
+# 7. Check what tools are available
+melisai capabilities
+```
+
+### Interpreting the Report
+
+The JSON report has four main sections:
+
+| Section | How to read it |
+|---------|----------------|
+| `summary.health_score` | 90-100 = healthy, 70-89 = some issues, <70 = needs attention |
+| `summary.anomalies` | Each has `severity` (warning/critical), `metric`, `message` |
+| `summary.recommendations` | Copy-paste the `commands` field to fix issues |
+| `categories.network[0].data` | Raw metrics — interfaces, TCP, conntrack, softnet, etc. |
+
+### Network Deep Diagnostics — Manual Inspection
+
+```bash
+# Conntrack table usage
+jq '.categories.network[0].data.conntrack' report.json
+# {"count": 15000, "max": 65536, "usage_pct": 22.9}
+
+# Softnet drops (per-CPU) — any "dropped" > 0 is bad
+jq '.categories.network[0].data.softnet_stats[] | select(.dropped > 0)' report.json
+
+# Listen overflows (accept queue full)
+jq '.categories.network[0].data | {listen_overflows, listen_drops}' report.json
+
+# NIC ring buffer (is it maxed out?)
+jq '.categories.network[0].data.interfaces[] | {name, driver, ring_rx_current, ring_rx_max, rx_discards}' report.json
+
+# IRQ imbalance (check if one CPU handles all network interrupts)
+jq '.categories.network[0].data.irq_distribution' report.json
+
+# TCP memory pressure
+jq '.categories.network[0].data | {prune_called, tcp_abort_on_memory, tcp_mem}' report.json
+```
+
+### Useful jq One-Liners
+
+```bash
+# All critical anomalies
+jq '.summary.anomalies[] | select(.severity == "critical")' report.json
+
+# All recommendations with commands
+jq '.summary.recommendations[] | {title, commands}' report.json
+
+# USE metrics for all resources
+jq '.summary.resources' report.json
+
+# Top CPU-consuming processes
+jq '.categories.process[0].data.top_by_cpu[:5]' report.json
+
+# BCC histogram percentiles
+jq '.categories.cpu[].histograms[]? | {name, p50, p99, max}' report.json
+```
+
+---
+
 ## Architecture
 
 ```
 
@@ -37,9 +37,29 @@ Collects metrics via BPF/eBPF tools (Brendan Gregg's ecosystem),
 procfs/sysfs, and standard utilities. Produces structured JSON
 reports optimized for AI-driven diagnostics and optimization.
 
-Tier 1: /proc, /sys, ss, dmesg (always works, no root)
-Tier 2: BCC tools — runqlat, biolatency, tcpconnlat, etc. (needs root + tools)
-Tier 3: Native eBPF (cilium/ebpf) — zero dependencies (needs root + kernel ≥ 4.15)`,
+Data collection tiers (automatic fallback):
+
+  Tier 1: /proc, /sys, ss, ethtool, dmesg (always works, no root)
+          CPU, memory, disk, network (deep: conntrack, softnet, IRQ,
+          NIC ring buffers, TCP extended stats), process, container.
+  Tier 2: 67 BCC tools — runqlat, biolatency, tcpconnlat, etc.
+          (needs root + bcc-tools installed)
+  Tier 3: Native eBPF (cilium/ebpf) — tcpretrans kprobe
+          (needs root + kernel ≥ 5.8 + BTF)
+
+Network deep diagnostics (Tier 1):
+  - Conntrack table usage and drops (/proc/sys/net/netfilter/)
+  - Softnet per-CPU stats: drops, time_squeeze (/proc/net/softnet_stat)
+  - IRQ distribution: NET_RX delta per CPU (/proc/softirqs)
+  - NIC hardware: driver, speed, MTU, queues, ring buffer, RPS, bond
+  - TCP extended: ListenOverflows, ListenDrops, PruneCalled, TCPAbortOnMemory
+  - UDP stats: RcvbufErrors, SndbufErrors (/proc/net/snmp)
+  - Socket memory: orphans, TCP mem pages (/proc/net/sockstat)
+  - 20+ sysctls: rmem_max, netdev_max_backlog, ip_local_port_range,
+    tcp_fastopen, tcp_notsent_lowat, default_qdisc, and more
+
+29 anomaly detection rules, health score (0-100), actionable recommendations.
+MCP server for interactive diagnostics from Claude Desktop / Cursor.`,
 		Version: version,
 	}
 
@@ -60,7 +80,22 @@ Tier 3: Native eBPF (cilium/ebpf) — zero dependencies (needs root + kernel ≥
 	collectCmd := &cobra.Command{
 		Use:   "collect",
 		Short: "Collect system performance metrics",
-		Long:  "Run all available collectors and produce a structured JSON report.",
+		Long: `Run all available collectors and produce a structured JSON report.
+
+Collectors run in two phases to avoid observer effect:
+  Phase 1: Tier 1 (procfs/sysfs) — CPU, memory, disk, network, process, container
+  Phase 2: Tier 2/3 (BCC/eBPF) — histograms, events, stack traces
+
+Network collector includes deep diagnostics:
+  conntrack, softnet stats, IRQ distribution, NIC hardware details,
+  TCP extended counters (ListenOverflows, PruneCalled, etc.)
+
+Profiles: quick (10s), standard (30s), deep (60s)
+
+Examples:
+  melisai collect --profile quick -o report.json
+  melisai collect --profile standard --focus network -o net.json
+  melisai collect --profile deep --pid 12345 -o app.json`,
 		RunE: func(cmd *cobra.Command, args []string) error {
 			cfg := collector.DefaultConfig()
 			cfg.Profile = collectProfile