Skip to content

Commit 27f3d6f

Browse files
vsilentCopilot
vsilent
and
Copilot
committed
feat(sniff): log source discovery + database persistence
- Create src/sniff/discovery.rs: LogSource, LogSourceType, discovery functions for system logs, Docker containers, and custom paths - Create src/database/repositories/log_sources.rs: CRUD for log_sources and log_summaries tables (follows existing alerts repository pattern) - Add log_sources and log_summaries tables to init_database() - Export docker module from lib.rs for reuse by sniff discovery - 14 unit tests (8 discovery + 6 repository) Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 2f6db16 commit 27f3d6f

21 files changed

Lines changed: 3446 additions & 7 deletions

‎.github/copilot-instructions.md‎

Lines changed: 113 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,113 @@
1+
# Stackdog Security — Copilot Instructions
2+
3+
## What This Project Is
4+
5+
Stackdog is a Rust-based security platform for Docker containers and Linux servers. It collects events via eBPF syscall monitoring, runs them through a rule/signature engine and optional ML anomaly detection, manages firewall responses (nftables/iptables + container quarantine), and exposes a REST + WebSocket API consumed by a React/TypeScript dashboard.
6+
7+
## Workspace Structure
8+
9+
This is a Cargo workspace with two crates:
10+
- `.` — Main crate (`stackdog`): HTTP server, all security logic
11+
- `ebpf/` — Separate crate (`stackdog-ebpf`): eBPF programs compiled for the kernel (uses `aya-ebpf`)
12+
13+
## Build, Test, and Lint Commands
14+
15+
```bash
16+
# Build
17+
cargo build
18+
cargo build --release
19+
20+
# Tests
21+
cargo test --lib # Unit tests only (in-source)
22+
cargo test --all # All tests including integration
23+
cargo test --lib -- events:: # Run tests for a specific module
24+
cargo test --lib -- rules::scorer # Run a single test by name prefix
25+
26+
# Code quality
27+
cargo fmt --all
28+
cargo clippy --all
29+
cargo audit # Dependency vulnerability scan
30+
31+
# Benchmarks
32+
cargo bench
33+
34+
# Frontend (in web/)
35+
npm test
36+
npm run lint
37+
npm run build
38+
```
39+
40+
## Environment Setup
41+
42+
Requires a `.env` file (copy `.env.sample`). Key variables:
43+
```
44+
APP_HOST=0.0.0.0
45+
APP_PORT=5000
46+
DATABASE_URL=stackdog.db
47+
RUST_BACKTRACE=full
48+
```
49+
50+
System dependencies (Linux): `libsqlite3-dev libssl-dev clang llvm pkg-config`
51+
52+
## Architecture
53+
54+
```
55+
Collectors (Linux only) Rule Engine Response
56+
eBPF syscall events → Signatures → nftables/iptables
57+
Docker daemon events → Threat scoring → Container quarantine
58+
Network events → ML anomaly det. → Alerting
59+
60+
REST + WebSocket API
61+
React/TypeScript UI
62+
```
63+
64+
**Key src/ modules:**
65+
66+
| Module | Purpose |
67+
|---|---|
68+
| `events/` | Core event types: `SyscallEvent`, `SecurityEvent`, `NetworkEvent`, `ContainerEvent` |
69+
| `rules/` | Rule engine, signature database, threat scorer |
70+
| `alerting/` | `AlertManager`, notification channels (Slack/email/webhook) |
71+
| `collectors/` | eBPF loader, Docker daemon events, network collector (Linux only) |
72+
| `firewall/` | nftables management, iptables fallback, `QuarantineManager` (Linux only) |
73+
| `ml/` | Candle-based anomaly detection (optional `ml` feature) |
74+
| `correlator/` | Event correlation engine |
75+
| `baselines/` | Baseline learning for anomaly detection |
76+
| `database/` | SQLite connection pool (`r2d2` + raw `rusqlite`), repositories |
77+
| `api/` | actix-web REST endpoints + WebSocket |
78+
| `response/` | Automated response action pipeline |
79+
80+
## Key Conventions
81+
82+
### Platform-Gating
83+
Linux-only modules (`collectors`, `firewall`) and deps (aya, netlink) are gated:
84+
```rust
85+
#[cfg(target_os = "linux")]
86+
pub mod firewall;
87+
```
88+
The `ebpf` and `ml` features are opt-in and must be enabled explicitly:
89+
```bash
90+
cargo build --features ebpf
91+
cargo build --features ml
92+
```
93+
94+
### Error Handling
95+
- Use `anyhow::{Result, Context}` for application/binary code
96+
- Use `thiserror` for library error types
97+
- Never use `.unwrap()` in production code; use `?` with `.context("...")`
98+
99+
### Database
100+
The project uses raw `rusqlite` with `r2d2` connection pooling. `DbPool` is `r2d2::Pool<SqliteConnectionManager>`. Tables are created with `CREATE TABLE IF NOT EXISTS` in `database::connection::init_database`. Repositories are in `src/database/repositories/` and receive a `&DbPool`.
101+
102+
### API Routes
103+
Each API sub-module exports a `configure_routes(cfg: &mut web::ServiceConfig)` function. All routes are composed in `api::configure_all_routes`, which is the single call site in `main.rs`.
104+
105+
### Test Location
106+
- **Unit tests**: `#[cfg(test)] mod tests { ... }` inside source files
107+
- **Integration tests**: `tests/` directory at workspace root
108+
109+
### eBPF Programs
110+
The `ebpf/` crate is compiled separately for the Linux kernel. User-space loading is handled by `src/collectors/ebpf/` using the `aya` library. Kernel-side programs use `aya-ebpf`.
111+
112+
### Async Runtime
113+
The main binary uses `#[actix_rt::main]`. Library code uses `tokio`. Avoid mixing runtimes.

‎.qwen/PROJECT_MEMORY.md‎

Lines changed: 277 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,277 @@
1+
# Stackdog Security - Project Memory
2+
3+
## Project Identity
4+
5+
**Name:** Stackdog Security
6+
**Version:** 0.1.0 (Security-focused rewrite)
7+
**Type:** Container and Linux Server Security Platform
8+
**License:** MIT
9+
10+
## Core Mission
11+
12+
> Provide real-time security monitoring, AI-powered threat detection, and automated response for Docker containers and Linux servers using Rust and eBPF technologies.
13+
14+
## Key Decisions
15+
16+
### Architecture Decisions
17+
18+
| ID | Decision | Rationale | Date |
19+
|----|----------|-----------|------|
20+
| **ARCH-001** | Use eBPF for syscall monitoring | Minimal overhead (<5% CPU), kernel-level visibility, safe (sandboxed) | 2026-03-13 |
21+
| **ARCH-002** | Use Candle for ML instead of Python | Native Rust, no Python dependencies, fast inference, maintained by HuggingFace | 2026-03-13 |
22+
| **ARCH-003** | Use nftables over iptables | Modern, faster, better batch support, iptables as fallback | 2026-03-13 |
23+
| **ARCH-004** | TDD development methodology | Better code quality, maintainability, regression prevention | 2026-03-13 |
24+
| **ARCH-005** | Functional programming principles | Immutability, fewer bugs, easier reasoning about code | 2026-03-13 |
25+
26+
### Technology Choices
27+
28+
| Component | Technology | Alternatives Considered |
29+
|-----------|-----------|------------------------|
30+
| **eBPF Framework** | aya-rs | libbpf (C), bcc (Python) |
31+
| **ML Framework** | Candle (HuggingFace) | PyTorch (Python), ONNX Runtime, linfa |
32+
| **Web Framework** | Actix-web 4.x | Axum, Rocket |
33+
| **Database** | SQLite + rusqlite + r2d2 | PostgreSQL, Redis |
34+
| **Firewall** | nftables (netlink) | iptables, firewalld |
35+
36+
## Project Structure
37+
38+
```
39+
stackdog/
40+
├── src/
41+
│ ├── collectors/ # Event collection (eBPF, Docker, etc.)
42+
│ ├── events/ # Event types and structures
43+
│ ├── ml/ # ML engine (Candle-based)
44+
│ ├── firewall/ # Firewall management (nftables/iptables)
45+
│ ├── response/ # Automated response actions
46+
│ ├── correlator/ # Event correlation
47+
│ ├── alerting/ # Alert system
48+
│ ├── api/ # REST API + WebSocket
49+
│ ├── config/ # Configuration
50+
│ ├── models/ # Data models
51+
│ ├── database/ # Database operations
52+
│ └── utils/ # Utilities
53+
├── ebpf/ # eBPF programs (separate crate)
54+
├── web/ # React/TypeScript frontend
55+
├── tests/ # Integration tests
56+
├── benches/ # Performance benchmarks
57+
└── models/ # Pre-trained ML models
58+
```
59+
60+
## Development Principles
61+
62+
### Clean Code (Robert C. Martin)
63+
64+
1. **DRY** - Don't Repeat Yourself
65+
2. **SRP** - Single Responsibility Principle
66+
3. **OCP** - Open/Closed Principle
67+
4. **DIP** - Dependency Inversion Principle
68+
5. **Functional First** - Immutability, From/Into traits, builder pattern
69+
70+
### TDD Workflow
71+
72+
```
73+
Red → Green → Refactor
74+
```
75+
76+
1. Write failing test
77+
2. Run test (verify failure)
78+
3. Implement minimal code to pass
79+
4. Run test (verify pass)
80+
5. Refactor (maintain passing tests)
81+
82+
### Code Review Checklist
83+
84+
- [ ] Tests written first (TDD)
85+
- [ ] All tests pass
86+
- [ ] Code formatted (`cargo fmt`)
87+
- [ ] No clippy warnings
88+
- [ ] DRY principle followed
89+
- [ ] Functions < 50 lines
90+
- [ ] Error handling comprehensive
91+
- [ ] Documentation for public APIs
92+
93+
## Key APIs and Interfaces
94+
95+
### Event Types
96+
97+
```rust
98+
// Core security event
99+
pub enum SecurityEvent {
100+
Syscall(SyscallEvent),
101+
Network(NetworkEvent),
102+
Container(ContainerEvent),
103+
Alert(AlertEvent),
104+
}
105+
106+
// Syscall event from eBPF
107+
pub struct SyscallEvent {
108+
pub pid: u32,
109+
pub uid: u32,
110+
pub syscall_type: SyscallType,
111+
pub timestamp: DateTime<Utc>,
112+
pub container_id: Option<String>,
113+
}
114+
```
115+
116+
### ML Interface
117+
118+
```rust
119+
// Feature vector for ML
120+
pub struct SecurityFeatures {
121+
pub syscall_rate: f64,
122+
pub network_rate: f64,
123+
pub unique_processes: u32,
124+
pub privileged_calls: u32,
125+
// ...
126+
}
127+
128+
// Threat score output
129+
pub enum ThreatScore {
130+
Normal,
131+
Low,
132+
Medium,
133+
High,
134+
Critical,
135+
}
136+
```
137+
138+
### Firewall Interface
139+
140+
```rust
141+
pub trait FirewallBackend {
142+
fn add_rule(&self, rule: &Rule) -> Result<()>;
143+
fn remove_rule(&self, rule: &Rule) -> Result<()>;
144+
fn batch_update(&self, rules: &[Rule]) -> Result<()>;
145+
fn block_container(&self, container_id: &str) -> Result<()>;
146+
fn quarantine_container(&self, container_id: &str) -> Result<()>;
147+
}
148+
```
149+
150+
## Configuration
151+
152+
### Environment Variables
153+
154+
```bash
155+
APP_HOST=0.0.0.0
156+
APP_PORT=5000
157+
DATABASE_URL=stackdog.db
158+
RUST_LOG=info
159+
RUST_BACKTRACE=full
160+
161+
# Security-specific
162+
EBPF_ENABLED=true
163+
FIREWALL_BACKEND=nftables # or iptables
164+
ML_ENABLED=true
165+
ML_MODEL_PATH=models/
166+
ALERT_THRESHOLD=0.75
167+
```
168+
169+
### Cargo Features
170+
171+
```toml
172+
[features]
173+
default = ["nftables", "ml"]
174+
nftables = ["netlink-packet-route"]
175+
iptables = ["iptables"]
176+
ml = ["candle-core", "candle-nn"]
177+
ebpf = ["aya"]
178+
```
179+
180+
## Testing Strategy
181+
182+
### Test Categories
183+
184+
| Category | Location | Command | Coverage Target |
185+
|----------|----------|---------|-----------------|
186+
| Unit | `src/**/*.rs` | `cargo test` | 80%+ |
187+
| Integration | `tests/integration/` | `cargo test --test integration` | Critical paths |
188+
| E2E | `tests/e2e/` | `cargo test --test e2e` | Key workflows |
189+
| Benchmark | `benches/` | `cargo bench` | Performance targets |
190+
191+
### Performance Targets
192+
193+
| Metric | Target |
194+
|--------|--------|
195+
| Event throughput | 100K events/sec |
196+
| ML inference latency | <10ms |
197+
| Firewall update | <1ms per rule |
198+
| Memory usage | <256MB baseline |
199+
| CPU overhead | <5% |
200+
201+
## Dependencies
202+
203+
### Core
204+
205+
- `actix-web` - Web framework
206+
- `aya` - eBPF framework
207+
- `candle-core`, `candle-nn` - ML framework
208+
- `bollard` - Docker API
209+
- `rusqlite` - SQLite driver
210+
- `r2d2` - Connection pool
211+
- `netlink-packet-route` - nftables
212+
- `tokio` - Async runtime
213+
214+
### Development
215+
216+
- `mockall` - Mocking for tests
217+
- `criterion` - Benchmarking
218+
- `cargo-audit` - Security audit
219+
- `cargo-deny` - Dependency linting
220+
221+
## Milestones
222+
223+
| Version | Target | Features |
224+
|---------|--------|----------|
225+
| v0.1.0 | Week 4 | eBPF collectors, basic rules |
226+
| v0.2.0 | Week 6 | Firewall integration |
227+
| v0.3.0 | Week 10 | ML anomaly detection |
228+
| v0.4.0 | Week 12 | Alerting system |
229+
| v0.5.0 | Week 16 | Web dashboard |
230+
| v1.0.0 | Week 18 | Production release |
231+
232+
## Risks and Mitigations
233+
234+
| Risk | Impact | Probability | Mitigation |
235+
|------|--------|-------------|------------|
236+
| eBPF kernel compatibility | High | Medium | Fallback to auditd |
237+
| ML model accuracy | High | Medium | Start with rule-based, iterate |
238+
| Performance overhead | High | Low | Benchmark early, optimize |
239+
| False positives | Medium | High | Tunable thresholds, learning period |
240+
241+
## Open Questions
242+
243+
1. **Model Training:** How to collect training data for ML models?
244+
- Decision: Start with synthetic data, then real-world collection
245+
246+
2. **Multi-node Support:** Single node first, cluster later?
247+
- Decision: Single node for v1.0, cluster in v2.0
248+
249+
3. **Kubernetes Support:** Include in scope?
250+
- Decision: Out of scope for v1.0, backlog for v2.0
251+
252+
## Resources
253+
254+
### Documentation
255+
256+
- [DEVELOPMENT.md](DEVELOPMENT.md) - Full development plan
257+
- [TODO.md](TODO.md) - Task tracking
258+
- [BUGS.md](BUGS.md) - Bug tracking
259+
- [CONTRIBUTING.md](CONTRIBUTING.md) - Contribution guidelines
260+
261+
### External
262+
263+
- [Rust Book](https://doc.rust-lang.org/book/)
264+
- [Candle Docs](https://docs.rs/candle-core)
265+
- [aya-rs Docs](https://aya-rs.dev/)
266+
- [eBPF Documentation](https://ebpf.io/)
267+
268+
## Contact
269+
270+
- **Project Lead:** Vasili Pascal
271+
- **Email:** info@try.direct
272+
- **Twitter:** [@VasiliiPascal](https://twitter.com/VasiliiPascal)
273+
- **Gitter:** [stackdog/community](https://gitter.im/stackdog/community)
274+
275+
---
276+
277+
*Last updated: 2026-03-13*

‎Cargo.toml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ ebpf = []
7474
[dev-dependencies]
7575
# Testing
7676
tokio-test = "0.4"
77+
tempfile = "3"
7778

7879
# Benchmarking
7980
criterion = { version = "0.5", features = ["html_reports"] }

0 commit comments

Comments
 (0)