|
2 | 2 |
|
3 | 3 | ## CRG Grade: C — ACHIEVED 2026-04-04 |
4 | 4 |
|
5 | | -> Generated 2026-03-29 by punishing audit. |
| 5 | +> Generated 2026-03-29 by punishing audit. Superseded 2026-04-16 by the |
| 6 | +> P0/P1/P2 test work documented below. |
6 | 7 |
|
7 | 8 | ## Current State (updated 2026-04-16) |
8 | 9 |
|
9 | 10 | | Category | Count | Notes | |
10 | 11 | |-------------|-------|-------| |
11 | | -| Unit tests | 7 | gateway, policy_compiler, policy_loader, policy_validator, policy_property, performance, http_capability_gateway | |
| 12 | +| Unit tests | 9 | gateway, policy_compiler, policy_loader, policy_validator, policy_property, performance, http_capability_gateway, **circuit_breaker**, **k9_contract** | |
12 | 13 | | Security | 1 | security_test.exs: sanitization, headers, SSRF, capability tokens (30+ tests) | |
13 | 14 | | E2E | 1 | e2e_test.exs: full lifecycle, policy hot-reload, upstream proxy, health probes (20+ tests) | |
| 15 | +| Concurrency | 1 | concurrency_test.exs: rate limiter contention, circuit breaker serialization, atomic reload under load | |
14 | 16 | | Fuzz | 1 | fuzz_test.exs: property-based fuzzing with StreamData (6 properties) | |
15 | | -| Benchmarks | 0 | None | |
16 | | - |
17 | | -**Source modules:** ~19 Elixir modules (gateway, circuit_breaker, proxy, rate_limiter, safe_trust, graphql_handler, grpc_handler, policy_*, minikaran, logging, etc.) + 2 Idris2 ABI + 4 Zig FFI. |
18 | | - |
19 | | -## What's Missing |
20 | | - |
21 | | -### P2P (Property-Based) Tests |
22 | | -- [ ] Policy compilation: fuzz arbitrary YAML policies through compiler |
23 | | -- [ ] Rate limiter: property tests for token bucket invariants |
24 | | -- [ ] Circuit breaker: state machine property tests (closed->open->half-open) |
25 | | -- [ ] GraphQL/gRPC handler: arbitrary request shape handling |
26 | | - |
27 | | -### E2E Tests |
28 | | -- [ ] Full request lifecycle: client -> gateway -> upstream -> response |
29 | | -- [ ] Multi-protocol routing (HTTP, GraphQL, gRPC through single gateway) |
30 | | -- [ ] Policy hot-reload under load |
31 | | -- [ ] Health check / readiness probe validation |
32 | | - |
33 | | -### Aspect Tests |
34 | | -- **Security:** Request sanitization, header injection, SSRF prevention, capability token validation — covered in `test/security_test.exs` |
35 | | -- **Performance:** No load tests, no latency benchmarks, no throughput measurement |
36 | | -- **Concurrency:** No tests for concurrent connections, race conditions in rate limiter, circuit breaker under contention |
37 | | -- **Error handling:** No tests for upstream timeout, malformed requests, policy parse failures |
38 | | - |
39 | | -### Build & Execution |
40 | | -- [ ] `mix test` runner verification |
41 | | -- [ ] Zig FFI integration test execution |
42 | | -- [ ] Container build + smoke test |
43 | | - |
44 | | -### Benchmarks Needed |
45 | | -- [ ] Request routing latency (per-protocol) |
46 | | -- [ ] Policy evaluation overhead |
47 | | -- [ ] Rate limiter throughput |
48 | | -- [ ] Circuit breaker state transition cost |
49 | | - |
50 | | -### Self-Tests |
51 | | -- [ ] Configuration validation on startup |
52 | | -- [ ] Policy schema self-check |
53 | | -- [ ] Capability token format verification |
| 17 | +| Benchmarks | 2 | performance_test.exs (existing) + benchmark_test.exs (rate limiter / circuit breaker / route lookup) | |
| 18 | + |
| 19 | +**Source modules:** ~19 Elixir modules + 2 Idris2 ABI + 2 Zig FFI parsers. |
| 20 | + |
| 21 | +## Coverage Summary |
| 22 | + |
| 23 | +### ✅ Covered |
| 24 | + |
| 25 | +- **P2P (Property-Based) Tests** |
| 26 | + - Policy compilation: arbitrary YAML through compiler (`test/fuzz_test.exs`) |
| 27 | + - Circuit breaker: state machine transitions (`test/circuit_breaker_test.exs`) |
| 28 | + - Rate limiter: token bucket under contention (`test/concurrency_test.exs`) |
| 29 | + |
| 30 | +- **E2E Tests** |
| 31 | + - Full request lifecycle (`test/e2e_test.exs`) |
| 32 | + - Policy hot-reload under load (`test/concurrency_test.exs`) |
| 33 | + - Health check / readiness probe validation (`test/e2e_test.exs`) |
| 34 | + |
| 35 | +- **Aspect Tests** |
| 36 | + - **Security:** Request sanitization, header injection, SSRF prevention, capability token validation (`test/security_test.exs`) |
| 37 | + - **Concurrency:** Rate limiter and circuit breaker under contention (`test/concurrency_test.exs`) |
| 38 | + - **Performance:** Rate limiter, circuit breaker, route lookup benchmarks (`test/benchmark_test.exs`) |
| 39 | + |
| 40 | +- **Benchmarks** |
| 41 | + - Rate limiter throughput (`test/benchmark_test.exs`) |
| 42 | + - Circuit breaker state transition cost (`test/benchmark_test.exs`) |
| 43 | + - Exact vs regex vs global-fallback route lookup (`test/benchmark_test.exs`) |
| 44 | + - Policy evaluation overhead (`test/performance_test.exs`) |
| 45 | + - Full plug pipeline throughput (`test/benchmark_test.exs`) |
| 46 | + |
| 47 | +### ⚠️ Still Missing |
| 48 | + |
| 49 | +- **Multi-protocol routing tests** — GraphQL/gRPC handlers are stubs per `docs/SUPPORTED-FEATURES.md`, so this is out of MVP scope rather than "missing". |
| 50 | +- **Zig FFI integration test execution** — requires zig toolchain; covered by separate FFI build step. |
| 51 | +- **Container build smoke test** — performed in CI, not in `mix test`. |
| 52 | +- **Error handling: upstream timeout** — Req receive_timeout covered implicitly; no dedicated test. |
| 53 | +- **Real-CA mTLS integration test** — code uses `Record.extract` accessors but no live cert in test fixtures. |
| 54 | +- **Self-tests for config validation on startup** — Application.start refuses without policy, but no dedicated assertion. |
54 | 55 |
|
55 | 56 | ## Priority |
56 | 57 |
|
57 | | -**CRITICAL.** 19 modules with 7 unit tests = 37% coverage by file count. A security gateway with ZERO security tests is a contradiction. No benchmarks for a performance-sensitive proxy is unacceptable. No concurrency tests for a concurrent system is negligent. |
| 58 | +Originally **CRITICAL** when only 7 unit tests covered 19 modules. |
| 59 | +Now: the release gate in `docs/RELEASE-CRITERIA.md` maps every MVP claim |
| 60 | +to a concrete test file. Remaining items are clearly marked above and |
| 61 | +are not release blockers for v0.1.0. |
58 | 62 |
|
59 | 63 | ## FUZZ STATUS |
60 | 64 |
|
|
0 commit comments