Integration tests can occupy a painful middle ground: too slow for rapid iteration, too incomplete for deployment confidence. Partial-stack integration tests that mock some components but run others create a "fake system" that passes tests but fails in production. The exception is focused integration tests — narrow-scope tests that exercise a single external dependency's real API with exhaustive edge-case coverage (see Pattern 1.5). Unit tests are fast but test code, not behavior. E2E tests are comprehensive but slow and brittle. We need a testing approach that provides real confidence without sacrificing developer velocity.
Stack tests run the complete Docker stack (app, databases, caches, queues) and verify behavior through the API only. No internal mocks, no backend shortcuts. The test treats the system as a black box: it spins up the stack, waits for readiness, makes API calls, and asserts on observable effects.
Stack tests are the primary verification mechanism for the design intent established through context harvesting. When the agent has harvested context, framed a problem, and designed a solution, stack tests validate that the solution works end-to-end — not just that individual functions return correct values, but that the full user journey behaves as designed.
The defining characteristic of a stack test is that it models an atomic user journey — a single, complete interaction from the user's perspective. Not "does the order service work?" but "a user places an order, pays, and sees their balance update." The test verifies the entire journey end-to-end, not individual components in isolation.
Stack tests differ from other approaches in key ways:
- Scope: Entire system — every service runs in real containers, plus external deps to the fullest possible extent
- Verification: API-level only — tests interact like external clients
- Granularity: One atomic user journey per test — a complete interaction from the user's perspective
- Isolation: Full isolation between test runs — no shared state
- Mock policy: Zero mocks for owned services — real databases, real Redis, real side-effects
- Failure diagnosticity: High — failures always indicate real bugs, never test artifacts
- Runs on a developer machine: The full stack must be executable locally — no cloud deployment, no staging environment, no infrastructure beyond Docker. Every member of the team (human or AI) can run the complete test suite on their own machine and get deterministic results
The naive approach to stack testing would be:
- Generate a unique
docker-compose.{test-name}.ymlwith dynamic ports - Run
docker compose up -dand wait for health checks - Execute user journeys via HTTP API
- Assert on primary responses AND second-order effects (cross-API verification, audit)
- Run
docker compose down -vto clean up everything
However, leaving the orchestration of Docker operations to the agent is brittle and non-deterministic. Each session, the agent must re-derive compose file generation, port allocation, health check polling, container naming, volume cleanup, and authentication setup — and get every detail right. The first time there's a problem — a port collision, a stale volume, a race condition in health polling — the agent spends tokens debugging infrastructure instead of application logic.
This leads to a defining insight for closed-loop testing (closely related to the WISC context engineering framework — Write, Isolate, Select, Compress):
Turn brittle agent-side orchestration into deterministic tooling. The first time trusting the agent to manually manage Docker operations causes a problem, invest in writing the tooling to fully automate that problem. Drive the agent to build a toolkit that encapsulates container lifecycle, port allocation, health check polling, authentication, and cleanup into a deterministic interface the agent invokes simply — await stack.start(), await stack.cleanup() — instead of re-deriving each operation every session.
In the reference project, this pattern produced StackTestUtils — a single class providing container lifecycle management, dynamic port allocation, authentication helpers, health check polling, database access for verification, log search for debugging, and blockchain transaction verification. The agent didn't write this in one pass. It accumulated over sessions as each brittle manual operation was automated into a reliable method. Once the toolkit existed, every subsequent agent session could run stack tests deterministically without re-deriving infrastructure logic.
The test structure remains the same — each test is one atomic user journey:
tests/stack/
01-app-startup.stack.test.ts # Journey: system comes online and reports healthy
02-authentication.stack.test.ts # Journey: user registers, logs in, receives a token
03-basic-crud.stack.test.ts # Journey: user creates, reads, updates, deletes a resource
04-checkout-complete.stack.test.ts # Journey: user adds items, checks out, payment succeeds
05-refund-and-reconciliation.stack.test.ts # Journey: user requests refund, balance updates
The framing matters. "Does authentication work?" tests a component. "A user registers, logs in, and receives a valid token" tests a journey. Journeys catch component-interaction bugs that component tests cannot.
Run sequentially, each test building confidence in layers. If 01-app-startup fails, the agent knows immediately: don't waste time debugging order processing logic — the foundation is broken.
| Dimension | Unit Tests | Focused Integration Tests | Stack Tests | E2E Tests |
|---|---|---|---|---|
| Scope | Single function/class | Adapter code + one real external dependency | Full system + external deps (to fullest possible extent) | Full system + external deps |
| Speed | Milliseconds | Seconds | Seconds to minutes | Minutes |
| Isolation | Complete (in-memory) | Partial (one real external service) | Complete (per-test containers) | Usually shared environments |
| Confidence Level | Low (implementation detail) | Medium-high (real external behavior) | High (production-like) | High (but flaky) |
| Mock Policy | Everything | Zero (test against real API) | Zero mocks | Zero mocks |
| Failure Diagnosticity | Low (false positives from mocks) | High (real dependency behavior) | High (real failures) | Low (timing, flakiness) |
| Typical Use | Algorithm correctness, module contracts | Exhaustive edge-case coverage of external dependency | System behavior, user journeys | Critical paths, smoke tests |
| Runs Locally | Always | Usually (testnet/sandbox) | Must — Docker only | Often requires cloud/staging |
Stack tests are not limited to driving backend APIs directly. For web applications, the same pattern applies with a browser automation layer like Playwright: spin up the full stack, then drive user journeys through the actual UI — form submissions, page transitions, rendered output — to verify that the combined frontend and backend work correctly end-to-end. The principle remains the same: real system, real dependencies, no mocks. Only the entry point changes from HTTP API calls to browser interactions.
Don't write "integration tests" that start a few services and mock others. You end up testing your mocks, not your system. Either test at the unit level (fast, isolated), use focused integration tests for exhaustive coverage of a single external dependency's edge cases (Pattern 1.5), or test at the stack level (complete, real). Partial-stack tests with mixed mocks give you the worst of both worlds: slow tests that don't prove anything.
Don't run stack tests for every code change during development. Use unit tests for rapid iteration. Run stack tests before committing or as a pre-commit hook.
Stack-first development is most achievable on greenfield projects designed with this approach from the start. When you control the architecture, you can ensure every component fits cleanly into a Docker stack and every service exposes an API surface that's testable from the outside.
For large, sprawling existing systems, full adoption may not be immediately practical:
- Docker complexity limits: Systems with dozens of microservices, specialized hardware dependencies, or complex networking may be too large to containerize as a single stack
- Dependency depth: Some systems have too many interdependent services to automate into a single deterministic stack
- Legacy constraints: Existing systems may have components that resist containerization (kernel modules, hardware integrations, licensed software)
In these cases, the stack-test approach can still be applied incrementally: identify the highest-value subsystems, extract them behind clean API boundaries, and stack-test those boundaries. The goal is to expand coverage over time, not to boil the ocean on day one.
The key insight: stack-first development is an architectural decision, not just a testing strategy. It shapes how you design services, define boundaries, and manage dependencies. Starting greenfield with this approach is straightforward. Retrofitting onto brownfield systems follows the Ratchet pattern: incremental extraction that only tightens, never loosens. Each subsystem brought under stack testing stays there — coverage ratchets forward one boundary at a time.
Stack tests have their limits. The further and deeper into a user journey sequence a test reaches, the more reasons exist for it to fail — and the harder it becomes to distinguish a real regression from a test that's exercising too many concerns at once. A test that verifies authentication, order creation, payment processing, fulfillment, notification, and audit logging in a single journey is brittle by construction: any one of those layers breaking causes the whole test to fail, and the diagnostic signal degrades with each added concern.
When stack tests become brittle in this way, the remedy is not to remove coverage but to reconcile the test design. Ask the agent to scan all stack tests for the affected domain area, then brainstorm a plan to produce a cleaner, simpler, and more reliable test structure that covers the same functionality. The reconciliation optimizes along three axes:
- De-duplicate: Remove overlapping assertions across tests that verify the same side effect through different journeys
- Group logically: Restructure tests so each journey exercises a coherent set of concerns rather than spanning every layer in the system
- Improve reliability: Reduce the surface area that each test depends on, so failures point to specific subsystems rather than the entire stack
This is a continuous maintenance activity, not a one-time fix. As the system grows, stack tests that were clean at inception accumulate scope. Periodic reconciliation keeps the test suite diagnostic rather than brittle.
The startup test needs to verify more than "is the server running?" A production system depends on external services — email relays, message queues, secrets managers, blockchain nodes — that may be configured but not healthy. The health endpoint should support a test mode that decorates its response with detailed service health information:
// GET /health?mode=test
{
"status": "healthy",
"services": {
"postgres": { "connected": true, "latency_ms": 3 },
"redis": { "connected": true, "latency_ms": 1 },
"email_relay": { "configured": true, "smtp_verified": true },
"message_queue": { "connected": true, "pending_jobs": 0 },
"secrets_manager": { "connected": true, "resolved_secrets": 14 }
},
"version": "2.4.1",
"env": "test"
}Why test mode on the health endpoint, not synthetic endpoints? Creating /test/db-health, /test/queue-health etc. is an anti-pattern: it adds test-only code to production, leaks infrastructure details, and requires maintenance alongside the real endpoints. Instead, the existing health endpoint accepts a ?mode=test query parameter that enriches its response with service-level diagnostics. The same endpoint serves both production health checks (simple status) and test-time validation (detailed diagnostics).
Stack tests need realistic data to exercise user journeys. A user can't place an order if the catalog is empty. A refund test can't run without a prior purchase. Bootstrapping is the process of loading test fixture data before domain tests begin.
The bootstrap step (typically 02-bootstrap-test-data.stack.test.ts) runs after the startup test and before domain tests. It loads the prerequisite data that domain journeys need: products in the catalog, pre-configured users, reference data, and system settings.
Bootstrap principles:
- Use the same internal service APIs that user-facing or admin functions use — bootstrap goes through the same code paths as production, not synthetic test-only endpoints that would never exist in a real deployment
- Direct database seeding is acceptable for bootstrapping, but it must use the same internal service layer and data access patterns the application uses — not raw SQL that bypasses validation, hooks, or business logic
- Each bootstrap test creates one category of fixture data — products, users, configuration
- Bootstrap tests are sequential: domain tests assume all bootstrap tests pass
- If a bootstrap test fails, all subsequent tests are meaningless — the diagnostic signal is clear
- Pattern 1.2 — Full-Loop Assertion Layering: How to structure assertions within stack tests
- Pattern 1.3 — Sequential/Additive Test Design: How to order stack tests for maximum diagnostic value
- Pattern 1.4 — Container Isolation: How to run stack tests concurrently without collision
- Pattern 1.5 — Real Dependencies in E2E/Integration and Stack Tests: Why stack tests use real dependencies — mocks are appropriate in unit tests, not in stack tests
- L0 — Unit Tests as Contract: Unit tests validate individual module contracts — stack tests validate system behavior
Back to L1 Overview | Previous: L0 Foundation | Next: Pattern 1.2