Skip to content

1.2-full-loop-assertions.md

Latest commit

 

History

History
76 lines (52 loc) · 4.78 KB

1.2-full-loop-assertions.md

File metadata and controls

76 lines (52 loc) · 4.78 KB
Raw

Pattern 1.2 — Full-Loop Assertion Layering

Full-Loop Assertion Layering

Problem

A test that asserts only on API response status codes gives false confidence. A 200 OK response doesn't mean the system worked correctly — it only means the API didn't crash. Side effects might be missing: database not updated, cache not invalidated, audit log not written, event not published. When tests fail, shallow assertions make diagnosis difficult.

Solution

Full-loop assertion layering structures checks at three levels of increasing distance from the primary action:

  1. Primary assertions: Direct response from the API (status, body structure, immediate fields)
  2. Second-order assertions: Derived effects verified through a DIFFERENT API endpoint than the one that performed the action — e.g., create a user via POST, then verify the user exists via the list endpoint. Cross-API verification proves data was persisted, not just that the operation returned a success object.
  3. Third-order assertions: Cross-functional verification via administrative or observability APIs — audit logs, email notifications, cross-endpoint consistency, authentication enforcement. These prove that the system's side effects (notification pipeline, audit trail, auth middleware) are working correctly.

Each layer provides diagnostic signal. If primary passes but second-order fails, the agent knows: core logic works, persistence is broken. If primary and second-order pass but third-order fails: happy path works, observability is broken.

In Practice

Example: "User places an order" journey with three-layer assertions

// Journey: A user adds items to cart, checks out, and the order is fulfilled
// Primary assertion: API response
const orderResponse = await api.post('/orders', {
  items: [{ productId: 'prod_123', quantity: 2 }],
  shippingAddress: { zip: '90210', country: 'US' }
});
expect(orderResponse.status).toBe(201);
expect(orderResponse.data.orderId).toBeDefined();

// Second-order: Cross-API verification — cart emptied via a DIFFERENT endpoint
// than the one that created the order. This proves the order was committed,
// not just that POST /orders returned a success object.
const cart = await api.get(`/cart/${userId}`);
expect(cart.data.items).toEqual([]);
expect(cart.data.subtotal).toBe('0');

// Third-order: Cross-functional verification via admin API
// Proves the audit pipeline processed the event correctly.
const adminClient = createAdminClient();
const auditLog = await adminClient.get(`/audit/${orderResponse.data.orderId}`);
expect(auditLog.data.event).toBe('ORDER_PLACED');
expect(auditLog.data.timestamp).toBeDefined();

// Third-order: Cross-endpoint consistency
// The order summary endpoint must agree with the order creation response.
const summary = await api.get(`/orders/${orderResponse.data.orderId}/summary`);
expect(summary.data.total).toBe(orderResponse.data.total);

Why this matters for agents:

  • Primary assertion fails: Input validation, routing, or controller logic broken
  • Second-order fails: Controller works but persistence or downstream effects broken
  • Third-order fails: Core system works but observability, audit, or cross-endpoint consistency broken

Each failure mode points the agent to a specific subsystem to investigate.

Anti-Pattern

Don't skip second and third-order assertions "to save time." Each assertion layer catches a distinct class of failure that the previous layer cannot. Primary assertions verify the happy path; second-order assertions verify persistence and side effects; third-order assertions verify cross-functional consistency. Skipping a layer means accepting a blind spot in your diagnostic signal — a bug in that layer will surface silently in production, not in your tests.

Don't implement any assertion layer by querying databases directly. Use the API, even if it's an admin-only API. Tests should interact like users (or admins), not like developers with database access. Every assertion must go through a public API endpoint — the stack test has no direct database, Redis, or internal service access.

Don't make assertions conditional on earlier ones passing. All layers should run and report independently. If the primary assertion fails, still check second and third — you might learn that the core logic works but response serialization is broken.

Cross-References

Back to L1 Overview | Previous: Pattern 1.1 | Next: Pattern 1.3