Track: C — Architecture and Platform Strategy
The webhook gateway runs on port 8004. External sources send events to POST /api/v1/webhooks/{source}.
External source → POST /api/v1/webhooks/{source}
→ Signature validation (HMAC-SHA256)
→ Idempotency check (delivery_id)
→ Audit log (webhook_events table, regardless of outcome)
→ Publish to Kafka topic webhook.events.{source}
- Register a webhook source via admin API.
- Set
WEBHOOK_SIGNING_KEY_{SOURCE_UPPER}environment variable. - Send webhook with headers:
X-Delivery-Id,X-Webhook-Signature,Content-Type.
| Code | Meaning |
|---|---|
| 202 | Accepted |
| 400 | Bad request |
| 401 | Signature mismatch |
| 409 | Duplicate delivery ID |
| 413 | Payload too large (>10 MB) |
| 503 | Source not registered |
- Plain hex format
- Stripe-compatible signature format (v1, v2)
- In-memory cache (5-min TTL)
- Environment variable
- AWS Secrets Manager
- Fallback default
Every inbound attempt is stored in webhook_events table regardless of outcome:
-- Recent events
SELECT * FROM webhook_events ORDER BY created_at DESC LIMIT 20;
-- Only failures
SELECT * FROM webhook_events WHERE status = 'failed' ORDER BY created_at DESC;
-- Duplicate delivery IDs
SELECT delivery_id, COUNT(*) FROM webhook_events GROUP BY delivery_id HAVING COUNT(*) > 1;| Symptom | Cause | Fix |
|---|---|---|
| 401 Signature mismatch | Wrong key, re-serialization difference, algorithm mismatch | Manually verify HMAC: `echo -n "" |
| 409 Duplicate | Idempotency guard working correctly | If first delivery failed, use replay |
| 413 Payload too large | Exceeds 10 MB limit | Reduce payload size |
| No matching source | Source not registered in admin API | Register source first |
No dual-key validation period — brief maintenance window. Update WEBHOOK_SIGNING_KEY_{SOURCE} env var and restart the service.
Use the delivery_id to correlate events across logs:
grep "<delivery-id>" /var/log/webhook/*.log