Skip to content

ARCHITECTURE.md

Latest commit

 

History

History
505 lines (411 loc) · 16.9 KB

ARCHITECTURE.md

File metadata and controls

505 lines (411 loc) · 16.9 KB
Raw

SynthoraAI API Gateway - Architecture

This document provides a comprehensive overview of the API Gateway architecture, design patterns, and technical decisions.

Table of Contents

Overview

The SynthoraAI API Gateway is built using modern architectural patterns and best practices to ensure:

  • High Availability: 99.9% uptime through redundancy and failover
  • Scalability: Horizontal scaling to handle increased load
  • Security: Multiple layers of security controls
  • Performance: Sub-100ms response times for cached requests
  • Observability: Comprehensive monitoring and logging

Technology Stack

Component Technology Purpose
Runtime Node.js 18+ JavaScript runtime
Language TypeScript 5.3 Type-safe development
Framework Express.js 4.18 Web application framework
Cache Redis 7.0 Caching and rate limiting
Documentation Swagger/OpenAPI API documentation
Logging Winston Structured logging
Testing Jest Unit and integration testing
Containerization Docker Deployment packaging
CI/CD GitHub Actions Automated testing and deployment

System Architecture

High-Level Architecture

┌──────────────────────────────────────────────────────────────┐
│                        Clients Layer                         │
│  (Frontend, Mobile Apps, Third-party Services)               │
└────────────────────────────┬─────────────────────────────────┘
                             │
                             ▼
┌──────────────────────────────────────────────────────────────┐
│                      Load Balancer                           │
│              (AWS ALB / Nginx / Cloudflare)                  │
└────────────────────────────┬─────────────────────────────────┘
                             │
                             ▼
┌──────────────────────────────────────────────────────────────┐
│                    API Gateway Cluster                       │
│  ┌────────────┐  ┌────────────┐  ┌────────────┐            │
│  │ Gateway 1  │  │ Gateway 2  │  │ Gateway N  │            │
│  │  (8080)    │  │  (8080)    │  │  (8080)    │            │
│  └─────┬──────┘  └─────┬──────┘  └─────┬──────┘            │
│        │                │                │                    │
│        └────────────────┴────────────────┘                   │
│                         │                                     │
│     ┌──────────────────────────────────────┐                │
│     │      Redis Cluster (Caching)         │                │
│     │  (Session, Cache, Rate Limiting)     │                │
│     └──────────────────────────────────────┘                │
└────────────────────────────┬─────────────────────────────────┘
                             │
                             ▼
         ┌───────────────────┴───────────────────┐
         │                                        │
         ▼                                        ▼
┌─────────────────┐                    ┌──────────────────┐
│  Microservices  │                    │   External APIs  │
│                 │                    │                  │
│  • Backend      │                    │  • Google AI     │
│  • Crawler      │                    │  • NewsAPI       │
│  • Newsletter   │                    │  • Resend        │
│  • Agentic AI   │                    │  • Pinecone      │
└─────────────────┘                    └──────────────────┘

Request Flow

1. Client Request
   ↓
2. Load Balancer
   ↓
3. API Gateway
   ├─→ Security Middleware
   │   ├─ CORS Check
   │   ├─ Helmet Security Headers
   │   └─ Rate Limiting
   ├─→ Authentication Middleware
   │   └─ JWT Validation
   ├─→ Request Validation
   │   └─ Input Sanitization
   ├─→ Cache Layer
   │   ├─ Check Redis Cache
   │   └─ Return if Hit
   ├─→ Circuit Breaker
   │   └─ Check Service Health
   ├─→ Proxy Service
   │   ├─ Route to Microservice
   │   ├─ Retry Logic
   │   └─ Response Transformation
   └─→ Response Middleware
       ├─ Cache Response
       ├─ Add Headers
       └─ Return to Client

Design Patterns

1. API Gateway Pattern

The gateway serves as a single entry point for all client requests, providing:

  • Unified Interface: Single API for multiple microservices
  • Request Routing: Intelligent routing based on path and headers
  • Protocol Translation: HTTP to various backend protocols
  • Request Aggregation: Combine multiple service calls

Implementation: See src/services/proxyService.ts

2. Circuit Breaker Pattern

Prevents cascading failures by monitoring service health:

// Circuit States
CLOSED  → Normal operation, requests flow through
OPEN    → Service failing, requests rejected immediately
HALF_OPEN → Testing if service recovered

Configuration:

  • Failure Threshold: 5 failures
  • Success Threshold: 2 successes
  • Timeout: 60 seconds
  • Reset Timeout: 30 seconds

Implementation: See src/utils/circuitBreaker.ts

3. Cache-Aside Pattern

Improves performance by caching frequently accessed data:

1. Check Cache → If Hit: Return
               → If Miss: Continue
2. Call Service
3. Store in Cache
4. Return to Client

Cache Strategy:

  • Articles List: 5 minutes TTL
  • Single Article: 10 minutes TTL
  • Related Articles: 30 minutes TTL
  • Bias Analysis: 1 hour TTL

Implementation: See src/middleware/cache.ts

4. Retry Pattern

Handles transient failures with exponential backoff:

Attempt 1: Immediate
Attempt 2: Wait 2 seconds
Attempt 3: Wait 4 seconds
Attempt 4: Wait 8 seconds

Implementation: See src/services/proxyService.ts

5. Rate Limiting Pattern

Protects against abuse with token bucket algorithm:

Standard: 100 requests / 15 minutes
Auth: 5 requests / 15 minutes
Public: 200 requests / 15 minutes

Implementation: See src/middleware/rateLimit.ts

Component Details

Authentication System

┌─────────────────────────────────────┐
│     Authentication Flow             │
├─────────────────────────────────────┤
│                                     │
│  1. Client sends credentials        │
│  2. Backend validates & returns JWT │
│  3. Client includes JWT in headers  │
│  4. Gateway validates JWT           │
│  5. Extracts user info              │
│  6. Forwards to backend             │
│                                     │
└─────────────────────────────────────┘

JWT Structure:

{
  "id": "user_id",
  "email": "user@example.com",
  "role": "admin|user",
  "iat": 1234567890,
  "exp": 1234654290
}

Caching Architecture

┌─────────────────────────────────────────┐
│          Redis Cache Structure           │
├─────────────────────────────────────────┤
│                                          │
│  cache:articles?page=1&limit=20         │
│  ├─ TTL: 300s                            │
│  └─ Value: {"success": true, ...}       │
│                                          │
│  cache:articles/:id                      │
│  ├─ TTL: 600s                            │
│  └─ Value: {"success": true, ...}       │
│                                          │
│  rl:192.168.1.1                          │
│  ├─ TTL: 900s                            │
│  └─ Value: 45 (request count)            │
│                                          │
└─────────────────────────────────────────┘

Circuit Breaker States

┌──────────────────────────────────────────────────┐
│            Circuit Breaker State Machine          │
├──────────────────────────────────────────────────┤
│                                                   │
│              ┌─────────────┐                      │
│              │   CLOSED    │                      │
│              │ (Normal)    │                      │
│              └──────┬──────┘                      │
│                     │                             │
│         Failures >= Threshold                     │
│                     │                             │
│                     ▼                             │
│              ┌─────────────┐                      │
│      ┌───────│    OPEN     │───────┐              │
│      │       │ (Failing)   │       │              │
│      │       └─────────────┘       │              │
│      │              │               │              │
│ Reset │       Timeout Elapsed       │ Fail        │
│ Timer │              │               │              │
│      │              ▼               │              │
│      │       ┌─────────────┐        │              │
│      └──────→│ HALF_OPEN   │────────┘              │
│              │ (Testing)   │                       │
│              └──────┬──────┘                       │
│                     │                              │
│         Success >= Threshold                       │
│                     │                              │
│                     ▼                              │
│              Back to CLOSED                        │
│                                                    │
└──────────────────────────────────────────────────┘

Security Architecture

Security Layers

  1. Network Layer

    • HTTPS/TLS 1.3 encryption
    • DDoS protection via Cloudflare
    • IP whitelisting for admin endpoints

  2. Application Layer

    • Helmet.js security headers
    • CORS policy enforcement
    • Input validation and sanitization
    • SQL/NoSQL injection prevention

  3. Authentication Layer

    • JWT with secure secret
    • Token expiration
    • Refresh token rotation
    • Session management

  4. Authorization Layer

    • Role-based access control (RBAC)
    • Resource-level permissions
    • Admin-only endpoints

  5. API Layer

    • Rate limiting
    • Request size limits
    • API key rotation
    • Service-to-service auth

Security Headers

{
  "Strict-Transport-Security": "max-age=31536000; includeSubDomains",
  "X-Content-Type-Options": "nosniff",
  "X-Frame-Options": "DENY",
  "X-XSS-Protection": "1; mode=block",
  "Content-Security-Policy": "default-src 'self'"
}

Scalability

Horizontal Scaling

The gateway is stateless and can be scaled horizontally:

1 instance → 100 RPS
2 instances → 200 RPS
N instances → N × 100 RPS

Auto-scaling Strategy

Metrics:

  • CPU > 70% → Scale up
  • Memory > 80% → Scale up
  • Request latency > 500ms → Scale up
  • CPU < 30% for 10min → Scale down

Configuration:

minReplicas: 2
maxReplicas: 10
targetCPUUtilization: 70
targetMemoryUtilization: 80

Database Scaling

Redis:

  • Master-slave replication
  • Redis Sentinel for HA
  • Redis Cluster for horizontal scaling

Monitoring & Observability

Metrics Collected

  1. Request Metrics

    • Total requests
    • Requests per second
    • Response times (p50, p95, p99)
    • Error rates

  2. Service Metrics

    • Circuit breaker states
    • Service availability
    • Success/failure rates
    • Retry counts

  3. Resource Metrics

    • CPU usage
    • Memory usage
    • Network I/O
    • Redis connections

  4. Business Metrics

    • Active users
    • Articles served
    • Cache hit rate
    • API usage per client

Logging Strategy

Log Levels:

  • error: Errors requiring immediate attention
  • warn: Warning conditions
  • info: Informational messages
  • debug: Detailed debugging information

Log Format:

{
  "timestamp": "2025-11-16T12:00:00.000Z",
  "level": "info",
  "service": "api-gateway",
  "message": "Request completed",
  "requestId": "req_123",
  "userId": "user_456",
  "method": "GET",
  "path": "/api/v1/articles",
  "statusCode": 200,
  "duration": 45
}

Alerts

Critical Alerts:

  • Service down > 1 minute
  • Error rate > 5%
  • Response time > 2 seconds
  • Circuit breaker OPEN

Warning Alerts:

  • CPU > 80%
  • Memory > 85%
  • Cache miss rate > 50%
  • Rate limit triggers > 100/min

Performance Optimization

Optimization Strategies

  1. Response Caching

    • Redis caching layer
    • Cache invalidation on updates
    • Conditional requests (ETags)

  2. Request Optimization

    • Compression (gzip, brotli)
    • Connection pooling
    • HTTP/2 support
    • Keep-alive connections

  3. Database Optimization

    • Redis connection pooling
    • Pipelining for batch operations
    • Lazy loading

  4. Code Optimization

    • Async/await for non-blocking I/O
    • Stream processing for large payloads
    • Efficient JSON parsing

Performance Targets

Metric Target Current
Response Time (p95) < 200ms ~150ms
Response Time (p99) < 500ms ~300ms
Throughput > 1000 RPS ~1200 RPS
Error Rate < 0.1% ~0.05%
Availability 99.9% 99.95%

Disaster Recovery

Backup Strategy

  1. Redis Backups

    • AOF (Append-Only File) enabled
    • RDB snapshots every 6 hours
    • Replication to standby

  2. Configuration Backups

    • Version controlled in Git
    • Encrypted secrets in vault
    • Infrastructure as Code

Recovery Procedures

  1. Service Failure

    • Circuit breaker opens
    • Traffic routed to healthy instances
    • Auto-recovery when healthy

  2. Redis Failure

    • Failover to replica
    • Cache warmup from backup
    • Graceful degradation without cache

  3. Complete Outage

    • Restore from backups
    • Traffic routed to DR region
    • RTO: 15 minutes
    • RPO: 5 minutes

Future Enhancements

  • GraphQL Gateway: Support for GraphQL queries
  • WebSocket Support: Real-time connections
  • Service Mesh Integration: Istio/Linkerd
  • Advanced Analytics: ML-based anomaly detection
  • Multi-region Deployment: Global load balancing
  • API Marketplace: Third-party API integrations

For questions or suggestions, please contact the SynthoraAI team.