# Adblock Compiler API

**Version:** 2.0.0

**Compiler-as-a-Service** for adblock filter lists. Transform, optimize, and combine filter lists from multiple sources with real-time progress tracking.

## Base URLs

- **Production**: `https://adblock-compiler.jayson-knight.workers.dev`
- **Local Development**: `http://localhost:8787`

## Features

- 🎯 Multi-Source Compilation
- ⚡ Performance (Gzip compression, caching, request deduplication)
- 🔄 Circuit Breaker with retry logic
- 📊 Visual Diff between compilations
- 📡 Real-time progress via SSE and WebSocket
- 🎪 Batch Processing
- 🌍 Universal (Deno, Node.js, Cloudflare Workers, browsers)

## Endpoints

### Metrics

#### `GET /api`
Returns API version, available endpoints, and usage examples.

#### `GET /metrics`
Returns aggregated metrics for the last 30 minutes.

---

### Compilation

#### `POST /compile`
Compile filter lists and return results as JSON. Results are cached for 1 hour. Supports request deduplication for concurrent identical requests.

**Request Body:**
```json
{
    "configuration": {
        "name": "My Filter List",
        "sources": [
            {
                "source": "https://example.com/filters.txt",
                "transformations": ["RemoveComments", "Validate"]
            }
        ],
        "transformations": ["Deduplicate", "RemoveEmptyLines"],
        "exclusions": ["||example.com^"],
        "inclusions": ["@@||safe.example.com^"]
    },
    "benchmark": true
}
```

**Response:**
```json
{
    "success": true,
    "rules": ["||ads.example.com^", "..."],
    "ruleCount": 1234,
    "compiledAt": "2026-01-14T05:00:00Z",
    "cached": false,
    "deduplicated": false,
    "metrics": {
        "totalDurationMs": 1234,
        "sourceCount": 1,
        "ruleCount": 1234
    }
}
```

**Responses:** `200` success, `429` rate limited, `500` error

---

#### `POST /compile/batch`
Compile multiple filter lists in parallel (max 10 per batch).

**Request Body:**
```json
{
    "requests": [
        { "id": "list1", "configuration": { "name": "List 1", "sources": [...] } },
        { "id": "list2", "configuration": { "name": "List 2", "sources": [...] } }
    ]
}
```

**Responses:** `200` results, `400` invalid request, `429` rate limited

---

### Streaming

#### `POST /compile/stream`
Compile with real-time progress via Server-Sent Events (SSE).

Streams events: `log`, `source:start`, `source:complete`, `source:error`, `transformation:start`, `transformation:complete`, `progress`, `result`, `done`, `error`, `diagnostic`, `cache`, `network`, `metric`.

See [[Streaming API|STREAMING_API]] for detailed documentation.

---

### Queue (Async)

#### `POST /compile/async`
Queue a compilation job for asynchronous processing. Returns immediately with a request ID.

**Request Body:**
```json
{
    "configuration": {
        "name": "My Filter List",
        "sources": [{ "source": "https://example.com/filters.txt" }]
    },
    "priority": "high"
}
```

**Response (202 Accepted):**
```json
{
    "success": true,
    "message": "Compilation job queued successfully",
    "requestId": "compile-1704931200000-abc123",
    "priority": "high"
}
```

Use `POST /compile` with the same configuration later to retrieve cached results.

#### `POST /compile/batch/async`
Queue multiple compilations for async processing.

#### `GET /queue/stats`
Returns queue health metrics and job statistics.

#### `GET /queue/results/{requestId}`
Retrieve results for a completed async compilation job.

See [[Queue Support|QUEUE_SUPPORT]] for detailed documentation.

---

### WebSocket

#### `GET /ws/compile`
Bidirectional WebSocket connection for real-time compilation with event streaming.

**Client → Server:** `compile`, `cancel`, `ping`

**Server → Client:** `welcome`, `pong`, `compile:started`, `event`, `compile:complete`, `compile:error`, `compile:cancelled`, `error`

**Features:**
- Up to 3 concurrent compilations per connection
- Automatic heartbeat (30s interval)
- Connection timeout (5 minutes idle)
- Session-based compilation tracking
- Cancellation support

See [[Streaming API|STREAMING_API]] for detailed WebSocket documentation.

---

## Schemas

### Configuration

```typescript
{
    name: string;           // Required
    description?: string;
    homepage?: string;
    license?: string;
    version?: string;
    sources: Source[];      // Required, non-empty
    transformations?: Transformation[];
    exclusions?: string[];          // Patterns: plain, *.wildcard, /regex/
    exclusions_sources?: string[];  // Files containing exclusion rules
    inclusions?: string[];
    inclusions_sources?: string[];
}
```

### Source

```typescript
{
    source: string;         // URL or file path — Required
    name?: string;
    type?: 'adblock' | 'hosts';
    transformations?: Transformation[];
    exclusions?: string[];
    inclusions?: string[];
}
```

### Transformation

Available transformations (applied in this order):

| Name | Description |
|------|-------------|
| `ConvertToAscii` | Convert internationalized domains to ASCII (Punycode) |
| `RemoveComments` | Remove comment lines (`!` and `#`) |
| `Compress` | Convert hosts format to adblock syntax |
| `RemoveModifiers` | Strip unsupported modifiers |
| `Validate` | Remove invalid/dangerous rules |
| `ValidateAllowIp` | Like Validate but keeps IP addresses |
| `Deduplicate` | Remove duplicate rules |
| `InvertAllow` | Convert blocking rules to allowlist |
| `RemoveEmptyLines` | Remove blank lines |
| `TrimLines` | Remove leading/trailing whitespace |
| `InsertFinalNewLine` | Add final newline |

### CompileResponse

```typescript
{
    success: boolean;
    rules?: string[];
    ruleCount?: number;
    compiledAt?: string;
    cached?: boolean;           // Served from cache
    deduplicated?: boolean;     // Request was deduplicated
    error?: string;
    metrics?: {
        totalDurationMs: number;
        sourceCount: number;
        ruleCount: number;
        transformationMetrics: TransformationMetric[];
    };
    previousVersion?: {         // For visual diff
        rules: string[];
        ruleCount: number;
        compiledAt: string;
    };
}
```

## Rate Limits

- **Per IP**: 60 requests/minute
- **Per endpoint**: 100 requests/minute
- **Streaming/WebSocket**: 10 requests/minute per IP
- **Async endpoints**: No rate limiting (queue handles backpressure)

Response on limit: `429 Too Many Requests` with `Retry-After` header.

## Examples

### cURL

```bash
# Simple compilation
curl -X POST https://adblock-compiler.jayson-knight.workers.dev/compile \
  -H "Content-Type: application/json" \
  -d '{
    "configuration": {
      "name": "My List",
      "sources": [{"source": "https://adguardteam.github.io/AdGuardSDNSFilter/Filters/filter.txt"}],
      "transformations": ["RemoveComments", "Deduplicate"]
    }
  }'

# Async compilation
curl -X POST https://adblock-compiler.jayson-knight.workers.dev/compile/async \
  -H "Content-Type: application/json" \
  -d '{"configuration": {"name": "My List", "sources": [...]}}'
```

### TypeScript

```typescript
const response = await fetch('https://adblock-compiler.jayson-knight.workers.dev/compile', {
    method: 'POST',
    headers: { 'Content-Type': 'application/json' },
    body: JSON.stringify({
        configuration: {
            name: 'My Filter List',
            sources: [{ source: 'https://example.com/filters.txt' }],
            transformations: ['RemoveComments', 'Deduplicate'],
        },
    }),
});

const { success, rules, ruleCount } = await response.json();
```

## OpenAPI Specification

A complete OpenAPI 3.0 specification is available:
- Endpoint: `GET /openapi.yaml`
- Interactive docs: `GET /api/docs`

See [[OpenAPI Support|OPENAPI_SUPPORT]] for details.

## Further Reading

- [[Quick Reference|QUICK_REFERENCE]] - Compact endpoint cheatsheet
- [[Streaming API|STREAMING_API]] - SSE and WebSocket documentation
- [[Queue Support|QUEUE_SUPPORT]] - Async compilation with Cloudflare Queues
- [[Client Libraries|clients]] - SDK examples for Python, TypeScript, Go