README.md

SVGBench Vercel Serverless Function

A Vercel serverless function that handles model calls for the SVGBench evaluation pipeline.

What it does

• Receives SVGBench evaluation requests via POST
• Makes model calls to Fireworks AI
• Returns model responses for local SVG evaluation
• Handles CORS and provides health check endpoint

Setup

Option 1: Local Development (Recommended)

1. Install Vercel CLI:

   npm install -g vercel

2. Navigate to this directory:

   cd eval_protocol/quickstart/svg_agent/vercel_svg_server

3. Create .env file with your API key (optional):

   cp .env.example .env
   # Edit .env and add your actual API key

   Note: The API key can be provided either:

   • In the request payload (automatically handled by RemoteRolloutProcessor)
   • In a local .env file (fallback option)

4. Start local development server:

   vercel dev

   Your function will be available at http://localhost:3000

Option 2: Production Deployment

1. Follow steps 1-2 above

2. Deploy to Vercel:

   vercel deploy

   Follow the prompts to:

   • Create a new project (or link existing)
   • Set project name (e.g., svgbench-server)

3. Deploy to production:

   vercel --prod

Note: The function receives the API key in the request payload (automatically handled by RemoteRolloutProcessor), but can also fall back to a local .env file if needed.

Usage

The function provides these endpoints:

POST /

Processes SVGBench evaluation requests.

Request format:

{
  "model": "fireworks_ai/accounts/fireworks/models/gpt-oss-120b",
  "messages": [
    {
      "role": "user",
      "content": "Your SVG generation prompt here"
    }
  ],
  "model_base_url": "https://api.fireworks.ai/inference/v1",
  "metadata": {
    "rollout_id": "some-unique-id"
  },
  "api_key": "your-fireworks-api-key",
  "completion_params": {
    "temperature": 0.8,
    "max_tokens": 32768
  }
}

Note: The api_key field is automatically populated by RemoteRolloutProcessor from your local FIREWORKS_API_KEY environment variable.

Response format:

{
  "status": "completed",
  "rollout_id": "some-unique-id",
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "SVG code generated by model"
      }
    }
  ]
}

GET /

Health check endpoint - returns server status.

Integration with Tests

Local Development

For local testing with vercel dev:

@evaluation_test(
    rollout_processor=RemoteRolloutProcessor(
        remote_base_url="http://localhost:3000",
        timeout_seconds=300,
    ),
    # ... other params
)

Production Deployment

For testing with deployed function:

@evaluation_test(
    rollout_processor=RemoteRolloutProcessor(
        remote_base_url="https://vercel-svg-server.vercel.app",
        timeout_seconds=300,
    ),
    # ... other params
)

Note: The RemoteRolloutProcessor automatically passes your local FIREWORKS_API_KEY environment variable to the Vercel function.

API Key Configuration

The function uses the following priority for API keys:

1. Request payload (req.api_key) - Automatically provided by RemoteRolloutProcessor
2. Local .env file (FIREWORKS_API_KEY) - Fallback for local development
3. Environment variable - Fallback for other deployment methods

This flexible approach works for both local development and production deployment.

Architecture

• Remote: Model calls (handled by this serverless function)
• Local: SVG extraction, rendering, evaluation, scoring (handled by test client)

This keeps the heavy SVG processing local while scaling model calls in the cloud.