judge

Judge Example (Direct Evaluation)

This example demonstrates how to use LaunchDarkly's judge functionality to evaluate specific input/output pairs directly, without an associated chat session.

Prerequisites

• Python 3.10 or higher
• Poetry installed
• A LaunchDarkly account and SDK key
• API keys for the provider you want to use (OpenAI, Bedrock, or Gemini)

Setup

1. Create the following config in your LaunchDarkly project. You can use a different key by setting the environment variable in your .env.

   • Create a Judge Config for evaluation. Default key: sample-ai-judge.

2. Create a .env file in this directory with the following variables:

   LAUNCHDARKLY_SDK_KEY=your-launchdarkly-sdk-key
   LAUNCHDARKLY_AI_JUDGE_KEY=sample-ai-judge
   OPENAI_API_KEY=your-openai-api-key
   

3. Install the required dependencies:

   poetry install

Run

poetry run judge