feat(metrics): Add Prometheus metrics integration for agent monitoring

[KirobotDev]

Summary

This PR adds Prometheus metrics integration to the OpenAI Agents SDK, enabling users to monitor agent performance in production environments.

Changes

• New module src/agents/metrics/ with:
  • PrometheusMetrics - Collector for LLM latency, token usage, errors, run counts/durations
  • MetricsHooks - RunHooks implementation for automatic metrics collection
  • Global functions enable_metrics(), get_metrics(), disable_metrics()
• Optional dependency prometheus-client>=0.21.0 added to pyproject.toml
• Example examples/metrics/prometheus_endpoint.py - FastAPI server with /metrics endpoint
• Tests tests/test_metrics.py - Comprehensive test coverage

Metrics Exposed

Metric	Type	Description
agents_llm_latency_seconds	Histogram	LLM API call latency
agents_tokens_total	Counter	Token usage (input/output)
agents_errors_total	Counter	Error count by type
agents_runs_total	Counter	Run count by agent/status
agents_run_duration_seconds	Histogram	Run duration
agents_turns_total	Counter	LLM turn count
agents_tool_executions_total	Counter	Tool execution count
agents_tool_latency_seconds	Histogram	Tool execution latency

Usage

from agents import Agent, Runner
from agents.metrics import PrometheusMetrics, enable_metrics, MetricsHooks

metrics = PrometheusMetrics()
enable_metrics(metrics)

agent = Agent(name="assistant", instructions="Help")
result = Runner.run_sync(agent, "Hello", hooks=[MetricsHooks()])

[seratch]

Thanks for sharing this, and I like the idea using the hooks this way. We don't plan to have this as part of the core SDK, but please feel free to share it as your own example and/or package for other developers!

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 084a14d86b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Match on_tool_start signature to RunHooks

RunHooks.on_tool_start is invoked with (context, agent, tool), but this override requires an extra input_data argument, so any run that uses MetricsHooks and executes a tool will raise a TypeError before the tool call completes. This makes metrics hooks unsafe for agents that use tools.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Rename lifecycle handlers to actual RunHooks callbacks

These handlers are declared as on_start/on_end (and on_error below), but the runner dispatches RunHooks via on_agent_start/on_agent_end and has no on_error callback. As a result, run start/end/error metrics are never emitted through MetricsHooks, so the integration silently misses core counters and durations.

Useful? React with 👍 / 👎.

[chatgpt-codex-connector]

Pass a RunHooks instance instead of a list in example

Runner.run expects hooks to be a single RunHooks object, not a list, so passing hooks=[hooks] will fail when the runner tries to call hook methods on the list. In this example, /solve and /chat requests will return errors instead of running the agent.

Useful? React with 👍 / 👎.

[KirobotDev]

ok I will do that thank you for your quick response :)