Skip to content

feat: instrument evals usage events via openadapt-telemetry#108

Merged
abrichr merged 3 commits into
mainfrom
feat/posthog-telemetry-instrumentation
Mar 5, 2026
Merged

feat: instrument evals usage events via openadapt-telemetry#108
abrichr merged 3 commits into
mainfrom
feat/posthog-telemetry-instrumentation

Conversation

@abrichr
Copy link
Copy Markdown
Member

@abrichr abrichr commented Mar 5, 2026

Summary

  • add thin openadapt_evals.telemetry wrapper over openadapt_telemetry.posthog
  • instrument benchmark runner with agent_run, agent_run_completed, and action_executed
  • instrument oa evals run/mock entrypoints and record_waa_demos.py (demo_recorded)
  • add dependency on openadapt-telemetry>=0.2.0

Validation

  • ruff check on openadapt_evals/telemetry.py, openadapt_evals/benchmarks/runner.py, openadapt_evals/cli/main.py
  • python bytecode compile check for those files + scripts/record_waa_demos.py

Notes

  • this PR assumes openadapt-telemetry release with PostHog support is published first

@abrichr abrichr merged commit bc0fe75 into main Mar 5, 2026
1 check passed
@abrichr abrichr deleted the feat/posthog-telemetry-instrumentation branch March 5, 2026 20:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant