feat(ai): add $ai_completion_id, $ai_system_fingerprint, $ai_request_id to OpenAI events

[johnsykim]

Summary

• Adds three new auto-captured properties to $ai_generation events for OpenAI and Azure OpenAI wrappers:
  • $ai_completion_id — response.id (e.g. chatcmpl-xxx, resp_xxx)
  • $ai_system_fingerprint — response.system_fingerprint
  • $ai_request_id — response._request_id (from OpenAI SDK's x-request-id header)
• Enables direct correlation between PostHog events and OpenAI's Logs dashboard (platform.openai.com/logs/{completion_id})
• Covers non-streaming, streaming, and Responses API (create + parse) for both OpenAI and Azure OpenAI

Test plan

• Verify $ai_completion_id, $ai_system_fingerprint, $ai_request_id appear in captured event properties for non-streaming Chat Completions
• Verify $ai_completion_id, $ai_system_fingerprint appear for streaming Chat Completions
• Verify $ai_completion_id appears for Responses API (create + parse)
• Verify properties are omitted (not set to undefined/null) when not present in the response
• Existing tests pass (added assertions for new properties in openai.test.ts)

Background & motivation

Upstream: Add $ai_completion_id to @posthog/ai OpenAI Wrapper

The @posthog/ai package wraps the OpenAI SDK and auto-emits $ai_generation events to PostHog. However, it does not capture the OpenAI response ID (response.id), which is the primary key used in OpenAI's Logs dashboard at platform.openai.com/logs/{completion_id}.

This makes it impossible to navigate from a PostHog $ai_generation event directly to the corresponding OpenAI log entry.

Why This Matters

• Debugging: When investigating a cost spike or error, engineers need to go from PostHog analytics → OpenAI's detailed log (which shows full prompt/response, token breakdown, latency).
• Correlation: Without response.id, there is no way to join PostHog events with OpenAI's own logging system.
• Support: OpenAI support tickets reference x-request-id — having this in PostHog makes it easy to file tickets with the right correlation ID.

Current Workaround

We use a multi-hop join via entity IDs:

1. Find PostHog event by entity ID (e.g., rolePlayId)
2. Query our ProviderCosts database table by the same entity ID
3. Extract completionId from the context JSON field
4. Navigate to platform.openai.com/logs/{completionId}

This only works for the ~6 call sites that write to ProviderCosts with a completionId. It does not scale to all call sites.

References

• OpenAI Logs dashboard: https://platform.openai.com/logs?api=chat-completions
• OpenAI Completion ID format: chatcmpl-{base62} (Chat Completions API), resp_{base62} (Responses API)
• OpenAI SDK _request_id: automatically attached from the x-request-id response header by the openai npm package

[vercel]

@johnsykim is attempting to deploy a commit to the PostHog Team on Vercel.

A member of the Team first needs to authorize it.

[greptile-apps]

Important Files Changed

Filename	Overview
packages/ai/src/utils.ts	Adds completionId, systemFingerprint, requestId to SendEventToPosthogParams and uses truthy spreads to include them in event properties only when present.
packages/ai/src/openai/index.ts	Extracts completion metadata from streaming chunks and non-streaming results; passes completionId/systemFingerprint for streaming, and all three for non-streaming paths.
packages/ai/src/openai/azure.ts	Mirrors index.ts changes for Azure OpenAI; correctly handles streaming, non-streaming, and Responses API paths.
packages/ai/tests/openai.test.ts	Adds assertions for the three new properties in non-streaming and streaming completions tests, and $ai_completion_id for responses parse; missing $ai_request_id positive-case coverage for Responses API paths.
.changeset/ai-completion-id.md	Correct minor-level changeset for @posthog/ai package.

Prompt To Fix All With AI

This is a comment left during a code review.
Path: packages/ai/tests/openai.test.ts
Line: 638

Comment:
**Missing `$ai_request_id` coverage for Responses API paths**

The PR description's test plan says "Verify `$ai_completion_id` appears for Responses API (create + parse)" but only the `parse` path is tested here. The `responses.create` non-streaming path (tested at ~line 1619 for web search) has no assertion for `$ai_completion_id`.

Additionally, the implementation adds `requestId: (result as any)._request_id` for both the `responses.create` and `responses.parse` paths (`index.ts` lines 557–561 and 630–634, `azure.ts` lines 489–493 and 558–560), but there is no test verifying that `$ai_request_id` is captured when `_request_id` is present on a Responses API result. The `mockOpenAiParsedResponse` does not set `_request_id`, so the positive case is never exercised.

Consider adding `_request_id` to `mockOpenAiParsedResponse` and asserting `$ai_request_id` here (and in the `responses.create` web-search test, or in a dedicated `responses.create` test):

```suggestion
    expect(properties['$ai_completion_id']).toBe('test-parsed-response-id')
    expect(properties['$ai_request_id']).toBe('req_test-parsed-request-id')
```

(After also adding `_request_id: 'req_test-parsed-request-id'` to `mockOpenAiParsedResponse`.)

How can I resolve this? If you propose a fix, please make it concise.

_{Reviews (3): Last reviewed commit: "fix: address Greptile review feedback" | Re-trigger Greptile}

[johnsykim]

@greptile review

[github-actions]

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.

[johnsykim]

@haacked @danielbachhuber Sorry for random tagging (please tell me if there is a better PostHog POC to reach out to). What's the best way to get PostHog team's attention on this PR? Is there a certain review and release process I have to follow?

[danielbachhuber]

@johnsykim Hi! I'm no longer with PostHog but, based on the history of the files, @richardsolomou, @carlos-marchal-ph, or @Radu-Raicea might be able to give you a review.

[github-actions]

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.

[richardsolomou]

Thanks for putting this together — the motivation (correlating PostHog events to OpenAI's logs dashboard) is a real pain point and the implementation is clean. Chatted with the team about the schema shape, and we'd like to adjust the approach before merging.

Design ask

1. Move OpenAI-specific fields under a $ai_provider_metadata blob — $ai_* has been provider-agnostic so far, and system_fingerprint / request_id are OpenAI-only concepts. Rather than living at the top level, they should go under a new $ai_provider_metadata property so each provider wrapper can surface its own metadata without polluting the shared namespace. Something like:

   $ai_provider_metadata: {
     system_fingerprint: '...',
     request_id: '...',
   }

   This also sets the pattern for Anthropic / Gemini wrappers to follow later.
2. Keep $ai_completion_id at the top level — response IDs generalize cleanly (Anthropic and Gemini both have equivalents), so this one belongs in the shared schema.

On the app side: we think rendering a link to the OpenAI log from the event inspector is valuable and we'll pick that up separately once this is merged!

Blocking

3. Build fails with TS2353 on packages/ai/tests/openai.test.ts:292 — _request_id isn't a public property of ChatCompletion, so the literal is rejected under strict TS. pnpm build fails on this branch and passes on main. CI for external PRs only runs Wiz/Graphite/Vercel, which is why this slipped past. Easiest fix:

   let mockOpenAiChatResponse: ChatCompletion & { _request_id?: string } = {} as ChatCompletion

Suggestions

4. Duplicated (result as any)._request_id pattern — appears 6 times across packages/ai/src/openai/index.ts and packages/ai/src/openai/azure.ts. A small helper (extractRequestId(result)) would consolidate the cast and the explanatory comment in one place.
5. Streaming paths don't capture requestId — only completionId / systemFingerprint are pulled from chunks. The OpenAI SDK exposes _request_id on the aggregated stream response (via .withResponse()), so there's room to capture it for streams too. Fine as a follow-up.

Let me know if anything's unclear — happy to answer questions as you work through the restructure.

[github-actions]

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.

[johnsykim]

Thanks a lot @richardsolomou. I'll raise a revision sometime soon.

[github-actions]

This PR hasn't seen activity in a week! Should it be merged, closed, or further worked on? If you want to keep it open, post a comment or remove the stale label – otherwise this will be closed in another week.

[carlos-marchal-ph]

Hi @johnsykim, taking this over from @richardsolomou! I agree with Richard's review. I'm going to mark this as a draft in the meantime, un-draft it yourself when it's ready for us again, and we'll get pinged to take a second look :)