Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .github/CODEOWNERS
Original file line number Diff line number Diff line change
Expand Up @@ -434,6 +434,7 @@
/packages/openai/data_stream/moderations @elastic/obs-infraobs-integrations
/packages/openai/data_stream/rate_limits @elastic/obs-infraobs-integrations
/packages/openai/data_stream/vector_stores @elastic/obs-infraobs-integrations
/packages/openai_chatgpt_enterprise @elastic/security-service-integrations
/packages/opencanary @elastic/security-service-integrations @elastic/sit-crest-contractors
/packages/oracle @elastic/obs-infraobs-integrations
/packages/oracle_otel @elastic/obs-infraobs-integrations
Expand Down
3 changes: 3 additions & 0 deletions packages/openai_chatgpt_enterprise/_dev/build/build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
dependencies:
ecs:
reference: git@v9.4.0
127 changes: 127 additions & 0 deletions packages/openai_chatgpt_enterprise/_dev/build/docs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,127 @@
# OpenAI ChatGPT Enterprise Integration for Elastic

## Overview

[OpenAI ChatGPT Enterprise](https://openai.com/enterprise) is the enterprise offering of ChatGPT, giving organizations administrative controls, security, and compliance capabilities for their use of ChatGPT and Codex. The OpenAI Compliance Logs Platform exposes an API that lets enterprises export compliance logs of activity across their workspace or organization, including application authentication activity such as connecting (linking) and disconnecting (unlinking) apps and connectors.

This integration for Elastic allows you to collect ChatGPT Enterprise compliance logs using the OpenAI Compliance Logs Platform API, then visualize the data in Kibana.

### Compatibility

This integration collects data from the [OpenAI Compliance Logs Platform API](https://help.openai.com/en/articles/9261474-compliance-api-for-enterprise-customers) and requires a ChatGPT Enterprise plan with the Compliance Logs Platform enabled.

### How it works

This integration periodically queries the OpenAI Compliance Logs Platform API to retrieve application authentication logs. Collection can be scoped to a single **workspace** or an entire **organization**, and follows a two-step (chained) flow:

1. The integration calls the list endpoint (`GET /v1/compliance/{workspaces|organizations}/{resource_id}/logs`) with the `event_type`, and paginates forward using the `last_end_time` cursor and `has_more` flag returned by the API. This returns metadata for each available log file.
2. For each listed file, the integration downloads its contents (`GET /v1/compliance/{workspaces|organizations}/{resource_id}/logs/{log_file_id}`). This endpoint redirects to a signed download URL that serves the log file as JSON Lines, and each line is ingested as a separate event.

On the first run, logs are pulled back as far as the configured initial interval. From the second collection onward, each run resumes from the `last_end_time` cursor returned by the previous request, so events are collected without gaps or duplication.

> Note: OpenAI retains compliance logs for a limited window (up to 30 days). Configure the initial interval accordingly when first enabling the integration.

## What data does this integration collect?

This integration collects log messages of the following type:

- `Application Authentication Log`: Collect ChatGPT Enterprise `APP_AUTH_LOG` events, covering application authentication activity such as linking and unlinking apps and connectors (endpoints: `/v1/compliance/{workspaces|organizations}/{resource_id}/logs` and `/v1/compliance/{workspaces|organizations}/{resource_id}/logs/{log_file_id}`).

### Supported use cases

Bringing ChatGPT Enterprise application authentication activity into Elastic lets security, compliance, and platform teams search, correlate, and investigate app-connection activity in one place instead of moving between separate tools.

The **Application Authentication Log** data stream provides visibility into which apps and connectors are linked or unlinked, who performed the action, and the client and request context associated with it. Use it to audit connector lifecycle changes, monitor app-authorization activity, and surface anomalous or high-risk link/unlink actions to support security oversight and auditing.

## What do I need to use this integration?

### From Elastic

You need Elasticsearch for storing and searching your data and Kibana for visualizing and managing it. You can use our hosted Elasticsearch Service on Elastic Cloud, which is recommended, or self-manage the Elastic Stack on your own hardware.

### From OpenAI

To collect data through the OpenAI Compliance Logs Platform API, you need to provide the **Compliance API key** and the **workspace or organization ID** whose logs you want to collect. Authentication is handled using the Compliance API key, which serves as the required credential.

#### Retrieve the Compliance API key and resource ID:

1. Ensure your organization has a **ChatGPT Enterprise** plan with the **Compliance Logs Platform** enabled.
2. Request a **Compliance API key** from OpenAI and ensure it is authorized for enterprise logs (contact OpenAI support to grant the key the `chatgpt.enterprise.compliance_export.read` scope).
3. Identify the **workspace ID** or **organization ID** whose compliance logs you want to collect.
4. Copy both the **Compliance API key** and the **resource ID** and store them securely for use in the integration configuration.

See [Compliance API for enterprise customers](https://help.openai.com/en/articles/9261474-compliance-api-for-enterprise-customers) for more details.

## How do I deploy this integration?

This integration supports both Elastic Agentless-based and Agent-based installations.

### Agent-based deployment

Elastic Agent must be installed. For more details, check the Elastic Agent [installation instructions](docs-content://reference/fleet/install-elastic-agents.md). You can install only one Elastic Agent per host.

### Agentless deployment

Agentless deployments are only supported in Elastic Serverless and Elastic Cloud environments. Agentless deployments provide a means to ingest data while avoiding the orchestration, management, and maintenance needs associated with standard ingest infrastructure. Using an agentless deployment makes manual agent deployment unnecessary, allowing you to focus on your data instead of the agent that collects it.

For more information, refer to [Agentless integrations](https://www.elastic.co/guide/en/serverless/current/security-agentless-integrations.html) and [Agentless integrations FAQ](https://www.elastic.co/guide/en/serverless/current/agentless-integration-troubleshooting.html).

### Configure

1. In the top search bar in Kibana, search for **Integrations**.
2. In the search bar, type **OpenAI ChatGPT Enterprise**.
3. Select the **OpenAI ChatGPT Enterprise** integration from the search results.
4. Select **Add OpenAI ChatGPT Enterprise** to add the integration.
5. Configure the integration:

* To **Collect ChatGPT Enterprise compliance logs**, you'll need to:

- Configure the **URL** (default `https://api.chatgpt.com`) and **Compliance API key**.
- Set the **Scope** to `workspace` or `organization`.
- Set the **Workspace / Organization ID** to the resource whose compliance logs are collected.
- Adjust the integration configuration parameters if required, including the Initial Interval, Interval, HTTP Client Timeout etc. to enable data collection.

Check warning on line 82 in packages/openai_chatgpt_enterprise/_dev/build/docs/README.md

View workflow job for this annotation

GitHub Actions / Lint user-facing content

Elastic.Latinisms: Latin terms and abbreviations are a common source of confusion. Use 'and so on' instead of 'etc'.

6. Select **Save and continue** to save the integration.

### Validation

#### Dashboard populated

1. In the top search bar in Kibana, search for **Dashboards**.
2. In the search bar, type **OpenAI ChatGPT Enterprise**, and verify the dashboard information is populated.

## Troubleshooting

For help with Elastic ingest tools, check [Common problems](https://www.elastic.co/docs/troubleshoot/ingest/fleet/common-problems).

## Scaling

For more information on architectures that can be used for scaling this integration, check the [Ingest Architectures](https://www.elastic.co/docs/manage-data/ingest/ingest-reference-architectures) documentation.

## Reference

### App Auth Log

The `app_auth_log` data stream captures ChatGPT Enterprise `APP_AUTH_LOG` events (application authentication activity such as linking and unlinking apps and connectors).

#### App Auth Log fields

{{ fields "app_auth_log" }}

### Example event

#### App Auth Log

{{ event "app_auth_log" }}

### Inputs used

{{ inputDocs }}

### API usage

These APIs are used with this integration:

* Application Authentication Log:
* List log files (endpoint: `/v1/compliance/{workspaces|organizations}/{resource_id}/logs`)
* Download log file (endpoint: `/v1/compliance/{workspaces|organizations}/{resource_id}/logs/{log_file_id}`)
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
version: '3.8'
services:
openai_chatgpt_enterprise:
image: docker.elastic.co/observability/stream:v0.22.0
hostname: openai_chatgpt_enterprise
ports:
- 8090
volumes:
- ./files:/files:ro
environment:
PORT: '8090'
command:
- http-server
- --addr=:8090
- --config=/files/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,171 @@
rules:
# =========================================================================
# PHASE 1 — list log files (paginated). Returns file metadata only; the
# JSONL content is fetched per-file in phase 2 (see the /logs/{id} rules).
# =========================================================================

# --- list page 2 (requested with ?after=2026-07-09T12:00:00Z) ---
- path: /v1/compliance/workspaces/ws-example-001/logs
methods: ['GET']
query_params:
event_type: APP_AUTH_LOG
limit: 2
after: "2026-07-09T12:00:00Z"
request_headers:
Authorization:
- "Bearer xxxx"
responses:
- status_code: 200
headers:
Content-Type:
- 'application/json'
body: |-
{{ minify_json `
{
"data": [
{
"id": "eclf_c3",
"event_type": "APP_AUTH_LOG",
"end_time": "2026-07-09T13:00:00Z",
"file_name": "APP_AUTH_LOG_2026-07-09T13:00:00+00:00.jsonl",
"file_size": 512,
"file_sha256": "c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3c3"
}
],
"has_more": false,
"last_end_time": "2026-07-09T13:00:00Z"
}
`}}

# --- list page 1 (initial request; no ?after) ---
- path: /v1/compliance/workspaces/ws-example-001/logs
methods: ['GET']
query_params:
event_type: APP_AUTH_LOG
limit: 2
request_headers:
Authorization:
- "Bearer xxxx"
responses:
- status_code: 200
headers:
Content-Type:
- 'application/json'
body: |-
{{ minify_json `
{
"data": [
{
"id": "eclf_a1",
"event_type": "APP_AUTH_LOG",
"end_time": "2026-07-09T10:11:00Z",
"file_name": "APP_AUTH_LOG_2026-07-09T10:11:00+00:00.jsonl",
"file_size": 1024,
"file_sha256": "a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1a1"
},
{
"id": "eclf_b2",
"event_type": "APP_AUTH_LOG",
"end_time": "2026-07-09T10:12:00Z",
"file_name": "APP_AUTH_LOG_2026-07-09T10:12:00+00:00.jsonl",
"file_size": 768,
"file_sha256": "b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2b2"
}
],
"has_more": true,
"last_end_time": "2026-07-09T12:00:00Z"
}
`}}

# =========================================================================
# PHASE 2 — download each log file's JSONL body. The real API 307-redirects
# to a signed URL; the mock serves the JSON Lines content directly (200).
# Each line is one APP_AUTH_LOG event and is emitted as a separate document.
# =========================================================================

# --- download file eclf_a1 (2 events) ---
- path: /v1/compliance/workspaces/ws-example-001/logs/eclf_a1
methods: ['GET']
request_headers:
Authorization:
- "Bearer xxxx"
responses:
- status_code: 200
headers:
Content-Type:
- 'application/x-ndjson'
body: |-
{{ minify_json `
{
"event_id": "7b3e4de1-2219-4d6c-9d6e-8f4d40c7c001",
"type": "APP_AUTH_LOG",
"timestamp": "2026-07-09T10:10:00.000000Z",
"principal": { "id": "be545252-ad04-4cfa-9ca5-deca58416151", "type": "CHATGPT_WORKSPACE" },
"actor": { "type": "ACCOUNT_USER", "user_id": "user-AbCdEf1234567890", "user_email": "user@example.org" },
"app_id": "asdk_app_0123456789abcdef0123456789abcdef",
"link_id": "link_00112233445566778899aabbccddeeff",
"action": "link"
}
`}}
{{ minify_json `
{
"event_id": "7b3e4de1-2219-4d6c-9d6e-8f4d40c7c002",
"type": "APP_AUTH_LOG",
"timestamp": "2026-07-09T10:11:00.000000Z",
"principal": { "id": "be545252-ad04-4cfa-9ca5-deca58416151", "type": "CHATGPT_WORKSPACE" },
"actor": { "type": "ACCOUNT_USER", "user_id": "user-AbCdEf1234567890", "user_email": "user@example.org" },
"app_id": null,
"link_id": "link_00112233445566778899aabbccddeeff",
"action": "unlink"
}
`}}

# --- download file eclf_b2 (1 event) ---
- path: /v1/compliance/workspaces/ws-example-001/logs/eclf_b2
methods: ['GET']
request_headers:
Authorization:
- "Bearer xxxx"
responses:
- status_code: 200
headers:
Content-Type:
- 'application/x-ndjson'
body: |-
{{ minify_json `
{
"event_id": "7b3e4de1-2219-4d6c-9d6e-8f4d40c7c003",
"type": "APP_AUTH_LOG",
"timestamp": "2026-07-09T10:12:00.000000Z",
"principal": { "id": "be545252-ad04-4cfa-9ca5-deca58416151", "type": "CHATGPT_WORKSPACE" },
"actor": { "type": "ACCOUNT_USER", "user_id": "user-ZyXwVu0987654321", "user_email": "admin@example.org" },
"app_id": "connector_fedcba9876543210fedcba9876543210",
"link_id": "link_ffeeddccbbaa99887766554433221100",
"action": "link"
}
`}}

# --- download file eclf_c3 (1 event) ---
- path: /v1/compliance/workspaces/ws-example-001/logs/eclf_c3
methods: ['GET']
request_headers:
Authorization:
- "Bearer xxxx"
responses:
- status_code: 200
headers:
Content-Type:
- 'application/x-ndjson'
body: |-
{{ minify_json `
{
"event_id": "7b3e4de1-2219-4d6c-9d6e-8f4d40c7c004",
"type": "APP_AUTH_LOG",
"timestamp": "2026-07-09T13:00:00.000000Z",
"principal": { "id": "be545252-ad04-4cfa-9ca5-deca58416151", "type": "CHATGPT_WORKSPACE" },
"actor": { "type": "ACCOUNT_USER", "user_id": "user-ZyXwVu0987654321", "user_email": "admin@example.org" },
"app_id": null,
"link_id": "link_ffeeddccbbaa99887766554433221100",
"action": "unlink"
}
`}}
6 changes: 6 additions & 0 deletions packages/openai_chatgpt_enterprise/changelog.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
# newer versions go on top

Check notice on line 1 in packages/openai_chatgpt_enterprise/changelog.yml

View workflow job for this annotation

GitHub Actions / Lint user-facing content

Elastic.Versions: Use 'later versions' instead of 'newer versions' when referring to versions.
- version: '0.1.0'
changes:
- description: Initial release of the OpenAI ChatGPT Enterprise app_auth_log data stream.
type: enhancement
link: https://github.com/elastic/integrations/pull/20203
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{"event_id":"7b3e4de1-2219-4d6c-9d6e-8f4d40c7c001","type":"APP_AUTH_LOG","principal":{"id":"be545252-ad04-4cfa-9ca5-deca58416151","type":"CHATGPT_WORKSPACE"},"actor":{"type":"ACCOUNT_USER","user_id":"user-AbCdEf1234567890","user_email":"user@example.org"},"timestamp":"2026-07-09T20:33:11.112450Z","app_id":"asdk_app_0123456789abcdef0123456789abcdef","link_id":"link_00112233445566778899aabbccddeeff","action":"link"}
{"event_id":"7b3e4de1-2219-4d6c-9d6e-8f4d40c7c002","type":"APP_AUTH_LOG","principal":{"id":"be545252-ad04-4cfa-9ca5-deca58416151","type":"CHATGPT_WORKSPACE"},"actor":{"type":"ACCOUNT_USER","user_id":"user-ZyXwVu0987654321","user_email":"admin@example.org"},"timestamp":"2026-07-09T23:10:55.000000Z","app_id":"connector_fedcba9876543210fedcba9876543210","link_id":"link_ffeeddccbbaa99887766554433221100","action":"link"}
{"event_id":"7b3e4de1-2219-4d6c-9d6e-8f4d40c7c003","type":"APP_AUTH_LOG","principal":{"id":"be545252-ad04-4cfa-9ca5-deca58416151","type":"CHATGPT_WORKSPACE"},"actor":{"type":"ACCOUNT_USER","user_id":"user-AbCdEf1234567890","user_email":"user@example.org"},"timestamp":"2026-07-10T08:15:02.500000Z","app_id":null,"link_id":"link_00112233445566778899aabbccddeeff","action":"unlink"}
Loading
Loading