Agent session streaming public preview (#61981)

Co-authored-by: amrithss <89491847+amrithss@users.noreply.github.com>
Co-authored-by: Jenni C <97056108+dihydroJenoxide@users.noreply.github.com>
Co-authored-by: mc <42146119+mchammer01@users.noreply.github.com>

Author: 4 people

content/admin/monitoring-activity-in-your-enterprise/reviewing-audit-logs-for-your-enterprise/streaming-the-audit-log-for-your-enterprise.md

@@ -26,6 +26,8 @@ Streaming audit log data has these benefits:
 * **Data continuity**. If you pause a stream, it retains a buffer for seven days, so there is no data loss for the first week. If the stream remains paused for more than seven days, it will resume from a point one week prior to the current time. If paused for three weeks or more, the stream won't retain any data and will start anew from the current timestamp.{% endif %}
 * **Data retention**. Keep your exported audit logs and Git events data as long as you need to.
 
+You can also stream agent session activity from {% data variables.product.prodname_copilot %} to the same destination you use for other enterprise audit events. This is currently in {% data variables.release-phases.public_preview %}. See [AUTOTITLE](/enterprise-cloud@latest/copilot/how-tos/administer-copilot/manage-for-enterprise/manage-agents/monitor-agentic-activity#tracking-agentic-activity-in-your-enterprise-through-the-audit-log) in the {% data variables.product.prodname_ghe_cloud %} documentation.
+
 You can set up{% ifversion pause-audit-log-stream %}, pause,{% endif %} or delete a stream at any time. The stream exports audit and Git events data for all of the organizations in your enterprise, for activity from the time the stream is enabled onwards.
 
 All streamed audit logs are sent as compressed JSON files. The filename format is in`YYYY/MM/HH/MM/<uuid>.json.gz`.
@@ -57,6 +59,7 @@ To set up the audit log stream, follow the instructions for your provider:
 * [Azure Event Hubs](#setting-up-streaming-to-azure-event-hubs)
 * [Datadog](#setting-up-streaming-to-datadog)
 * [Google Cloud Storage](#setting-up-streaming-to-google-cloud-storage)
+* [Microsoft Purview](#setting-up-streaming-to-microsoft-purview) (Copilot agent session events only)
 * [Splunk](#setting-up-streaming-to-splunk)
 
 {% ifversion ghec %}
@@ -265,6 +268,20 @@ To set up streaming to Google Cloud Storage, create a service account in Google
 1. To verify that {% data variables.product.prodname_dotcom %} can connect and write to the Google Cloud Storage bucket, click **Check endpoint**.
 {% data reusables.enterprise.verify-audit-log-streaming-endpoint %}
 
+### Setting up streaming to Microsoft Purview
+
+{% data reusables.copilot.agent-session-streaming-availability-note %}
+
+Microsoft Purview only supports streaming {% data variables.product.prodname_copilot_short %} agent session events. For more information on Microsoft Purview, see [Learn about the Microsoft Purview portal](https://learn.microsoft.com/en-us/purview/purview-portal) in the Microsoft documentation.
+
+To set up streaming to Microsoft Purview, configure streaming in {% data variables.product.github %}, then authorize access through Microsoft Entra.
+
+1. Configure streaming in {% data variables.product.github %}. See [Enabling audit log streaming of {% data variables.product.prodname_copilot_short %} agent session events](#enabling-audit-log-streaming-of-copilot-agent-session-events) below.
+{% data reusables.enterprise.navigate-to-log-streaming-tab %}
+1. Select the **Configure stream** dropdown and click **Microsoft Purview**.
+1. Click **Authorize with Entra**.
+1. When you're redirected to Microsoft Entra, add the {% data variables.product.github %} app and authorize it for your tenant.
+
 ### Setting up streaming to Splunk
 
 To stream audit logs to Splunk's HTTP Event Collector (HEC) endpoint, make sure that the endpoint is configured to accept HTTPS connections. See [Set up and use HTTP Event Collector in Splunk Web](https://docs.splunk.com/Documentation/Splunk/latest/Data/UsetheHTTPEventCollector) in the Splunk documentation.
@@ -328,3 +345,20 @@ To restart streaming, click **Resume stream**.
 1. Click **Save**.
 
 {% endif %}
+
+{% ifversion ghec %}
+
+## Enabling audit log streaming of {% data variables.product.prodname_copilot_short %} agent session events
+
+{% data reusables.copilot.agent-session-streaming-availability-note %}
+
+{% data reusables.enterprise-accounts.access-enterprise %}
+1. At the top of the page, click AI Controls 
+1. Under "Copilot", select **Enabled everywhere** for "Copilot Usage Records Streaming" 
+1. Configure a streaming destination. See [Setting up audit log streaming](#setting-up-audit-log-streaming).
+
+You can also retrieve {% data variables.product.prodname_copilot_short %} usage data through the REST API by selecting **Enabled everywhere** for "Copilot Usage Records API". See [AUTOTITLE](/rest/copilot/copilot-usage-metrics#get-copilot-usage-records-for-an-enterprise).
+
+For help interpreting the audit log events for agentic activity, see [AUTOTITLE](/copilot/reference/agentic-audit-log-events#streaming-audit-log-events).
+
+{% endif %}

content/copilot/concepts/agents/enterprise-management.md

@@ -44,6 +44,8 @@ An agent session encompasses an entire interaction with {% data variables.copilo
 
 Enterprise administrators can use AI Controls to view active and recent agent sessions, track audit log events, and search agentic activity in your enterprise using filters. See [AUTOTITLE](/copilot/how-tos/administer-copilot/manage-for-enterprise/manage-agents/monitor-agentic-activity) and [AUTOTITLE](/copilot/reference/agent-session-filters).
 
+For long-term retention and analysis, you can stream enterprise audit log data to supported destinations. For more information, see [AUTOTITLE](/enterprise-cloud@latest/admin/monitoring-activity-in-your-enterprise/reviewing-audit-logs-for-your-enterprise/streaming-the-audit-log-for-your-enterprise) and [AUTOTITLE](/copilot/reference/agentic-audit-log-events).
+
 For billing information on agent sessions, see [AUTOTITLE](/billing/concepts/product-billing/github-copilot-premium-requests#usage-by-copilot-cloud-agent).
 
 ## Third-party agents

content/copilot/how-tos/administer-copilot/manage-for-enterprise/manage-agents/monitor-agentic-activity.md

@@ -19,8 +19,18 @@ category:
 
 ## Tracking agentic activity in your enterprise through the audit log
 
+Track agentic activity on {% data variables.product.github %} or through streaming to an external destination.
+
+### Viewing agentic activity in the audit log on {% data variables.product.github %}
+
 {% data reusables.enterprise-accounts.access-enterprise %}
 {% data reusables.enterprise-accounts.ai-controls-tab %}
-1. To see a list of agentic activity in your enterprise over the last 180 days, at the bottom of the page, click {% octicon "log" aria-hidden="true" aria-label="log" %} **Audit logs**.
+1. At the bottom of the page, click {% octicon "log" aria-hidden="true" aria-label="log" %} **Audit logs**.
+
+### Streaming agentic activity from the audit log
+
+{% data reusables.copilot.agent-session-streaming-availability-note %}
+
+To enable streaming for {% data variables.product.prodname_copilot_short %} agent session events and configure a streaming destination from your enterprise audit log settings, see [AUTOTITLE](/enterprise-cloud@latest/admin/monitoring-activity-in-your-enterprise/reviewing-audit-logs-for-your-enterprise/streaming-the-audit-log-for-your-enterprise#enabling-audit-log-streaming-of-copilot-agent-session-events).
 
-For help interpreting the audit log events for agentic activity, see [AUTOTITLE](/copilot/reference/agentic-audit-log-events).
+In addition to streaming, you can also retrieve Copilot usage data through the REST API. See [AUTOTITLE](/rest/copilot/copilot-usage-metrics#get-copilot-usage-records-for-an-enterprise).

content/copilot/reference/agentic-audit-log-events.md

@@ -10,11 +10,33 @@ category:
   - Learn about Copilot
 ---
 
-You can apply the `actor:Copilot` filter to your enterprise audit log to view agentic activity over the last 180 days. The following key fields can help you interpret agentic events:
+You can apply the `actor:Copilot` filter to your enterprise audit log to view agentic activity over the last 180 days.
+
+The following key fields can help you interpret agentic events:
 
 | Field | Description | Example value |
 | --- | --- | --- |
 | `action` | The action performed by the agent, such as creating a pull request. | `pull_request.create` |
 | `actor_is_agent` | Indicates whether the actor is an AI agent. This will always be `true` for agentic audit log events. | `true` |
 | `agent_session_id` | A unique identifier linking to the specific agent session that generated the event. This field only appears when the event is the result of an agent session. | `012345a6-b7c8-9012-de3f-45gh678i9012` |
 | `user` | The person who initiated the agentic event. | `octocat` |
+
+## Streaming audit log events
+
+{% data reusables.copilot.agent-session-streaming-availability-note %}
+
+The table below shows the fields that are included in each streamed {% data variables.product.prodname_copilot_short %} API usage record.
+
+Each streamed record is a JSON object with the following properties:
+
+| Field | Type | Description |
+| --- | --- | --- |
+| `type` | `string` | The record type (`request` or `response`). |
+| `user_id` | `integer` | The ID of the user who made the request. |
+| `enterprise_id` | `integer` | The ID of the enterprise. |
+| `endpoint` | `string` | The {% data variables.product.prodname_copilot_short %} API endpoint the usage record interacted with. |
+| `body` | `string` | The request or response body (JSON-encoded string). |
+| `@timestamp` | `integer` | Milliseconds since Unix epoch. |
+| `truncated` | `boolean` | `true` when the `body` field has been trimmed to fit within the 1 MB document size limit. Omitted (treated as `false`) when the body is delivered in full. |
+| `event_id` | `string` | Unique event identifier for this streamed usage record. |
+| `github_request_id` | `string` | {% data variables.product.github %} request identifier associated with the {% data variables.product.prodname_copilot_short %} usage event. |

data/reusables/copilot/agent-session-streaming-availability-note.md

@@ -0,0 +1 @@
+>[!NOTE] This feature is in {% data variables.release-phases.public_preview %} and is available for enterprises that use {% data variables.product.prodname_emus %} and for enterprises that use {% data variables.product.prodname_ghe_cloud %} with data residency.