Skip to content

Add optimization-insights reference for Foundry agent trace skill#1943

Draft
xiaomi7732 wants to merge 2 commits intomicrosoft:mainfrom
xiaomi7732:dev/saars/add-optimization-insgihts
Draft

Add optimization-insights reference for Foundry agent trace skill#1943
xiaomi7732 wants to merge 2 commits intomicrosoft:mainfrom
xiaomi7732:dev/saars/add-optimization-insgihts

Conversation

@xiaomi7732
Copy link
Copy Markdown
Member

@xiaomi7732 xiaomi7732 commented Apr 17, 2026

What

Adds optimization-insights.md to the trace skill's references/ folder. This reference guides the agent through calling the Agent Insights API to surface actionable optimization insights (latency, token usage, errors, tool failures, evaluations) from Application Insights telemetry.

What's in the file

The reference covers a 6-step workflow:

  1. Resolve Parameters — Required (subscription, resource group, component name from App Insights resource ID) and optional (agent name, project ID) parameters, with a clear Required/Optional distinction
  2. Acquire Access Tokenaz account get-access-token with the correct https://ai.azure.com audience
  3. Call the Insights Endpoint — Broad-first query (no agent/project filters), with instructions to append optional filters to narrow. Includes full query parameter reference and optional custom thresholds
  4. Handle Empty Results — 3-step fallback when totalCount is 0: remove filters → widen time range → verify telemetry exists via Search Traces. Includes guidance on discovering actual agent names from metadata.agentName in results (which may differ from agent-metadata.yaml)
  5. Present Insights — Display grouped by severity (Critical → Warning → Improvement) with drill-down links
  6. Drill Into Specific Insights — Routes each insight type (Error, Latency, Token, Evaluation, Tool) to the appropriate trace reference

Also includes an error handling table covering empty results, auth errors, date format issues, and 404s, plus a note about the eastus2euap region limitation.

Design decisions

  • Broad-first querying: The API's agent and projectId parameters are optional, and filtering too aggressively can return 0 results when the metadata agent name doesn't match what telemetry records (e.g., hosted agents emit <name>:<version> or internal IDs). The workflow starts unfiltered and narrows on request.
  • Empty-results fallback: Rather than treating 0 results as a dead end, the skill guides through progressive relaxation of filters and time ranges before concluding there's no data.

Affected files

  • plugin/skills/microsoft-foundry/foundry-agent/trace/references/optimization-insights.md (new file)

Example:

Screenshot 2026-04-17 152456 Screenshot 2026-04-17 153253

…mpty-results handling

- Make agent name and projectId optional filters instead of prerequisites
- Start with broad query (no filters) then narrow, matching trace.md conventions
- Add Step 4 (Handle Empty Results) with fallback sequence: remove filters,
  widen time range, verify telemetry exists
- Document agent name mismatch between metadata and telemetry
- Add totalCount: 0 scenarios to Error Handling table

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings April 17, 2026 22:43
@xiaomi7732 xiaomi7732 marked this pull request as draft April 17, 2026 22:43
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a new trace-skill reference that guides users through calling the Foundry Agent Insights API (backed by App Insights telemetry) to surface actionable optimization recommendations, and wires it into the trace skill’s entry points/triggers.

Changes:

  • Expanded trace.md routing/trigger phrases to include “optimization insights” and added an entry-point link to the new reference.
  • Added optimization-insights.md reference describing a broad-first workflow (token acquisition, insights call, empty-result fallbacks, and drill-down routing).

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File Description
plugin/skills/microsoft-foundry/foundry-agent/trace/trace.md Adds “optimization insights” intent coverage and links to the new reference from the entry-points table.
plugin/skills/microsoft-foundry/foundry-agent/trace/references/optimization-insights.md New step-by-step reference for retrieving and presenting optimization insights and routing to existing trace drill-down references.

Start with a **broad query** (no agent or project filter):

```powershell
$URI = "https://eastus2euap.api.azureml.ms/notification/v1-beta1/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/microsoft.insights/components/<component-name>/:insights?startDateTimeUtc=<start>"
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In the sample URI, the provider segment is providers/microsoft.insights. Elsewhere (and in actual App Insights ARM IDs) this is typically providers/Microsoft.Insights. Using the canonical casing here will better match the source resource ID and reduce copy/paste confusion (and potential 404s if this endpoint is case-sensitive).

Suggested change
$URI = "https://eastus2euap.api.azureml.ms/notification/v1-beta1/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/microsoft.insights/components/<component-name>/:insights?startDateTimeUtc=<start>"
$URI = "https://eastus2euap.api.azureml.ms/notification/v1-beta1/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/Microsoft.Insights/components/<component-name>/:insights?startDateTimeUtc=<start>"

Copilot uses AI. Check for mistakes.
Start with a **broad query** (no agent or project filter):

```powershell
$URI = "https://eastus2euap.api.azureml.ms/notification/v1-beta1/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/microsoft.insights/components/<component-name>/:insights?startDateTimeUtc=<start>"
Copy link

Copilot AI Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The example uses startDateTimeUtc=<start> but <start> isn’t defined elsewhere and doesn’t match the parameter name in the table. Consider renaming the placeholder to something explicit (e.g., <startDateTimeUtc>) and/or showing a concrete ISO 8601 example (and optionally endDateTimeUtc) so the request can be copy/pasted without guessing the format.

Suggested change
$URI = "https://eastus2euap.api.azureml.ms/notification/v1-beta1/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/microsoft.insights/components/<component-name>/:insights?startDateTimeUtc=<start>"
$URI = "https://eastus2euap.api.azureml.ms/notification/v1-beta1/subscriptions/<subscription-id>/resourceGroups/<resource-group>/providers/microsoft.insights/components/<component-name>/:insights?startDateTimeUtc=2025-01-01T00:00:00Z&endDateTimeUtc=2025-01-02T00:00:00Z"

Copilot uses AI. Check for mistakes.
The reference file described trace-derived telemetry analysis (latency
outliers, token anomalies, trends) rather than optimization actions.
Rename to trace-insights to better reflect the actual API behavior.

- Rename optimization-insights.md -> trace-insights.md
- Update title to 'Trace Insights - Surface Agent Health from Telemetry'
- Update entry point and trigger phrases in trace.md

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants