Skip to content

Commit 0823dd4

Browse files
arcappcirclenurdincerozcanovunc
authored
docs(AI-110): AI Insights and Build Insights Report documentation (#1251)
* AI-110: Add documentation for AI Insights and Build Insights Report * Update AI Insights documentation for the Build Insights Report * docs(AI-110): refine Build Insights Report wording and bulletize generate steps * docs(AI-110): italicize example prompts instead of quotes * docs(AI-110): punchier AI Insights intro sentence --------- Co-authored-by: nurdincer <nsuedadincer@gmail.com> Co-authored-by: Arc <ozcan@appcircle.io> Co-authored-by: ozcanovunc <ozcanovunc@outlook.com>
1 parent 72ab13d commit 0823dd4

6 files changed

Lines changed: 160 additions & 11 deletions

File tree

docs/appcircle-ai/ai-assistants/appcircle-claude-assistant.md

Lines changed: 8 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -7,21 +7,16 @@ sidebar_position: 3
77

88
# Appcircle Claude Assistant
99

10-
Appcircle provides an **Appcircle Claude Plugin** that connects Claude to the Appcircle platform, bundling the [Appcircle MCP Server](/appcircle-ai/appcircle-mcp-server) with Appcircle-aware skills so you can ask Claude about your Appcircle organization or about how to use Appcircle.
10+
The **Appcircle Claude Plugin** connects Claude to the Appcircle platform, bundling the [Appcircle MCP Server](/appcircle-ai/appcircle-mcp-server) with Appcircle-aware skills so you can ask Claude about your Appcircle organization or how to use Appcircle.
1111

12-
The plugin is open source: [appcircle-claude-plugin on GitHub](https://github.com/appcircleio/appcircle-claude-plugin).
13-
14-
## What's Included
12+
## How to Use Appcircle Claude Assistant
1513

16-
- **MCP Server**: registers `mcp.appcircle.io` as an MCP server named `appcircle`, exposing build, signing, distribution, publish, and reporting tools so Claude can query your organization. See [Appcircle MCP Server](/appcircle-ai/appcircle-mcp-server) for the full list of tools.
17-
- **Skills**: Appcircle-aware skills that extend Claude's capabilities. More skills are added over time and are automatically namespaced under `appcircle:`.
14+
The plugin registers the [Appcircle MCP Server](/appcircle-ai/appcircle-mcp-server) and the following Appcircle-aware skills, automatically namespaced under `appcircle:`:
1815

1916
| Skill | Purpose |
2017
|-------|---------|
2118
| `appcircle:doc-assistant` | Answers Appcircle questions using official sources (`docs.appcircle.io` and `appcircle.io`) |
22-
| `appcircle:build-insights-report` | Renders a visual Build Insights Report (health & trends, root cause, workflow quality, artifact health, queue time, and CI maturity) from the `get_build_insights_report` MCP tool |
23-
24-
## How to Use Appcircle Claude Assistant
19+
| `appcircle:build-insights-report` | Renders a visual Build Insights Report (health & trends, root cause, workflow quality, artifact health, queue time, and CI maturity) from the `get_build_insights_report` MCP tool. See [AI Insights](/appcircle-ai/ai-insights) for the full metric reference |
2520

2621
Claude Code and claude.ai install the plugin differently, so follow the steps for the surface you use. MCP tools are only available in Claude Code for now; installing the plugin in claude.ai gets you the `doc-assistant` skill, not the MCP tools.
2722

@@ -55,3 +50,7 @@ You can install this plugin's skills in claude.ai too. MCP tools remain Claude C
5550
3. Choose **Create plugin**, then **Add marketplace**.
5651
4. Enter the repository URL: `https://github.com/appcircleio/appcircle-claude-plugin`.
5752
5. Ask Claude an Appcircle question, for example "How do I set up automatic code signing for my iOS builds," and Claude will use the `doc-assistant` skill to answer.
53+
54+
To access the source code of this plugin, please use the following link:
55+
56+
https://github.com/appcircleio/appcircle-claude-plugin

docs/appcircle-ai/ai-insights.md

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,134 @@
1+
---
2+
title: AI Insights
3+
description: Understand the Appcircle Build Insights Report - every section and metric returned by the get_build_insights_report MCP tool, and how to generate the visual report with the Appcircle Claude Assistant.
4+
tags: [appcircle ai, ai insights, build insights]
5+
sidebar_position: 4
6+
---
7+
8+
import ContentRef from '@site/src/components/ContentRef';
9+
import Screenshot from '@site/src/components/Screenshot';
10+
11+
# AI Insights
12+
13+
Turn your build history into answers. AI Insights reads your CI data and generates the Build Insights Report: trends, root causes, and a maturity score at a glance.
14+
15+
## Build Insights Report
16+
17+
The **Build Insights Report** gives you a snapshot of your CI health: build trends, failure root causes, workflow quality, artifact health, queue time, and an overall maturity score. Use the [Appcircle Claude Assistant](/appcircle-ai/ai-assistants/appcircle-claude-assistant) to get a visual report, or use the [Appcircle MCP client](/appcircle-ai/appcircle-mcp-server) directly.
18+
19+
### How to Generate a Report
20+
21+
**With the Appcircle Claude Assistant**
22+
23+
- Ask in natural language, for example *give me a build insights report for last month* or *how is our CI maturity*.
24+
- The `appcircle:build-insights-report` skill makes one call to `get_build_insights_report` and renders the response as a report.
25+
26+
**With Any Other MCP Client**
27+
28+
- Call `get_build_insights_report` directly from any MCP-compatible client (Cursor, VS Code, Codex, and so on).
29+
- You get the same metrics as structured JSON, which you can read, feed into another tool, or visualize yourself.
30+
31+
### Report Sections
32+
33+
The report has six sections.
34+
35+
#### Maturity Assessment
36+
37+
A single CI maturity score (0-100) for the period, with a label of **Developing**, **Practicing**, **Advancing**, or **Optimizing**, and the delta against the previous period. The overall score rolls up four weighted dimensions:
38+
39+
| Dimension | What it measures |
40+
|---|---|
41+
| Reliability | Build success rate, mean time to recovery (MTTR), number of flaky build profiles (same commit both passing and failing), and profiles with recurring warning hotspots |
42+
| Discipline | Workflow completeness against Appcircle's recommended steps, and adherence to configuration best practices |
43+
| Speed | How consistent build durations are (P95/P50 ratio) and an overall speed score across active profiles |
44+
| Security | Signing identity health, environment variable group usage, and certificates or provisioning profiles nearing expiry |
45+
46+
Below the scores, a ranked **Top Improvements** list surfaces the highest-impact actions for the period, each with the affected build profiles. Typical items include:
47+
48+
- Resolving flaky profiles
49+
- Migrating [custom scripts to Git](/workflows/common-workflow-steps/custom-script-from-git)
50+
- Configuring PR workflows
51+
- Configuring [environment variable groups](/environment-variables)
52+
- Renewing [signing identities](/signing-identities) that are close to expiring
53+
54+
<Screenshot url='https://cdn.appcircle.io/docs/assets/AI-110-Build-Insights-Maturity-Assessment.png'/>
55+
56+
#### Health Snapshot & Trends
57+
58+
**Health Snapshot** is a point-in-time view of the period:
59+
60+
- Summary cards for total builds, success rate, failed builds, and canceled/timeout builds, each with the change versus the previous period.
61+
- **Top active profiles**: the busiest build profiles, with build counts and success rates.
62+
- **Inactive profiles**: profiles with no builds in the last 30 days, as cleanup or archiving candidates.
63+
- **Top build activity**: the most active profile/branch combinations.
64+
- **Top manual and automated triggers**: who is starting the builds and how (manual or trigger).
65+
- An AI-authored summary of the period's overall volume and success-rate signal.
66+
67+
<Screenshot url='https://cdn.appcircle.io/docs/assets/AI-110-Build-Insights-Health-Snapshot.png'/>
68+
69+
**Trends** plots the same window over time:
70+
71+
- Daily pass rate and fail rate.
72+
- Daily build volume, broken down by success, failed, canceled, and timeout.
73+
- Per-profile build duration, both as a chart of median (P50) build time over the period and as a table of P50, P95, the P95/P50 ratio, a trend (faster/stable/slower), and a consistency rating (consistent/moderate/high variance) for each active profile.
74+
75+
<Screenshot url='https://cdn.appcircle.io/docs/assets/AI-110-Build-Insights-Trends.png'/>
76+
77+
#### Root Cause
78+
79+
Failure analysis for the period:
80+
81+
- **Top failing steps**: the workflow steps most responsible for failures, with the profile and branch combinations they failed on most.
82+
- **Mean time to recovery**: the longest failure-to-success gaps, by profile and branch.
83+
- **Flaky builds**: profiles and branches where the same commit produced both a pass and a fail, with an estimate of wasted build time.
84+
- **Warning hotspots**: profiles and branches that repeatedly pass with warnings, and which step triggers them most.
85+
- **Zero success workflows**: workflows that had runs in the period but never passed, with the step that always failed.
86+
87+
<Screenshot url='https://cdn.appcircle.io/docs/assets/AI-110-Build-Insights-Root-Cause.png'/>
88+
89+
#### Workflow Quality
90+
91+
How well each profile's workflows match Appcircle's recommended flow:
92+
93+
- A completeness score (0-100) for the most active Push workflow and the most active PR workflow per profile, with the workflow name. A profile with no Push or PR workflow configured shows that state explicitly rather than a zero score.
94+
- A tiered breakdown of missing steps across all evaluated workflows, grouped as **Critical** (must have), **Important** (great to have), and **Advanced** (good to have), each linking out to the relevant Appcircle workflow step documentation.
95+
- **Best practice checks**: configuration issues found across active profiles, such as custom scripts not sourced from Git, missing PR workflows, or [versioning](/versioning) that is not automated.
96+
97+
<Screenshot url='https://cdn.appcircle.io/docs/assets/AI-110-Build-Insights-Trends-Workflow-Quality.png'/>
98+
99+
#### Artifact Health
100+
101+
- Average primary artifact size per profile for the period, as a ranked bar chart.
102+
- Call-outs for the profile whose artifact grew the most and the one that shrank the most compared to the previous period (when a previous-period baseline exists).
103+
104+
#### Queue Time
105+
106+
How long builds waited before starting, tree-wide for the period:
107+
108+
- Average, P50, P95, and total wait time, across however many builds had queue data.
109+
- A daily trend of average wait time.
110+
- When wait times are elevated or trending up, a recommendation card that names which [machine plan](/infrastructure/machine-plans) absorbed the most wait and suggests an upgrade path (Standard, Velocity, Ultra) or increasing concurrency.
111+
112+
<Screenshot url='https://cdn.appcircle.io/docs/assets/AI-110-Build-Insights-Artifact-Queue.png'/>
113+
114+
### FAQ
115+
116+
#### What's the Default Time Range?
117+
118+
The last 30 days, compared against the 30 days before that for deltas. Pass `start_date` and `end_date` to the tool to use a different window; the tool compares it against a preceding period of the same length.
119+
120+
#### Can I Get This Report Outside the Appcircle Claude Assistant?
121+
122+
Yes, for the underlying data. Any MCP client can call `get_build_insights_report` and get the same metrics as JSON. The visual HTML report rendering shown on this page is produced by the `appcircle:build-insights-report` skill, which is currently exclusive to the [Appcircle Claude Assistant](/appcircle-ai/ai-assistants/appcircle-claude-assistant).
123+
124+
#### Why Is a Section Missing From My Report?
125+
126+
A section can be absent because there was not enough build activity to compute it for the requested period (for example, no builds, no queued builds, or no profiles with a comparable previous period), which is not an error and not a zero value. It can also be absent because your [access token](/appcircle-ai/appcircle-mcp-server#authentication) lacks the required permission for the data that section depends on. Generating the full report requires a token with at least the **Manager** role. Check your token's role permissions if a section you expect to see is consistently absent.
127+
128+
#### Does the Report Include Builds From Sub-Organizations?
129+
130+
Only if you set `include_sub_orgs` to `true` when calling the tool. It defaults to `false`.
131+
132+
#### Why Might Workflow Quality Section Be Missing?
133+
134+
Workflow Quality is computed from profiles that have at least one Push or PR trigger configured. If no profiles in the organization have Push or PR triggers set up, the section is omitted from the report.

docs/appcircle-ai/appcircle-mcp-server.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -92,7 +92,7 @@ You can exclude one or more toolsets when running the server (via environment va
9292
| Tool | Description | Permission |
9393
|------|-------------|------------|
9494
| `get_build_history_report` | Get build history report with pagination and optional filters (date range, build profile, organization). | Viewer |
95-
| `get_build_insights_report` | Get an aggregated Build Insights Report over build history: health snapshot and trends, root cause, artifact health, workflow quality, queue time, and maturity assessment. Defaults to the last 30 days; supports optional date range, section filtering, and sub-organization scope. | Viewer |
95+
| `get_build_insights_report` | Get an aggregated Build Insights Report over build history: health snapshot and trends, root cause, artifact health, workflow quality, queue time, and maturity assessment. Defaults to the last 30 days; supports optional date range, section filtering, and sub-organization scope. See [AI Insights](/appcircle-ai/ai-insights) for the full metric reference. | Viewer |
9696
| `get_signing_report` | Get signing report with pagination and optional filters (date range, organization, OS, build status). | Viewer |
9797
| `get_distribution_app_version_report` | Get daily usage report for distributed app versions with pagination and filters (date range, profile, OS, organization). | Viewer |
9898
| `get_distribution_sent_report` | Get daily usage report for distributed app sharing with pagination and filters (date range, profile, OS, organization). | Viewer |

docs/appcircle-ai/index.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,3 +22,9 @@ The Appcircle MCP Server exposes Appcircle platform capabilities through the Mod
2222
Use the Appcircle GPT Assistant and Appcircle Claude Assistant for quick answers, step-by-step guidance, and troubleshooting support.
2323

2424
<ContentRef url="/appcircle-ai/ai-assistants">AI Assistants</ContentRef>
25+
26+
## [AI Insights](/appcircle-ai/ai-insights)
27+
28+
Understand the Build Insights Report: health and trends, root cause, workflow quality, artifact health, queue time, and CI maturity scoring from the `get_build_insights_report` MCP tool.
29+
30+
<ContentRef url="/appcircle-ai/ai-insights">AI Insights</ContentRef>

docs/release-notes.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -45,7 +45,7 @@ import RedisDomainCaution from '@site/docs/self-hosted-appcircle/install-server/
4545
- A Password visibility toggle has been added to Appcircle login screen. <AccountBadge/> <CloudBadge/>
4646
- The [Appcircle Standard macOS Pool (arm64)](/infrastructure/ios-build-infrastructure) has the latest RC release of Xcode `26.6` `(RC 2)` installed on runners. We strongly recommend extensive testing of your workflows to ensure compatibility and stability with this release. <InfrastructureBadge/> <CloudBadge/>
4747
- The [Appcircle Standard macOS Pool (arm64)](/infrastructure/ios-build-infrastructure) has the latest Beta release of Xcode `27.0` installed on runners. We strongly recommend extensive testing of your workflows to ensure compatibility and stability with this release.
48-
- Introduced the [Appcircle Claude Assistant](/appcircle-ai/ai-assistants/appcircle-claude-assistant), a Claude plugin skill that provides answers about Appcircle, including step-by-step instructions and troubleshooting support, along with the new Build Insights Report skill and MCP tool, which generates a CI health report covering maturity, trends, root cause, workflow quality, artifact health, and queue time. <CloudBadge/>
48+
- Introduced the [Appcircle Claude Assistant](/appcircle-ai/ai-assistants/appcircle-claude-assistant), a Claude plugin skill that provides answers about Appcircle, including step-by-step instructions and troubleshooting support, along with the new [Build Insights Report](/appcircle-ai/ai-insights) skill and MCP tool, which generates a CI health report covering maturity, trends, root cause, workflow quality, artifact health, and queue time. <CloudBadge/>
4949

5050
### 🐞 Fixes
5151

docs/tags.yml

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -288,6 +288,11 @@ appcircle ai:
288288
description: "Documentation related to Appcircle AI"
289289
permalink: /appcircle-ai
290290

291+
"ai insights":
292+
label: "ai insights"
293+
description: "Description for ai insights"
294+
permalink: /ai-insights
295+
291296
"appcircle assistant":
292297
label: "appcircle assistant"
293298
description: "Description for appcircle assistant"
@@ -583,6 +588,11 @@ build:
583588
description: "Description for build artifacts"
584589
permalink: /build-artifacts
585590

591+
"build insights":
592+
label: "build insights"
593+
description: "Description for build insights"
594+
permalink: /build-insights
595+
586596
"build configuration":
587597
label: "build configuration"
588598
description: "Description for build configuration"

0 commit comments

Comments
 (0)