Skip to content

Commit c423920

Browse files
dcramerdcramer
committed
feat(core): catalog plugin resource events
1 parent 4a45427 commit c423920

48 files changed

Lines changed: 1898 additions & 587 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎TERMINOLOGY.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,10 @@ Canonical words used across Junior's code and documentation.
99
- **Source**: where an inbound event came from, such as Slack, local CLI,
1010
scheduler, or plugin dispatch.
1111
- **Destination**: where Junior sends output or side effects.
12+
- **Resource event**: one normalized change published by a plugin, identified
13+
by namespace, identifier, event type, and an idempotency key.
14+
- **Resource subscription**: a temporary conversation association that delivers
15+
matching resource events back into that conversation.
1216
- **Inbound message**: one normalized source event made available to the agent.
1317
- **Agent input**: the inbound content, context, and runtime metadata selected
1418
for a turn.

‎packages/docs/src/content/docs/extend/github-plugin.md‎

Lines changed: 29 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -46,16 +46,16 @@ You can optionally declare `appPermissions` when registering the plugin. Junior
4646

4747
Set these values in the host environment:
4848

49-
| Variable | Required | Purpose |
50-
| -------------------------- | -------- | ------------------------------------------------------------------- |
51-
| `GITHUB_APP_ID` | Yes | GitHub App identity. |
52-
| `GITHUB_APP_CLIENT_ID` | Yes | GitHub App OAuth client id for user-token auth. |
53-
| `GITHUB_APP_CLIENT_SECRET` | Yes | GitHub App OAuth client secret for user-token auth. |
54-
| `GITHUB_APP_PRIVATE_KEY` | Yes | GitHub App signing key. |
55-
| `GITHUB_INSTALLATION_ID` | Yes | Repository or organization installation target. |
56-
| `GITHUB_APP_BOT_NAME` | Yes | Git author and committer display name. |
57-
| `GITHUB_APP_BOT_EMAIL` | Yes | App bot noreply email used for Git attribution and work ownership. |
58-
| `GITHUB_WEBHOOK_SECRET` | No | Webhook signing secret for deployment and PR watches and reporting. |
49+
| Variable | Required | Purpose |
50+
| -------------------------- | -------- | ---------------------------------------------------------------------- |
51+
| `GITHUB_APP_ID` | Yes | GitHub App identity. |
52+
| `GITHUB_APP_CLIENT_ID` | Yes | GitHub App OAuth client id for user-token auth. |
53+
| `GITHUB_APP_CLIENT_SECRET` | Yes | GitHub App OAuth client secret for user-token auth. |
54+
| `GITHUB_APP_PRIVATE_KEY` | Yes | GitHub App signing key. |
55+
| `GITHUB_INSTALLATION_ID` | Yes | Repository or organization installation target. |
56+
| `GITHUB_APP_BOT_NAME` | Yes | Git author and committer display name. |
57+
| `GITHUB_APP_BOT_EMAIL` | Yes | App bot noreply email used for Git attribution and work ownership. |
58+
| `GITHUB_WEBHOOK_SECRET` | No | Webhook signing secret for deployment, pull request, and issue events. |
5959

6060
`GITHUB_INSTALLATION_ID` selects the GitHub App installation for the deployment.
6161
`GITHUB_APP_BOT_EMAIL` uses the GitHub noreply format
@@ -129,9 +129,11 @@ Git smart-HTTP push classification is repository-scoped, not branch-scoped. It d
129129
If your team works across multiple repositories, have users include `owner/repo` in their GitHub request whenever the target is not obvious from the conversation.
130130
That only helps when those repositories are covered by the same GitHub App installation ID.
131131

132-
## Watch pull request events
132+
## Watch pull request and issue events
133133

134-
When `GITHUB_WEBHOOK_SECRET` is configured, `github_createPullRequest` and `github_getPullRequest` return a subscribable pull request resource. Junior can then subscribe the current Slack conversation to high-signal PR events for either a newly created or existing PR when watching it serves the user's request.
134+
When `GITHUB_WEBHOOK_SECRET` is configured, GitHub tools can return subscribable
135+
pull request and issue resources. Junior can temporarily watch those resources
136+
and send matching updates back to the current Slack thread.
135137

136138
Subscribed events run headlessly as Junior's `resource-event` system actor.
137139
They can use repository-scoped installation credentials to commit and push a
@@ -141,15 +143,19 @@ review, still require explicit delegated user authorization.
141143

142144
Supported GitHub webhook deliveries become these Junior resource events:
143145

144-
| GitHub delivery | Junior event types |
145-
| ------------------------------------- | ----------------------------------------------------------------- |
146-
| `check_suite` completed | `checks.failed`, `checks.recovered` |
147-
| `issue_comment` created on a PR | `comment.created` |
148-
| `pull_request_review` submitted | `review.approved`, `review.changes_requested`, `review.commented` |
149-
| `pull_request_review_comment` created | `review_comment.created` |
150-
| `pull_request` closed | `state.merged`, `state.closed_unmerged` |
146+
| GitHub delivery | Junior event types |
147+
| ------------------------------------- | -------------------------------------------------------------------------------------------------------- |
148+
| `check_suite` completed | `pull_request.checks.failed`, `pull_request.checks.recovered` |
149+
| `issue_comment` created on a PR | `pull_request.comment.created` |
150+
| `pull_request_review` submitted | `pull_request.review.approved`, `pull_request.review.changes_requested`, `pull_request.review.commented` |
151+
| `pull_request_review_comment` created | `pull_request.review_comment.created` |
152+
| `pull_request` closed | `pull_request.merged`, `pull_request.closed_unmerged` |
153+
| `issues` opened, closed, or reopened | `issue.opened`, `issue.closed`, `issue.reopened` |
154+
| `issue_comment` created on an issue | `issue.comment.created` |
151155

152-
`state.merged` and `state.closed_unmerged` complete the subscription after Junior accepts the event. Other supported events keep the watch active until the subscription expires, is cancelled, or reaches its configured TTL.
156+
`pull_request.merged` and `pull_request.closed_unmerged` complete a
157+
temporary pull request watch after Junior accepts the event. Other events keep
158+
the watch active until it expires or is cancelled.
153159

154160
Webhook events are delivered as normal queued conversation messages. They do not interrupt active work, bypass Slack routing, or act as user-authored commands. Junior uses the subscription intent to decide whether to reply, take a follow-up action, or stay silent.
155161

@@ -245,7 +251,7 @@ expected follow-up in the original conversation.
245251
- The host-side lease is bounded by the sandbox session and token expiry. It is not exposed as reusable long-lived auth inside the sandbox.
246252
- GitHub webhooks are accepted only when the `X-Hub-Signature-256` header matches `GITHUB_WEBHOOK_SECRET`.
247253
- Resource event subscriptions are conversation-scoped. Core owns subscription records, dedupe, TTL, and mailbox delivery; the GitHub plugin owns signature verification, provider normalization, and its pull request and issue outcome projections.
248-
- Resource-event turns do not inherit a subscriber's user credential. Bot-owned issue, pull request, and smart-HTTP push operations use scoped installation credentials; human-owned operations still enter the normal authorization flow.
254+
- Resource-watch turns do not inherit a subscriber's user credential. Bot-owned issue, pull request, and smart-HTTP push operations use scoped installation credentials; human-owned operations still enter the normal authorization flow.
249255
- The write boundary is the App installation scope, the single-repository token scope, and Junior's endpoint allowlist. `appPermissions` declarations do not narrow write tokens.
250256

251257
## Failure modes
@@ -256,8 +262,8 @@ expected follow-up in the original conversation.
256262
- `github_getDeployment` returns `403`: grant the GitHub App `Deployments: read`, approve the updated permission on its installation, and retry.
257263
- Deployment metadata is available but Junior never offers to watch it: `GITHUB_WEBHOOK_SECRET` is missing. Set it, redeploy, and run `github_getDeployment` again.
258264
- GitHub webhook delivery returns `401`: the webhook secret in GitHub App settings does not match `GITHUB_WEBHOOK_SECRET`, or GitHub did not send `X-Hub-Signature-256`. Update the app webhook secret and retry the delivery.
259-
- GitHub webhook delivery returns `202 Ignored`: the delivery was signed correctly but does not map to a supported deployment or PR watch event. Use one of the configured event types above.
260-
- GitHub delivery succeeds but no Slack follow-up appears: confirm the original conversation has an active resource event subscription for that PR and event type. A successful webhook alone does not create a subscription.
265+
- GitHub webhook delivery returns `202 Ignored`: the delivery was signed correctly but does not map to a supported deployment, pull request, or issue event. Use one of the configured event types above.
266+
- GitHub delivery succeeds but no Slack follow-up appears: confirm the original conversation has an active resource watch for that identifier and event type. A successful webhook alone does not create a watch.
261267
- Missing repository context: Junior could not determine which repository to use. Include `owner/repo` directly in the GitHub request, or configure a default GitHub repository for that thread, and retry.
262268
- A `403` response that says to use `github_createIssue` or `github_createPullRequest` is a Junior routing denial, not evidence of missing App permissions. Retry with the named tool.
263269
- Private OAuth prompt for a human-identity operation such as a pull request review: the actor has not authorized the GitHub App yet, or the stored user-to-server token expired. Complete the private authorization prompt; do not paste personal access tokens into the chat or sandbox.

‎packages/junior-evals/evals/agent/subscriptions.eval.ts‎

Lines changed: 58 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@ describeEval("Resource Event Subscriptions", slackEvals, (it) => {
2929
criteria: rubric({
3030
pass: [
3131
"The reply confirms the exact deployment target will be monitored through event-based updates.",
32+
"The reply makes clear that the watch is temporary and says when it expires.",
3233
],
3334
fail: [
3435
"Do not claim a polling task or recurring schedule was created.",
@@ -41,22 +42,24 @@ describeEval("Resource Event Subscriptions", slackEvals, (it) => {
4142
expect.arrayContaining([
4243
expect.objectContaining({
4344
name: "github_getDeployment",
45+
status: "ok",
4446
arguments: {
4547
commitSha,
4648
environment: "Production",
4749
repo: "getsentry/junior-prod",
4850
},
4951
}),
5052
expect.objectContaining({
51-
name: "subscribeToResourceEvents",
53+
name: "watchResourceEvents",
54+
status: "ok",
5255
arguments: expect.objectContaining({
5356
events: expect.arrayContaining([
5457
"deployment.succeeded",
5558
"deployment.failed",
5659
"deployment.error",
5760
]),
58-
provider: "github",
59-
resourceRef: `github:deployment-source:getsentry/junior-prod:production:${commitSha}`,
61+
namespace: "github",
62+
identifier: `deployment-source:getsentry/junior-prod:production:${commitSha}`,
6063
resourceType: "deployment_source",
6164
}),
6265
}),
@@ -72,16 +75,19 @@ describeEval("Resource Event Subscriptions", slackEvals, (it) => {
7275
}) => {
7376
const result = await run({
7477
overrides: {
78+
github_resource_events: true,
7579
plugin_dirs: ["fixtures/resource-event-plugins"],
80+
plugin_packages: ["@sentry/junior-github"],
7681
},
7782
initialEvents: [
7883
mention(
79-
"/eval-resource-events Use the provider to create a pull request titled 'Prefer event subscriptions', then check it every five minutes and tell this thread if checks fail, review feedback arrives, it merges, or it closes.",
84+
"$eval-resource-events Use the provider to create a pull request titled 'Prefer event subscriptions', then check it every five minutes and tell this thread if checks fail, review feedback arrives, it merges, or it closes.",
8085
),
8186
],
8287
criteria: rubric({
8388
pass: [
8489
"The reply confirms the pull request will be monitored through event-based updates for the requested outcomes.",
90+
"The reply makes clear that the watch is temporary and says when it expires.",
8591
],
8692
fail: [
8793
"Do not ask the user to monitor GitHub manually.",
@@ -94,6 +100,7 @@ describeEval("Resource Event Subscriptions", slackEvals, (it) => {
94100
expect.arrayContaining([
95101
expect.objectContaining({
96102
name: "callMcpTool",
103+
status: "ok",
97104
arguments: expect.objectContaining({
98105
tool_name:
99106
"mcp__eval-resource-events__create-watchable-pull-request",
@@ -103,26 +110,27 @@ describeEval("Resource Event Subscriptions", slackEvals, (it) => {
103110
}),
104111
}),
105112
expect.objectContaining({
106-
name: "subscribeToResourceEvents",
113+
name: "watchResourceEvents",
114+
status: "ok",
107115
arguments: expect.objectContaining({
108-
provider: "github",
109-
resourceRef: "github:pull_request:getsentry/junior#208",
116+
namespace: "github",
117+
identifier: "getsentry/junior#208",
110118
resourceType: "pull_request",
111119
events: expect.arrayContaining([
112-
"checks.failed",
113-
"review.changes_requested",
114-
"review.commented",
115-
"review_comment.created",
116-
"state.merged",
117-
"state.closed_unmerged",
120+
"pull_request.checks.failed",
121+
"pull_request.review.changes_requested",
122+
"pull_request.review.commented",
123+
"pull_request.review_comment.created",
124+
"pull_request.merged",
125+
"pull_request.closed_unmerged",
118126
]),
119127
}),
120128
result: expect.objectContaining({
121129
stop_watching: {
122130
execution_tool: "executeTool",
123131
execution_example: {
124132
tool_name: "stopWatchingResources",
125-
arguments: {},
133+
arguments: { id: expect.stringMatching(/^resub_/) },
126134
},
127135
},
128136
}),
@@ -134,7 +142,7 @@ describeEval("Resource Event Subscriptions", slackEvals, (it) => {
134142
);
135143
});
136144

137-
it("when a follow-up stops monitoring, cancel the conversation's subscriptions before confirming", async ({
145+
it("when a follow-up stops monitoring, cancel the selected watch before confirming", async ({
138146
run,
139147
}) => {
140148
const thread = {
@@ -144,11 +152,13 @@ describeEval("Resource Event Subscriptions", slackEvals, (it) => {
144152
};
145153
const result = await run({
146154
overrides: {
155+
github_resource_events: true,
147156
plugin_dirs: ["fixtures/resource-event-plugins"],
157+
plugin_packages: ["@sentry/junior-github"],
148158
},
149159
initialEvents: [
150160
mention(
151-
"/eval-resource-events Create a pull request titled 'Stop resource monitoring', watch its checks and review feedback, and keep me posted here.",
161+
"$eval-resource-events Create a pull request titled 'Stop resource monitoring', watch its checks and review feedback, and keep me posted here.",
152162
{ thread },
153163
),
154164
],
@@ -165,16 +175,40 @@ describeEval("Resource Event Subscriptions", slackEvals, (it) => {
165175
}),
166176
});
167177

168-
expect(toolCalls(result.session)).toEqual(
178+
const calls = toolCalls(result.session);
179+
expect(calls).toEqual(
169180
expect.arrayContaining([
170181
expect.objectContaining({
171-
name: "subscribeToResourceEvents",
182+
name: "watchResourceEvents",
183+
status: "ok",
172184
}),
173185
expect.objectContaining({
174186
name: "stopWatchingResources",
187+
status: "ok",
175188
}),
176189
]),
177190
);
191+
const watch = calls.find((call) => call.name === "watchResourceEvents");
192+
const stop = calls.find((call) => call.name === "stopWatchingResources");
193+
if (!watch || watch.status !== "ok") {
194+
throw new Error("Expected a successful resource watch tool call");
195+
}
196+
if (
197+
!watch.result ||
198+
typeof watch.result !== "object" ||
199+
Array.isArray(watch.result) ||
200+
typeof watch.result.id !== "string"
201+
) {
202+
throw new Error("Resource watch result did not contain an id");
203+
}
204+
expect(stop).toMatchObject({
205+
arguments: { id: watch.result.id },
206+
result: {
207+
stoppedIds: [watch.result.id],
208+
watching_status: "stopped",
209+
},
210+
status: "ok",
211+
});
178212
});
179213

180214
it("when a subscribed PR check fails, summarize the failure and suggest next steps", async ({
@@ -184,11 +218,11 @@ describeEval("Resource Event Subscriptions", slackEvals, (it) => {
184218
initialEvents: [
185219
resourceEventNotification({
186220
eventKey: "github-delivery-checks-failed",
187-
eventType: "checks.failed",
221+
eventType: "pull_request.checks.failed",
188222
intent:
189223
"Watch the pull request Junior opened for CI failures before review.",
190224
label: "GitHub PR getsentry/junior#691",
191-
resourceRef: "github:pull_request:getsentry/junior#691",
225+
identifier: "getsentry/junior#691",
192226
trustedSummary:
193227
'GitHub PR getsentry/junior#691 checks failed on workflow "test" for commit abcdef123456.',
194228
}),
@@ -222,11 +256,11 @@ describeEval("Resource Event Subscriptions", slackEvals, (it) => {
222256
githubWebhook({
223257
eventName: "check_suite",
224258
subscription: {
225-
events: ["checks.recovered"],
259+
events: ["pull_request.checks.recovered"],
226260
intent:
227261
"Let the original Slack thread know when Junior's pull request lands.",
228262
label: "GitHub PR getsentry/junior#702",
229-
resourceRef: "github:pull_request:getsentry/junior#702",
263+
identifier: "getsentry/junior#702",
230264
resourceType: "pull_request",
231265
},
232266
body: {
@@ -251,11 +285,11 @@ describeEval("Resource Event Subscriptions", slackEvals, (it) => {
251285
initialEvents: [
252286
resourceEventNotification({
253287
eventKey: "github-delivery-pr-merged",
254-
eventType: "state.merged",
288+
eventType: "pull_request.merged",
255289
intent:
256290
"Let the original Slack thread know when Junior's pull request lands.",
257291
label: "GitHub PR getsentry/junior#702",
258-
resourceRef: "github:pull_request:getsentry/junior#702",
292+
identifier: "getsentry/junior#702",
259293
trustedSummary: "GitHub PR getsentry/junior#702 was merged.",
260294
}),
261295
],

‎packages/junior-evals/evals/conversation/routing.eval.ts‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,8 @@ describeEval("Conversation Routing", slackEvals, (it) => {
2828
eventType: "issue.linked",
2929
intent: "Track linked infrastructure work in this Slack thread.",
3030
label: "Linear issue OPS-123",
31-
provider: "linear",
32-
resourceRef: "linear:issue:OPS-123",
31+
namespace: "linear",
32+
identifier: "OPS-123",
3333
thread: steeringThread,
3434
trustedSummary: "Linear issue OPS-123 was linked to this thread.",
3535
}),

0 commit comments

Comments
 (0)