Skip to content

ci: Add AI triage automation for GitHub issues#1576

Draft
devin-ai-integration[bot] wants to merge 2 commits into
mainfrom
devin/1770310672-ai-triage-workflow
Draft

ci: Add AI triage automation for GitHub issues#1576
devin-ai-integration[bot] wants to merge 2 commits into
mainfrom
devin/1770310672-ai-triage-workflow

Conversation

@devin-ai-integration
Copy link
Copy Markdown
Contributor

Summary

Adds automated AI triage for GitHub issues in this repository. When a new issue is opened, Devin AI will automatically analyze it and post a triage report with root cause analysis and recommended next steps.

This PR adds two workflows:

  • ai-triage.yml: Triggers on new issues (automatic) or via /ai-triage slash command (manual). Invokes Devin with the !agent_connector_triage playbook.
  • issue-slash-commands.yml: Dispatches /ai-triage and /ai-fix slash commands from issue comments.

Companion PR: The playbook referenced by this workflow is being added in airbytehq/oncall#11169.

Review & Testing Checklist for Human

  • Verify secrets are configured: This workflow requires OCTAVIA_BOT_APP_ID, OCTAVIA_BOT_PRIVATE_KEY, and DEVIN_AI_API_KEY secrets in the repo settings
  • Create the ai-triage label: The workflow adds this label to issues, so it should exist in the repo
  • Merge oncall PR first: The playbook at !agent_connector_triage must exist before this workflow can succeed
  • Note: /ai-fix is registered but has no workflow: The slash command dispatcher registers /ai-fix but there's no corresponding workflow yet - this is intentional for future expansion

Test plan: After merging both PRs and configuring secrets, create a test issue in this repo and verify that Devin automatically posts a triage comment.

Notes

Requested by: Alexandre Burgoni (@alexandrebrg)
Link to Devin run: https://app.devin.ai/sessions/6ee0a9d7dbb142b495efa3b2f300f24a

devin-ai-integration Bot and others added 2 commits February 5, 2026 16:58
Co-Authored-By: alexandre@airbyte.io <alexandre@airbyte.io>
Co-Authored-By: alexandre@airbyte.io <alexandre@airbyte.io>
@devin-ai-integration
Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Comment on lines +10 to +43
if: ${{ !github.event.issue.pull_request }}
runs-on: ubuntu-latest
steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Authenticate as GitHub App
uses: actions/create-github-app-token@v2
id: get-app-token
with:
owner: "airbytehq"
repositories: "airbyte-agent-connectors"
app-id: ${{ secrets.OCTAVIA_BOT_APP_ID }}
private-key: ${{ secrets.OCTAVIA_BOT_PRIVATE_KEY }}

- name: Slash Command Dispatch
id: scd
uses: peter-evans/slash-command-dispatch@v3
with:
token: ${{ steps.get-app-token.outputs.token }}
commands: |
ai-triage
ai-fix
static-args: |
issue=${{ github.event.issue.number }}
comment-id=${{ github.event.comment.id }}

- name: Edit comment with error message
if: steps.scd.outputs.error-message
uses: peter-evans/create-or-update-comment@v1
with:
comment-id: ${{ github.event.comment.id }}
body: |
> Error: ${{ steps.scd.outputs.error-message }}

Check warning

Code scanning / CodeQL

Workflow does not contain permissions Medium

Actions job or workflow does not limit the permissions of the GITHUB_TOKEN. Consider setting an explicit permissions block, using the following as a minimal starting point: {contents: read}

Copilot Autofix

AI 4 months ago

In general, the fix is to add an explicit permissions block either at the workflow root (applies to all jobs) or inside the slashCommandDispatch job, granting only the scopes needed. This documents the workflow’s requirements and prevents it from inheriting overly broad defaults such as contents: write.

For this particular workflow, it needs to: (1) read issue and comment data (already available via the event payload and via the GitHub App token), and (2) create or update comments on issues via peter-evans/create-or-update-comment@v1. That action, when not configured with a custom token, uses GITHUB_TOKEN and requires issues: write (or pull-requests: write for PR comments). Since the workflow is explicitly limited to non-PR issues (if: ${{ !github.event.issue.pull_request }}), we can safely grant issues: write and set contents: read minimally. The best fix is to add a job-level permissions block under slashCommandDispatch: with contents: read and issues: write. No imports or other code changes are necessary.

Concretely, in .github/workflows/issue-slash-commands.yml, under slashCommandDispatch: and before runs-on: ubuntu-latest, insert:

    permissions:
      contents: read
      issues: write

This keeps functionality intact while limiting GITHUB_TOKEN to only what’s needed.

Suggested changeset 1
.github/workflows/issue-slash-commands.yml

Autofix patch

Autofix patch
Run the following command in your local git repository to apply this patch
cat << 'EOF' | git apply
diff --git a/.github/workflows/issue-slash-commands.yml b/.github/workflows/issue-slash-commands.yml
--- a/.github/workflows/issue-slash-commands.yml
+++ b/.github/workflows/issue-slash-commands.yml
@@ -8,6 +8,9 @@
   slashCommandDispatch:
     # Only allow slash commands on issues (not PRs)
     if: ${{ !github.event.issue.pull_request }}
+    permissions:
+      contents: read
+      issues: write
     runs-on: ubuntu-latest
     steps:
       - name: Checkout code
EOF
@@ -8,6 +8,9 @@
slashCommandDispatch:
# Only allow slash commands on issues (not PRs)
if: ${{ !github.event.issue.pull_request }}
permissions:
contents: read
issues: write
runs-on: ubuntu-latest
steps:
- name: Checkout code
Copilot is powered by AI and may make mistakes. Always verify output.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant