Skip to content

fixing autopilot#524

Closed
tschm wants to merge 7 commits into
mainfrom
autopilot
Closed

fixing autopilot#524
tschm wants to merge 7 commits into
mainfrom
autopilot

Conversation

@tschm

@tschm tschm commented Mar 26, 2026

Copy link
Copy Markdown
Member

Description

Summary by CodeRabbit

  • Chores
    • Switched internal repository analysis tooling to GitHub Copilot using GPT-4.1.
    • Updated the analysis step to append timestamped entries to the repository report, preserving cumulative history.
    • Modified CI job authentication and installation steps for the analysis task to use the updated tooling.
    • Adjusted analysis prompts and execution flow to enforce append-only reporting behavior.

@coderabbitai

coderabbitai Bot commented Mar 26, 2026

Copy link
Copy Markdown
Contributor

Warning

Rate limit exceeded

@tschm has exceeded the limit for the number of commits that can be reviewed per hour. Please wait 0 minutes and 18 seconds before requesting another review.

⌛ How to resolve this issue?

After the wait time has elapsed, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

We recommend that you space out your commits to avoid hitting the rate limit.

🚦 How do rate limits work?

CodeRabbit enforces hourly rate limits for each developer per organization.

Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout.

Please see our FAQ for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: c22748b9-5852-4a3c-8754-c20c2958a2cd

📥 Commits

Reviewing files that changed from the base of the PR and between e5314c0 and d9fcf20.

📒 Files selected for processing (3)
  • .github/workflows/autopilot.yml
  • .github/workflows/autopilot_v2.yml
  • .rhiza/make.d/agentic.mk
📝 Walkthrough

Walkthrough

Replaced Anthropic/Claude with GitHub Copilot across the analysis pipeline: agent model changed to gpt-4.1, analyser instructed to append dated entries to REPOSITORY_ANALYSIS.md, CI installs Copilot CLI (curl installer) and uses GITHUB_TOKEN (secrets.GH_PAT || github.token), and Make target now invokes Copilot CLI.

Changes

Cohort / File(s) Summary
Agent Configuration
\.github/agents/analyser.md
Model updated from claude-sonnet-4.5gpt-4.1; output instruction changed to append dated entries to REPOSITORY_ANALYSIS.md (use shell redirection, e.g., cat >> REPOSITORY_ANALYSIS.md).
CI/CD Workflow
\.github/workflows/autopilot.yml
Replaced Anthropic/Claude CLI installation with GitHub Copilot CLI installer (curl + shell); authentication env changed from ANTHROPIC_API_KEY to `GITHUB_TOKEN: ${{ secrets.GH_PAT
Make Build Targets
.rhiza/make.d/agentic.mk
analyse-repo target now depends on install-copilot and runs $(COPILOT_BIN) with --model "$(DEFAULT_AI_MODEL)" --agent analyser -p "..."; previous install-claude/$(CLAUDE_BIN) invocation and prompt updated to require appending a dated entry.

Sequence Diagram(s)

sequenceDiagram
  participant GH as "GitHub Actions"
  participant Installer as "Copilot Installer"
  participant Make as "make analyse-repo"
  participant Copilot as "Copilot CLI"
  participant File as "REPOSITORY_ANALYSIS.md"

  GH->>Installer: run curl+shell installer
  GH->>Make: invoke `make analyse-repo` (with GITHUB_TOKEN)
  Make->>Copilot: execute `$(COPILOT_BIN)` (--model, --agent, -p)
  Copilot->>File: append dated entry (e.g., `cat >> REPOSITORY_ANALYSIS.md`)
  Copilot-->>Make: return status
  Make-->>GH: job result
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

Poem

🐰 I swapped a Claude hat for Copilot bright,

gpt-4.1 now types into the night,
Installer curls, tokens realigned,
Dated notes appended, repository signed,
A happy rabbit hops — analysis done!

🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Title check ❓ Inconclusive The title 'fixing autopilot' is vague and generic, failing to convey the specific nature of the changes made to the PR. Use a more descriptive title that identifies the key changes, such as 'Replace Claude CLI with GitHub Copilot CLI in autopilot workflow' or 'Switch autopilot from Anthropic to GitHub Copilot integration'.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch autopilot

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
.rhiza/make.d/agentic.mk (1)

9-9: ⚠️ Potential issue | 🟡 Minor

Conditional assignment has no effect due to root Makefile.

The ?= operator means this value (gpt-4.1) is only set if DEFAULT_AI_MODEL is not already defined. However, the root Makefile unconditionally sets DEFAULT_AI_MODEL=claude-sonnet-4.5, so this line has no effect.

Either:

  1. Remove this line since it's misleading, or
  2. Update the root Makefile to use ?= if you want this file's value to serve as a fallback
Option: Remove the ineffective assignment
 COPILOT_BIN ?= $(shell command -v copilot 2>/dev/null || echo "$(INSTALL_DIR)/copilot")
 CLAUDE_BIN ?= $(shell command -v claude 2>/dev/null || echo "$(HOME)/.local/bin/claude")
-DEFAULT_AI_MODEL ?= gpt-4.1
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.rhiza/make.d/agentic.mk at line 9, The conditional assignment
"DEFAULT_AI_MODEL ?= gpt-4.1" is ineffective because the root Makefile
unconditionally sets DEFAULT_AI_MODEL=claude-sonnet-4.5; either remove this
redundant line from .rhiza/make.d/agentic.mk or change the root Makefile's
assignment to use the conditional operator (?=) so this file can act as a
fallback; locate the symbol DEFAULT_AI_MODEL and decide whether to delete the
"?= gpt-4.1" entry here or alter the root Makefile's assignment to
"DEFAULT_AI_MODEL ?=" to make the fallback behavior consistent.
🧹 Nitpick comments (1)
.github/workflows/autopilot.yml (1)

206-207: Note: solve-issues job still uses Claude CLI.

The solve-issues job continues to use Claude CLI (@anthropic-ai/claude-code) while the analyse job now uses Copilot. This appears intentional for this PR, but creates a mixed toolchain. Consider documenting this split or planning to unify in a follow-up.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/workflows/autopilot.yml around lines 206 - 207, The workflow
currently installs Claude CLI in the solve-issues job while the analyse job uses
Copilot, creating a mixed toolchain; update the workflow to either (a) document
this intentional split by adding a comment or job-level annotation referencing
the solve-issues job and the analyse job and the package name
`@anthropic-ai/claude-code`, or (b) unify the toolchain by replacing the
solve-issues installation step that runs "npm install -g
`@anthropic-ai/claude-code`" with the Copilot equivalent used by the analyse job;
ensure you modify the solve-issues job block (job name solve-issues) accordingly
and include a short note explaining the choice if you keep the split.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In @.github/agents/analyser.md:
- Line 4: Align the model setting to avoid confusion: update the model value in
.github/agents/analyser.md (the "model:" entry) to match the root Makefile's
DEFAULT_AI_MODEL (claude-sonnet-4.5) and the sibling
.github/agents/summarise.md, or alternatively change DEFAULT_AI_MODEL in the
Makefile to gpt-4.1 and update the summarise.md file to gpt-4.1—ensure the three
places (the "model:" key in .github/agents/analyser.md, the DEFAULT_AI_MODEL in
the Makefile, and the "model:" in .github/agents/summarise.md) are all
identical.

In @.rhiza/make.d/agentic.mk:
- Around line 19-24: The analyse-repo Make target is passing
conflicting/incorrect Copilot CLI flags: it sets --model "$(DEFAULT_AI_MODEL)"
which conflicts with the agent's declared model (gpt-4.1), uses --agent analyser
without ensuring the CLI path convention, and combines --agent with a prompt
(-p) which can produce unexpected behavior; fix by updating the analyse-repo
recipe (target "analyse-repo") to either remove the explicit --model so the
agent file's model is used, or set --model to match the agent (gpt-4.1), ensure
the --agent argument points to the agent file in the expected form used by the
CLI (e.g., the full agent path if the CLI requires it or confirm that "analyser"
maps to .github/agents/analyser.md), and stop combining -p with --agent (use the
agent file alone or the prompt alone) to avoid ambiguity; reference COPILOT_BIN,
DEFAULT_AI_MODEL, and the analyser agent file when making the change.

---

Outside diff comments:
In @.rhiza/make.d/agentic.mk:
- Line 9: The conditional assignment "DEFAULT_AI_MODEL ?= gpt-4.1" is
ineffective because the root Makefile unconditionally sets
DEFAULT_AI_MODEL=claude-sonnet-4.5; either remove this redundant line from
.rhiza/make.d/agentic.mk or change the root Makefile's assignment to use the
conditional operator (?=) so this file can act as a fallback; locate the symbol
DEFAULT_AI_MODEL and decide whether to delete the "?= gpt-4.1" entry here or
alter the root Makefile's assignment to "DEFAULT_AI_MODEL ?=" to make the
fallback behavior consistent.

---

Nitpick comments:
In @.github/workflows/autopilot.yml:
- Around line 206-207: The workflow currently installs Claude CLI in the
solve-issues job while the analyse job uses Copilot, creating a mixed toolchain;
update the workflow to either (a) document this intentional split by adding a
comment or job-level annotation referencing the solve-issues job and the analyse
job and the package name `@anthropic-ai/claude-code`, or (b) unify the toolchain
by replacing the solve-issues installation step that runs "npm install -g
`@anthropic-ai/claude-code`" with the Copilot equivalent used by the analyse job;
ensure you modify the solve-issues job block (job name solve-issues) accordingly
and include a short note explaining the choice if you keep the split.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 845ea641-5e58-4325-96d3-730f4b94427a

📥 Commits

Reviewing files that changed from the base of the PR and between 0c2e1da and d18cf05.

📒 Files selected for processing (3)
  • .github/agents/analyser.md
  • .github/workflows/autopilot.yml
  • .rhiza/make.d/agentic.mk

name: analyser
description: Ongoing technical journal for repository analysis
model: claude-sonnet-4.5
model: gpt-4.1

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Model mismatch with root Makefile and sibling agent file.

The model is set to gpt-4.1 here, but:

  1. The root Makefile defines DEFAULT_AI_MODEL=claude-sonnet-4.5 (unconditional assignment), which will be passed via --model flag in the CLI invocation
  2. The sibling agent .github/agents/summarise.md uses model: claude-sonnet-4.5

This creates confusion about which model is actually used. Either align this file with the root Makefile's claude-sonnet-4.5, or update the root Makefile to use gpt-4.1 consistently.

Option A: Align with root Makefile
 ---
 name: analyser
 description: Ongoing technical journal for repository analysis
-model: gpt-4.1
+model: claude-sonnet-4.5
 ---
🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.github/agents/analyser.md at line 4, Align the model setting to avoid
confusion: update the model value in .github/agents/analyser.md (the "model:"
entry) to match the root Makefile's DEFAULT_AI_MODEL (claude-sonnet-4.5) and the
sibling .github/agents/summarise.md, or alternatively change DEFAULT_AI_MODEL in
the Makefile to gpt-4.1 and update the summarise.md file to gpt-4.1—ensure the
three places (the "model:" key in .github/agents/analyser.md, the
DEFAULT_AI_MODEL in the Makefile, and the "model:" in
.github/agents/summarise.md) are all identical.

Comment thread .rhiza/make.d/agentic.mk
Comment on lines +19 to +24
analyse-repo: install-copilot ## run the analyser agent to update REPOSITORY_ANALYSIS.md
@"$(COPILOT_BIN)" \
--allow-tool 'shell' \
--model "$(DEFAULT_AI_MODEL)" \
--agent analyser \
-p "Analyze the repository and append a new dated entry to REPOSITORY_ANALYSIS.md"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Likely root cause of pipeline failure: Copilot CLI flag syntax and model conflict.

The pipeline reports analyse-repo step failed (make returned Error 1). Several potential issues:

  1. Model mismatch: The --model "$(DEFAULT_AI_MODEL)" will pass claude-sonnet-4.5 (from root Makefile), but the agent file declares model: gpt-4.1. Verify how Copilot CLI resolves this conflict.

  2. Agent flag syntax: The --agent analyser flag assumes Copilot CLI knows to look in .github/agents/analyser.md. Verify this is the correct path convention.

  3. Combined -p and --agent: Using both a prompt (-p) and an agent file together may have unexpected behavior.

GitHub Copilot CLI --agent flag syntax and usage
🧰 Tools
🪛 GitHub Actions: Autopilot

[error] 20-20: analyse-repo step failed (make returned Error 1).

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In @.rhiza/make.d/agentic.mk around lines 19 - 24, The analyse-repo Make target
is passing conflicting/incorrect Copilot CLI flags: it sets --model
"$(DEFAULT_AI_MODEL)" which conflicts with the agent's declared model (gpt-4.1),
uses --agent analyser without ensuring the CLI path convention, and combines
--agent with a prompt (-p) which can produce unexpected behavior; fix by
updating the analyse-repo recipe (target "analyse-repo") to either remove the
explicit --model so the agent file's model is used, or set --model to match the
agent (gpt-4.1), ensure the --agent argument points to the agent file in the
expected form used by the CLI (e.g., the full agent path if the CLI requires it
or confirm that "analyser" maps to .github/agents/analyser.md), and stop
combining -p with --agent (use the agent file alone or the prompt alone) to
avoid ambiguity; reference COPILOT_BIN, DEFAULT_AI_MODEL, and the analyser agent
file when making the change.

@tschm tschm closed this Mar 26, 2026
@tschm
tschm deleted the autopilot branch March 26, 2026 14:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant