feat: PraisonAI PR Reviewer Integration#1332
Conversation
|
Warning Rate limit exceeded
Your organization is not enrolled in usage-based pricing. Contact your admin to enable usage-based pricing to continue reviews beyond the rate limit, or try again in 1 minutes and 51 seconds. ⌛ How to resolve this issue?After the wait time has elapsed, a review can be triggered using the We recommend that you space out your commits to avoid hitting the rate limit. 🚦 How do rate limits work?CodeRabbit enforces hourly rate limits for each developer per organization. Our paid plans have higher rate limits than the trial, open-source and free plans. In all cases, we re-allow further reviews after a brief timeout. Please see our FAQ for further information. ℹ️ Review info⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: ⛔ Files ignored due to path filters (2)
📒 Files selected for processing (23)
📝 WalkthroughWalkthroughThis release introduces an automated PR review workflow via GitHub Actions configured with PraisonAI agents, adds authentication to multiple HTTP endpoints, hardens file/shell operations against path traversal and environment variable injection, implements AST-based validation for inline Python code execution, and bumps package versions from 4.5.126 to 4.5.127. Changes
Sequence Diagram(s)sequenceDiagram
participant GitHub as GitHub
participant Runner as GitHub Actions Runner
participant Installer as pip Installer
participant PraisonAI as PraisonAI CLI
participant Config as .github/praisonai-reviewer.yaml
participant API as GitHub API
GitHub->>Runner: Trigger on PR event or `@praisonai` comment
Runner->>Runner: Checkout repo (fetch-depth: 0)
Runner->>Runner: Generate GitHub App token
Runner->>Installer: Install praisonaiagents[all]
Installer-->>Runner: Installation complete
Runner->>PraisonAI: Execute: praisonai agents --file .github/praisonai-reviewer.yaml
PraisonAI->>Config: Load reviewer configuration
Config-->>PraisonAI: Agent role & review mission
PraisonAI->>API: Fetch PR diff & context via GITHUB_TOKEN
API-->>PraisonAI: PR changes & metadata
PraisonAI->>PraisonAI: Execute code_review task (AST validation, logic checks)
PraisonAI->>API: Post review comment to PR
API-->>GitHub: Comment published
Estimated code review effort🎯 4 (Complex) | ⏱️ ~50 minutes Possibly related issues
Possibly related PRs
Suggested labels
Poem
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
ⓘ You are approaching your monthly quota for Qodo. Upgrade your plan Review Summary by QodoAdd PraisonAI automated PR reviewer integration
WalkthroughsDescription• Adds automated PR review workflow using PraisonAI agents • Configures GitHub App authentication and token generation • Defines lead reviewer agent with code quality assessment tasks • Includes comprehensive setup documentation and workflow template Diagramflowchart LR
A["GitHub PR Event"] -->|Triggers| B["GitHub Actions Workflow"]
B -->|Generates Token| C["GitHub App Auth"]
C -->|Executes| D["PraisonAI Agents"]
D -->|Reviews Code| E["Lead Reviewer Agent"]
E -->|Outputs| F["Markdown PR Comment"]
File Changes1. .github/praisonai-reviewer.yaml
|
Code Review by Qodo
|
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces an automated pull request reviewer using PraisonAI, providing a configuration file, a setup guide, and a GitHub Actions workflow template. The review identifies a logic error in the workflow's conditional triggers that would prevent manual activation via comments, recommends migrating to the official GitHub Action for token generation to ensure security and reliability, and suggests clarifying the setup instructions in the documentation.
| jobs: | ||
| review: | ||
| runs-on: ubuntu-latest | ||
| if: github.event.pull_request.draft == false && (github.event_name == 'pull_request' || contains(github.event.comment.body, '@praisonai')) |
There was a problem hiding this comment.
The current if condition will cause the workflow to be skipped when triggered by an issue_comment (e.g., via @praisonai). This is because github.event.pull_request is null for such events, making the expression github.event.pull_request.draft == false evaluate to false. The condition should be updated to handle both event types explicitly and ensure the comment is on a pull request.
if: (github.event_name == 'pull_request' && github.event.pull_request.draft == false) || (github.event_name == 'issue_comment' && github.event.issue.pull_request && contains(github.event.comment.body, '@praisonai'))
| uses: tibdex/github-app-token@v2 | ||
| with: | ||
| app_id: ${{ secrets.PRAISONAI_APP_ID }} | ||
| private_key: ${{ secrets.PRAISONAI_APP_PRIVATE_KEY }} |
There was a problem hiding this comment.
The tibdex/github-app-token action is unmaintained. It is recommended to use the official actions/create-github-app-token action from GitHub for better security and reliability. Note that the input parameters for the official action use hyphens (app-id, private-key) instead of underscores.
uses: actions/create-github-app-token@v1
with:
app-id: ${{ secrets.APP_ID }}
private-key: ${{ secrets.PRIVATE_KEY }}
| - `PRAISONAI_APP_PRIVATE_KEY`: The generated Private Key (PEM format). | ||
| - `OPENAI_API_KEY` (or other LLM key) for PraisonAI to use. | ||
|
|
||
| 2. Ensure `.github/workflows/praisonai-pr-review.yml` is present in your default branch. |
There was a problem hiding this comment.
The setup guide should explicitly instruct the user to copy the template file to the target workflow path to ensure the integration is correctly configured and easy to follow.
| 2. Ensure `.github/workflows/praisonai-pr-review.yml` is present in your default branch. | |
| 2. Copy examples/yaml/praisonai-pr-review.yml.template to .github/workflows/praisonai-pr-review.yml in your default branch. |
There was a problem hiding this comment.
Pull request overview
Adds a GitHub Actions + GitHub App–based configuration to run PraisonAI as an automated PR reviewer, along with a default reviewer agent YAML and setup documentation.
Changes:
- Add a PR reviewer GitHub Actions workflow (and an example template) triggered on PR events and
@praisonaiissue comments. - Add a default
.github/praisonai-reviewer.yamlagent configuration for PR reviews. - Add a setup guide documenting required GitHub App permissions and repository secrets.
Reviewed changes
Copilot reviewed 23 out of 25 changed files in this pull request and generated 6 comments.
| File | Description |
|---|---|
PRAISONAI_PR_REVIEWER_SETUP.md |
Documents manual setup steps for the PR reviewer integration. |
examples/yaml/praisonai-pr-review.yml.template |
Provides a copy/paste workflow template for consumers. |
.github/workflows/praisonai-pr-review.yml |
Adds the actual GitHub Actions workflow to run the reviewer. |
.github/praisonai-reviewer.yaml |
Defines the default reviewer agent/task configuration. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| runs-on: ubuntu-latest | ||
| if: github.event.pull_request.draft == false && (github.event_name == 'pull_request' || contains(github.event.comment.body, '@praisonai')) | ||
|
|
There was a problem hiding this comment.
The job-level if: condition references github.event.pull_request.draft, which is not present for issue_comment events. As written, manual triggers via @praisonai will be skipped (expression evaluates false/null) and it may also behave unexpectedly on PR events because contains(github.event.comment.body, ...) is undefined there. Consider splitting the condition by event type (pull_request vs issue_comment) and, for issue_comment, optionally gating by github.event.comment.author_association (e.g., MEMBER/OWNER/COLLABORATOR) to avoid untrusted users triggering runs/costs in public repos.
| - name: Install PraisonAI | ||
| run: pip install praisonaiagents[all] |
There was a problem hiding this comment.
This workflow installs praisonaiagents[all], but the CLI invoked later is praisonai .... The praisonai console script is provided by the praisonai package (see src/praisonai/pyproject.toml), so this will fail with praisonai: command not found unless praisonai is installed. Install praisonai (optionally with the needed extras) instead of, or in addition to, praisonaiagents.
| - name: Run PraisonAI PR Review | ||
| env: | ||
| GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }} | ||
| OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} | ||
| run: | | ||
| praisonai agents --file .github/praisonai-reviewer.yaml |
There was a problem hiding this comment.
praisonai agents --file ... does not match the CLI in this repo. The supported way to execute a YAML file is praisonai run <file> (see src/praisonai/praisonai/cli/commands/run.py). Update the command accordingly so the workflow actually runs.
| review: | ||
| runs-on: ubuntu-latest | ||
| if: github.event.pull_request.draft == false && (github.event_name == 'pull_request' || contains(github.event.comment.body, '@praisonai')) | ||
|
|
There was a problem hiding this comment.
The template uses the same job-level if: that references github.event.pull_request.draft, which won’t exist for issue_comment events. Manual @praisonai triggers will be skipped unless you rewrite the condition to handle pull_request vs issue_comment payloads separately (and consider gating by commenter association to prevent untrusted triggers).
| - name: Install PraisonAI | ||
| run: pip install praisonaiagents[all] |
There was a problem hiding this comment.
The template installs praisonaiagents[all] but then runs the praisonai CLI. praisonai is a different package and provides the console script; without installing it, the workflow will fail. Update the install step to install praisonai (and any required extras).
| - name: Run PraisonAI PR Review | ||
| env: | ||
| GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }} | ||
| OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} | ||
| run: | | ||
| # Use PraisonAI to review the PR | ||
| praisonai agents --file .github/praisonai-reviewer.yaml |
There was a problem hiding this comment.
The template runs praisonai agents --file ..., but the CLI’s file execution entrypoint is praisonai run <yaml> in this repo. Update the command so the template is copy/paste runnable.
![qodo-code-review[bot]](https://avatars.githubusercontent.com/in/484649?s=60&v=4){kind=small}
| - name: Run PraisonAI PR Review | ||
| env: | ||
| GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }} | ||
| OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} | ||
| run: | | ||
| praisonai agents --file .github/praisonai-reviewer.yaml |
There was a problem hiding this comment.
1. Invalid cli invocation 🐞 Bug ≡ Correctness
The workflow runs praisonai agents --file ..., but the agents subcommand only supports run/list and does not accept --file, so the job will fail immediately and never produce a review.
Agent Prompt
### Issue description
The workflow calls `praisonai agents --file .github/praisonai-reviewer.yaml`, but the `agents` subcommand doesn’t support `--file`, so the workflow fails.
### Issue Context
This repo’s documented YAML execution mode is `praisonai <path-to-yaml>` (e.g., `praisonai agents.yaml`).
### Fix Focus Areas
- .github/workflows/praisonai-pr-review.yml[32-40]
- examples/yaml/praisonai-pr-review.yml.template[32-41]
### What to change
- Replace the run command with the positional YAML invocation, e.g.:
- `praisonai .github/praisonai-reviewer.yaml`
- Keep the rest of the workflow the same unless additional context is needed by PraisonAI.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
| jobs: | ||
| review: | ||
| runs-on: ubuntu-latest | ||
| if: github.event.pull_request.draft == false && (github.event_name == 'pull_request' || contains(github.event.comment.body, '@praisonai')) |
There was a problem hiding this comment.
2. Manual trigger never runs 🐞 Bug ≡ Correctness
The job if: condition references github.event.pull_request.draft, which is not present for issue_comment events, so @praisonai comments will not trigger the reviewer job.
Agent Prompt
### Issue description
`issue_comment` events don’t have `github.event.pull_request.draft`, so the job-level `if:` prevents comment-triggered runs.
### Issue Context
The repo already has a working pattern for comment triggers that checks `github.event.issue.pull_request` and gates by association.
### Fix Focus Areas
- .github/workflows/praisonai-pr-review.yml[3-13]
### What to change
- Rewrite the `if:` as a multi-line expression that handles each event type separately, e.g.:
- For PR events: require `github.event.pull_request.draft == false`
- For comment events: require `github.event.issue.pull_request` and the trigger phrase in `github.event.comment.body`
- Ensure the `contains()` call only evaluates for `issue_comment` events to avoid null dereferences.
ⓘ Copy this prompt and use it to remediate the issue with your preferred AI generation tools
|
@copilot Do a thorough review of this PR. Read ALL existing reviewer comments above first. Review areas:
|
|
@claude You are the FINAL architecture reviewer. Read ALL comments above from Gemini, Qodo, CodeRabbit, and Copilot carefully before responding. Phase 1: Review per AGENTS.md
Phase 2: FIX Valid Issues Phase 3: Final Verdict |
|
Claude finished @MervinPraison's task —— View job • Final Architecture Review Completed ✅Acting as the FINAL architecture reviewer. Analyzed all previous reviewer comments and implemented fixes for valid issues. Phase 1: Architecture Review per AGENTS.md
Phase 2: Fix Valid Issues ✅
All Critical Issues Fixed:
Phase 3: Final Verdict ✅
SummaryFiles Modified:
Architecture Validation:
Security Improvements:
Final Decision: This PR implementation is now ready for production use after fixing all identified issues. |
Greptile SummaryThis PR introduces a PraisonAI automated PR reviewer via a new GitHub Actions workflow and agent configuration, and bundles a security hardening pass across several components (auth enforcement, path traversal protection, sandbox validation, redirect blocking, env-var sanitization) along with a version bump to Key changes:
Issues found:
Confidence Score: 3/5The PR bundles solid security hardening across many components but its headline feature (the automated PR reviewer) has two functional blockers: the Two P1 logic bugs exist in the primary feature being introduced: the
|
| Filename | Overview |
|---|---|
| .github/workflows/praisonai-pr-review.yml | New GitHub Actions workflow for automated PR review; contains a critical if condition bug that makes the issue_comment trigger dead code, missing SHA-pinning on third-party action, and no commenter permission check. |
| .github/praisonai-reviewer.yaml | Agent configuration for the PR reviewer; the agent has no GitHub API tools or diff-fetching mechanism, so it cannot actually see PR changes and will produce hallucinated reviews. |
| src/praisonai-agents/praisonaiagents/ui/agui/agui.py | Added auth check (bearer token or localhost-only fallback) and removed wildcard CORS headers; token comparison uses non-constant-time != instead of secrets.compare_digest. |
| src/praisonai/praisonai/app/agentos.py | Added auth check with bearer token/localhost-only fallback and fixed parameter naming collision; same timing-attack-vulnerable token comparison as agui.py. |
| src/praisonai/praisonai/cli/features/job_workflow.py | Added AST-based security validation for inline Python sandbox; blocked functions (getattr, setattr) are contradictorily re-exposed via _safe_builtins, enabling a sandbox bypass via stored references. |
| src/praisonai-agents/praisonaiagents/mcp/mcp.py | Strips sensitive env vars (API keys, tokens, secrets) from child MCP process environments; overly broad pattern may silently break MCP tools that legitimately require those credentials. |
| src/praisonai-agents/praisonaiagents/tools/file_tools.py | Added path traversal and symlink escape protections for directory listing; clean security improvements with no issues found. |
| src/praisonai-agents/praisonaiagents/tools/shell_tools.py | Removed os.path.expandvars() from command and cwd arguments to prevent environment variable injection; straightforward security hardening. |
| src/praisonai-agents/praisonaiagents/tools/web_crawl_tools.py | Disabled redirect following in httpx and urllib to mitigate SSRF via open redirects; may break crawling of URLs that legitimately rely on redirects (e.g., HTTP→HTTPS, canonical redirects). |
| src/praisonai/praisonai/gateway/server.py | Extended auth check to block non-localhost access when no token is configured; correctly uses secrets.compare_digest for token comparison. |
| src/praisonai/praisonai/recipe/registry.py | Added protection against symlink/hardlink attacks during tar extraction; clean, correct security fix. |
| src/praisonai/praisonai/api/call.py | Added null-guard tools_path and before os.path.exists() to prevent crash when TOOLS_PATH env var is unset. |
Sequence Diagram
sequenceDiagram
participant GH as GitHub
participant WF as Actions Workflow
participant App as GitHub App Token
participant AI as PraisonAI Agent
participant LLM as LLM (OpenAI)
GH->>WF: pull_request event (opened/sync)
Note over WF: if: PR.draft==false ✅
WF->>App: tibdex/github-app-token@v2 (APP_ID + PRIVATE_KEY)
App-->>WF: GITHUB_TOKEN
WF->>AI: praisonai agents --file .github/praisonai-reviewer.yaml
Note over AI: ⚠️ No tools to fetch PR diff — No GitHub API integration
AI->>LLM: Review the code changes... (no actual diff provided)
LLM-->>AI: Hallucinated review
Note over AI: No step to post review back to PR
GH->>WF: issue_comment event (@praisonai)
Note over WF: if: PR.draft==false → null==false → false ❌
Note over WF: Job SKIPPED — trigger is dead code
Comments Outside Diff (1)
-
src/praisonai/praisonai/cli/features/job_workflow.py, line 312-323 (link)Sandbox bypass:
getattr/setattrblocked by AST checker but available in_safe_builtinsThe AST validator (lines 302–304) blocks direct calls to
getattrandsetattrby name. However, those functions are then explicitly placed in_safe_builtins(lines 322–323), making them reachable through indirection:# This passes the AST checker — 'g' is not 'getattr' at call site g = getattr result = g(type, '__' + 'mro__') # accesses type.__mro__, bypassing both checks
The string concatenation also defeats the
ast.Constantcheck since'__'and'mro__'individually are not inblocked_attrs.Fix: Remove
getattr,setattr,hasattr, anddelattrfrom_safe_builtins, or accept that these builtins are intentionally available and remove them from the AST blocklist (acknowledging the sandbox is advisory, not hardened).
Reviews (1): Last reviewed commit: "chore: move setup guide to examples dire..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| jobs: | ||
| review: | ||
| runs-on: ubuntu-latest | ||
| if: github.event.pull_request.draft == false && (github.event_name == 'pull_request' || contains(github.event.comment.body, '@praisonai')) |
There was a problem hiding this comment.
issue_comment trigger is dead code
The job-level if condition evaluates github.event.pull_request.draft == false first. When the event is issue_comment, github.event.pull_request does not exist and returns null. In GitHub Actions expression evaluation, null == false is false, so the entire condition short-circuits to false — the job will never run for issue comments, making the @praisonai trigger non-functional.
The fix is to guard each branch independently:
| if: github.event.pull_request.draft == false && (github.event_name == 'pull_request' || contains(github.event.comment.body, '@praisonai')) | |
| if: (github.event_name == 'pull_request' && github.event.pull_request.draft == false) || (github.event_name == 'issue_comment' && contains(github.event.comment.body, '@praisonai')) |
|
|
||
| - name: Generate GitHub App Token | ||
| id: generate_token | ||
| uses: tibdex/github-app-token@v2 |
There was a problem hiding this comment.
tibdex/github-app-token@v2 references a mutable tag. If the upstream repository is compromised or the tag is force-pushed, the action could run arbitrary code with your PRAISONAI_APP_PRIVATE_KEY secret. The GitHub security hardening guide and Actions best practices both recommend pinning third-party actions to a full commit SHA:
| uses: tibdex/github-app-token@v2 | |
| uses: tibdex/github-app-token@3beb63f4bd073e61482598c45c71c1019b59b73a # v2 |
You can find the current SHA for the v2 tag in the tibdex/github-app-token releases page.
| issue_comment: | ||
| types: [created] |
There was a problem hiding this comment.
issue_comment commenter
The issue_comment trigger fires for any comment by any user (including external contributors, bots, or bad actors) who can comment on the repo. Commenting @praisonai will invoke the full workflow with repo secrets. Consider adding a collaborator/org-member permission check, e.g.:
if: |
(github.event_name == 'pull_request' && github.event.pull_request.draft == false) ||
(github.event_name == 'issue_comment' &&
contains(github.event.comment.body, '@praisonai') &&
(github.event.comment.author_association == 'OWNER' ||
github.event.comment.author_association == 'MEMBER' ||
github.event.comment.author_association == 'COLLABORATOR'))| code_review: | ||
| description: Review the code changes in the pull request. Look for logic errors, formatting issues, security flaws, and performance bottlenecks. Output the review as a detailed markdown comment. | ||
| expected_output: A detailed PR review in markdown format. |
There was a problem hiding this comment.
The code_review task instructs the agent to "review the code changes in the pull request," but no GitHub API tools or any file-access tools are configured for the agent. In the workflow, the repo is checked out, but the agent is given no mechanism to:
- Determine the PR number or base/head SHAs
- Run
git diffor read changed files - Post a comment back to the PR
Without a tool like gh pr diff or a GitHub REST tool, the LLM will produce a generic, fabricated review that does not reflect the actual code changes. At minimum, the task description should provide the diff inline (e.g., via a shell step that captures git diff output and passes it to the agent), or the agent configuration should include GitHub tools.
| if not auth_header or not auth_header.startswith("Bearer ") or auth_header.split(" ")[1] != token: | ||
| raise HTTPException(status_code=401, detail="Unauthorized") |
There was a problem hiding this comment.
auth_header.split(" ")[1] != token uses a non-constant-time string comparison, allowing a remote attacker to potentially infer the token character by character through timing measurements. The same issue exists in src/praisonai/praisonai/app/agentos.py at line 128.
The gateway/server.py already correctly uses secrets.compare_digest. Use the same approach here:
| if not auth_header or not auth_header.startswith("Bearer ") or auth_header.split(" ")[1] != token: | |
| raise HTTPException(status_code=401, detail="Unauthorized") | |
| if not auth_header or not auth_header.startswith("Bearer ") or not hmac.compare_digest(auth_header.split(" ")[1], token): |
Add import hmac at the top of the module (or use import secrets; secrets.compare_digest(...)). The same fix should be applied in agentos.py line 128.
| env = os.environ.copy() | ||
| # Sanitize environment to prevent leaking caller credentials | ||
| sensitive_pattern = re.compile(r'(?i)(api_key|token|secret|password|credential)') | ||
| env = {k: v for k, v in env.items() if not sensitive_pattern.search(k)} |
There was a problem hiding this comment.
The broad pattern (?i)(api_key|token|secret|password|credential) will strip OPENAI_API_KEY, ANTHROPIC_API_KEY, GITHUB_TOKEN, and any other variable matching those substrings before passing the environment to child MCP processes. Many MCP servers (e.g., LLM-backed tools, GitHub MCP) require exactly these variables to function.
Since this only applies when the caller doesn't supply an explicit env kwarg, legitimate workflows that rely on the inherited environment will silently fail. Consider documenting this behavior or making the allowlist/denylist configurable, rather than applying a blanket strip.
Manual configuration of the automated PR reviewer integration via GitHub App.
Summary by CodeRabbit
Release Notes
New Features
Security & Improvements
Documentation
Chores