-
-
Notifications
You must be signed in to change notification settings - Fork 1.1k
feat: integrate PraisonAI PR Reviewer in CI/CD pipeline #1333
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
4b806f8
0889f80
ca4ee2e
a2b44dc
8907022
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,116 @@ | ||
| framework: "praisonai" | ||
| topic: "Pull Request Code Review Analysis" | ||
|
|
||
| roles: | ||
| security_reviewer: | ||
| role: "Security Code Reviewer" | ||
| goal: "Identify security vulnerabilities, authentication issues, input validation problems, and unsafe code practices in the PR changes" | ||
| backstory: "You are a cybersecurity expert specializing in code analysis. You have extensive experience in identifying common security flaws like injection attacks, authentication bypasses, exposed secrets, and unsafe data handling patterns." | ||
| tools: | ||
| - "execute_command" | ||
|
|
||
| performance_reviewer: | ||
| role: "Performance Code Reviewer" | ||
| goal: "Analyze code changes for performance implications, identify bottlenecks, inefficient algorithms, and resource usage issues" | ||
| backstory: "You are a performance optimization specialist with deep knowledge of algorithm efficiency, memory management, and system performance. You excel at spotting code that could cause performance degradation." | ||
| tools: | ||
| - "execute_command" | ||
|
|
||
| maintainability_reviewer: | ||
| role: "Code Quality & Maintainability Reviewer" | ||
| goal: "Evaluate code structure, readability, documentation, naming conventions, and adherence to best practices" | ||
| backstory: "You are a software engineering expert focused on code quality and maintainability. You ensure code follows established patterns, is well-documented, and will be easy for future developers to understand and modify." | ||
| tools: | ||
| - "execute_command" | ||
|
|
||
| lead_reviewer: | ||
| role: "Lead Technical Reviewer" | ||
| goal: "Synthesize all review feedback, make final recommendations, and post comprehensive review comments to the GitHub PR" | ||
| backstory: "You are a senior technical lead responsible for final review decisions. You coordinate input from security, performance, and maintainability reviewers to provide balanced, actionable feedback to the development team." | ||
| tools: | ||
| - "execute_command" | ||
|
Comment on lines
+8
to
+31
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
All four agents are granted the This is a well-documented attack class against LLM agents that use tool access. Mitigations to consider:
|
||
|
|
||
|
qodo-code-review[bot] marked this conversation as resolved.
|
||
| steps: | ||
| - name: security_analysis | ||
| agent: security_reviewer | ||
| action: | | ||
| Analyze the PR diff for security issues: | ||
| 1. Extract PR details: `echo "$PR_DATA"` | ||
| 2. Get changed files: `echo "$CHANGED_FILES"` | ||
| 3. Review full diff: `gh pr diff ${PR_NUMBER}` | ||
| 4. Look for: | ||
| - Hardcoded secrets, API keys, passwords | ||
| - SQL injection vulnerabilities | ||
| - XSS vulnerabilities | ||
| - Authentication/authorization bypasses | ||
| - Unsafe file operations | ||
| - Command injection risks | ||
| - Missing input validation | ||
| - Exposed sensitive data | ||
| 5. Document findings with file paths and line numbers | ||
|
Comment on lines
+37
to
+50
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The agent steps reference At runtime, The workflow's - name: Run PraisonAI PR Review
env:
GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }}
OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
PR_NUMBER: ${{ github.event.pull_request.number || github.event.issue.number || inputs.pr_number }}
CHANGED_FILES: ${{ toJson(github.event.pull_request.changed_files) }}
run: |
praisonai agents --file .github/praisonai-reviewer.yaml |
||
| expected_output: "Detailed security analysis report with specific vulnerabilities found, severity levels, and remediation suggestions" | ||
|
|
||
| - name: performance_analysis | ||
| agent: performance_reviewer | ||
| action: | | ||
| Analyze the PR diff for performance issues: | ||
| 1. Review algorithm complexity changes | ||
| 2. Check for inefficient database queries | ||
| 3. Identify memory leaks or excessive allocations | ||
| 4. Look for hot-path regressions | ||
| 5. Check for blocking I/O operations | ||
| 6. Analyze module-level imports that could slow startup | ||
| 7. Review caching implementations | ||
| 8. Check for recursive operations without termination | ||
| 9. Document performance concerns with impact estimates | ||
| expected_output: "Performance analysis report highlighting bottlenecks, efficiency concerns, and optimization recommendations" | ||
|
|
||
| - name: maintainability_analysis | ||
| agent: maintainability_reviewer | ||
| action: | | ||
| Analyze the PR diff for code quality and maintainability: | ||
| 1. Check naming conventions consistency | ||
| 2. Evaluate code structure and organization | ||
| 3. Review documentation completeness | ||
| 4. Check for DRY violations | ||
| 5. Analyze error handling patterns | ||
| 6. Review test coverage for changes | ||
| 7. Check adherence to repository coding standards | ||
| 8. Evaluate API design consistency | ||
| 9. Look for technical debt introduction | ||
| expected_output: "Code quality analysis with specific recommendations for improving maintainability and readability" | ||
|
|
||
| - name: final_review | ||
| agent: lead_reviewer | ||
| action: | | ||
| Synthesize all reviews and post final comprehensive feedback: | ||
| 1. Collect input from security, performance, and maintainability reviews | ||
| 2. Prioritize findings by severity (Critical, High, Medium, Low) | ||
| 3. Create structured review comment using this format: | ||
|
|
||
| ## 📋 Review Summary | ||
| [2-3 sentence overview and assessment] | ||
|
|
||
| ## 🔍 General Feedback | ||
| [Overall patterns and observations] | ||
|
|
||
| ## 🎯 Specific Feedback | ||
| ### 🔴 Critical (if any) | ||
| [Security vulnerabilities, breaking changes, major bugs] | ||
|
|
||
| ### 🟡 High (if any) | ||
| [Performance issues, design flaws, significant bugs] | ||
|
|
||
| ### 🟢 Medium (if any) | ||
| [Code quality improvements, minor optimizations] | ||
|
|
||
| ### 🔵 Low (if any) | ||
| [Documentation, naming suggestions, minor refactoring] | ||
|
|
||
| ## ✅ Highlights (if any) | ||
| [Positive aspects worth mentioning] | ||
|
|
||
| 4. Post comprehensive review: `echo "[REVIEW_CONTENT]" | gh pr comment ${PR_NUMBER} -F -` | ||
| 5. If critical issues found, suggest not merging until resolved | ||
| expected_output: "Final review comment posted to GitHub PR with comprehensive analysis from all reviewers" | ||
| dependencies: [security_analysis, performance_analysis, maintainability_analysis] | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| name: PraisonAI PR Reviewer | ||
|
|
||
| on: | ||
| pull_request: | ||
| types: [opened, synchronize, reopened, ready_for_review] | ||
| issue_comment: | ||
| types: [created] | ||
| workflow_dispatch: | ||
| inputs: | ||
| pr_number: | ||
| description: 'Pull Request Number' | ||
| required: true | ||
| type: string | ||
|
|
||
| jobs: | ||
| review: | ||
| runs-on: ubuntu-latest | ||
| if: > | ||
| (github.event_name == 'pull_request' && github.event.pull_request.draft == false) || | ||
| github.event_name == 'workflow_dispatch' || | ||
| (github.event_name == 'issue_comment' && | ||
| github.event.issue.pull_request && | ||
| contains(github.event.comment.body, '@praisonai') && | ||
| github.actor != 'github-actions[bot]' && | ||
| github.event.comment.user.type != 'Bot' && | ||
| (github.event.comment.author_association == 'OWNER' || | ||
| github.event.comment.author_association == 'MEMBER' || | ||
| github.event.comment.author_association == 'COLLABORATOR')) | ||
|
|
||
| steps: | ||
| - name: Determine checkout ref | ||
| id: dest | ||
| run: | | ||
| if [ "${{ github.event_name }}" = "issue_comment" ]; then | ||
| echo "ref=refs/pull/${{ github.event.issue.number }}/head" >> "$GITHUB_OUTPUT" | ||
| elif [ "${{ github.event_name }}" = "workflow_dispatch" ]; then | ||
| echo "ref=refs/pull/${{ inputs.pr_number }}/head" >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "ref=${{ github.event.pull_request.head.sha }}" >> "$GITHUB_OUTPUT" | ||
| fi | ||
|
|
||
| - name: Checkout Repository | ||
| uses: actions/checkout@v4 | ||
| with: | ||
| ref: ${{ steps.dest.outputs.ref }} | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Generate GitHub App Token | ||
| id: generate_token | ||
| uses: tibdex/github-app-token@v2 | ||
| with: | ||
| app_id: ${{ secrets.PRAISONAI_APP_ID }} | ||
| private_key: ${{ secrets.PRAISONAI_APP_PRIVATE_KEY }} | ||
|
|
||
|
greptile-apps[bot] marked this conversation as resolved.
|
||
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: '3.11' | ||
|
|
||
| - name: Install PraisonAI | ||
| run: pip install "praisonai[all]" | ||
|
|
||
| - name: Run PraisonAI PR Review | ||
| env: | ||
| GITHUB_TOKEN: ${{ steps.generate_token.outputs.token }} | ||
| OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }} | ||
| run: | | ||
| praisonai agents --file .github/praisonai-reviewer.yaml | ||
|
greptile-apps[bot] marked this conversation as resolved.
|
||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
run_shell_commanddoesn’t appear to be a valid built-in tool name in this repo’s PraisonAI agents stack (the shell tool isexecute_command, which is also tracked in the approval registry). With the current tool name, agents will fail to resolve the tool at runtime. Switch these toexecute_command(and ensure the workflow auto-approves it in CI).