Skip to content

Commit 11f9246

Browse files
github-actions[bot]GitHub CopilotCopilot
authored
Import AI Moderator workflow from github/gh-aw (#159)
This commit imports the AI Moderator workflow which automatically detects spam, link spam, and AI-generated content in GitHub issues, comments, and pull requests. Changes: - Added workflows/ai-moderator.md with the workflow definition - Added docs/ai-moderator.md with comprehensive documentation - Updated README.md to include the workflow in the Triage Workflows section Source: https://github.com/github/gh-aw/blob/main/.github/workflows/ai-moderator.md The workflow will be compiled by CI when this PR is merged. Co-authored-by: GitHub Copilot <copilot@github.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent 82dcb1c commit 11f9246

3 files changed

Lines changed: 197 additions & 0 deletions

File tree

‎README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7,6 +7,7 @@ A sample family of reusable [GitHub Agentic Workflows](https://github.github.com
77
### Triage Workflows
88

99
- [🏷️ Issue Triage](docs/issue-triage.md) - Triage issues and pull requests
10+
- [🛡️ AI Moderator](docs/ai-moderator.md) - Automatically detect and moderate spam, link spam, and AI-generated content
1011

1112
## Fault Analysis Workflows
1213

‎docs/ai-moderator.md‎

Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
# 🛡️ AI Moderator
2+
3+
> For an overview of all available workflows, see the [main README](../README.md).
4+
5+
The [AI Moderator workflow](../workflows/ai-moderator.md?plain=1) automatically detects spam, link spam, and AI-generated content in GitHub issues, comments, and pull requests. This workflow helps maintain quality discussions and protect your repository from malicious or low-quality contributions.
6+
7+
## Installation
8+
9+
````bash
10+
# Install the 'gh aw' extension
11+
gh extension install github/gh-aw
12+
13+
# Add the workflow to your repository
14+
gh aw add-wizard githubnext/agentics/ai-moderator
15+
````
16+
17+
This walks you through adding the workflow to your repository.
18+
19+
You must also [choose a coding agent](https://github.github.com/gh-aw/reference/engines/) and add an API key secret for the agent to your repository.
20+
21+
You can't start a run of this workflow directly as it is triggered automatically when issues, comments, or pull requests are created.
22+
23+
## Configuration
24+
25+
This workflow works out of the box with sensible defaults. You can customize:
26+
27+
- The labels to apply for different detection types (spam, link-spam, ai-generated, ai-inspected)
28+
- The user roles that are skipped (defaults to admins, maintainers, write access, and triage)
29+
- The bots to skip (defaults to github-actions and copilot)
30+
- Rate limiting settings to control how often the workflow runs
31+
- Detection criteria and thresholds in the workflow prompt
32+
33+
After editing run `gh aw compile` to update the workflow and commit all changes to the default branch.
34+
35+
## What it reads from GitHub
36+
37+
- New issues and their content
38+
- Comments on issues and pull requests
39+
- Pull request diffs and file changes
40+
- Repository context to evaluate content relevance
41+
- User information and roles
42+
43+
## What it creates
44+
45+
- Adds labels to issues and pull requests (`spam`, `link-spam`, `ai-generated`, or `ai-inspected`)
46+
- Hides comments that are detected as spam
47+
- Requires `issues: write` and `pull-requests: write` permissions (note: the workflow is configured with read-only by default, so you'll need to adjust permissions if you want it to take action)
48+
49+
## What web searches it performs
50+
51+
This workflow does not perform web searches.
52+
53+
## Human in the loop
54+
55+
- Review labels applied to ensure accurate spam detection
56+
- Monitor for false positives and adjust detection criteria if needed
57+
- Override moderation decisions when the AI makes mistakes
58+
- Validate hidden comments and unhide legitimate content if necessary
59+
- Adjust the workflow configuration based on your repository's needs
60+
- Consider exempting trusted contributors from moderation if desired
61+
- Disable or uninstall the workflow if moderation automation is not providing value

‎workflows/ai-moderator.md‎

Lines changed: 135 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,135 @@
1+
---
2+
timeout-minutes: 5
3+
roles: all
4+
on:
5+
issues:
6+
types: [opened]
7+
lock-for-agent: true
8+
issue_comment:
9+
types: [created]
10+
lock-for-agent: true
11+
pull_request:
12+
types: [opened]
13+
forks: "*"
14+
skip-roles: [admin, maintainer, write, triage]
15+
skip-bots: [github-actions, copilot]
16+
rate-limit:
17+
max: 5
18+
window: 60
19+
concurrency:
20+
group: "gh-aw-${{ github.workflow }}-${{ github.event.issue.number || github.event.pull_request.number }}"
21+
cancel-in-progress: false
22+
engine: codex
23+
tools:
24+
github:
25+
mode: local
26+
read-only: true
27+
toolsets: [default]
28+
permissions:
29+
contents: read
30+
issues: read
31+
pull-requests: read
32+
safe-outputs:
33+
add-labels:
34+
allowed: [spam, ai-generated, link-spam, ai-inspected]
35+
target: "*"
36+
hide-comment:
37+
max: 5
38+
allowed-reasons: [spam]
39+
threat-detection: false
40+
source: https://github.com/github/gh-aw/blob/main/.github/workflows/ai-moderator.md
41+
---
42+
# AI Moderator
43+
44+
You are an AI-powered moderation system that automatically detects spam, link spam, and AI-generated content in GitHub issues and comments.
45+
46+
## Context
47+
48+
1. Use the GitHub MCP server tools to fetch the original context (see github context), unsanitized content directly from GitHub API
49+
2. Do NOT use the pre-sanitized text from the activation job - fetch fresh content to analyze the original user input
50+
3. **For Pull Requests**: Use `pull_request_read` with method `get_diff` to fetch the PR diff and analyze the changes for spam patterns
51+
52+
## Detection Tasks
53+
54+
Perform the following detection analyses on the content:
55+
56+
### 1. Generic Spam Detection
57+
58+
Analyze for spam indicators:
59+
- Promotional content or advertisements
60+
- Irrelevant links or URLs
61+
- Repetitive text patterns
62+
- Low-quality or nonsensical content
63+
- Requests for personal information
64+
- Cryptocurrency or financial scams
65+
- Content that doesn't relate to the repository's purpose
66+
67+
### 2. Link Spam Detection
68+
69+
Analyze for link spam indicators:
70+
- Multiple unrelated links
71+
- Links to promotional websites
72+
- Short URL services used to hide destinations (bit.ly, tinyurl, etc.)
73+
- Links to cryptocurrency, gambling, or adult content
74+
- Links that don't relate to the repository or issue topic
75+
- Suspicious domains or newly registered domains
76+
- Links to download executables or suspicious files
77+
78+
### 3. AI-Generated Content Detection
79+
80+
Analyze for AI-generated content indicators:
81+
- Use of em-dashes (—) in casual contexts
82+
- Excessive use of emoji, especially in technical discussions
83+
- Perfect grammar and punctuation in informal settings
84+
- Constructions like "it's not X - it's Y" or "X isn't just Y - it's Z"
85+
- Overly formal paragraph responses to casual questions
86+
- Enthusiastic but content-free responses ("That's incredible!", "Amazing!")
87+
- "Snappy" quips that sound clever but add little substance
88+
- Generic excitement without specific technical engagement
89+
- Perfectly structured responses that lack natural conversational flow
90+
- Responses that sound like they're trying too hard to be engaging
91+
92+
Human-written content typically has:
93+
- Natural imperfections in grammar and spelling
94+
- Casual internet language and slang
95+
- Specific technical details and personal experiences
96+
- Natural conversational flow with genuine questions or frustrations
97+
- Authentic emotional reactions to technical problems
98+
99+
## Actions
100+
101+
Based on your analysis:
102+
103+
1. **For Issues** (when issue number is present):
104+
- If generic spam is detected, use the `add-labels` safe output to add the `spam` label to the issue
105+
- If link spam is detected, use the `add-labels` safe output to add the `link-spam` label to the issue
106+
- If AI-generated content is detected, use the `add-labels` safe output to add the `ai-generated` label to the issue
107+
- Multiple labels can be added if multiple types are detected
108+
- **If no warnings or issues are found** and the content appears legitimate and on-topic, use the `add-labels` safe output to add the `ai-inspected` label to indicate the issue has been reviewed and no threats were found
109+
- **If workflow_dispatch** was used, ensure the labels are applied to the correct issue/PR as specified in the input URL when calling `add-labels`
110+
111+
2. **For Comments** (when comment ID is present):
112+
- If any type of spam, link spam, or AI-generated spam is detected:
113+
- Use the `hide-comment` safe output to hide the comment with reason 'spam'
114+
- Also add appropriate labels to the parent issue as described above
115+
- If the comment appears legitimate and on-topic, add the `ai-inspected` label to the parent issue
116+
117+
3. **For Pull Requests** (when pull request number is present):
118+
- Fetch the PR diff using `pull_request_read` with method `get_diff`
119+
- Analyze the diff for spam patterns:
120+
- Large amounts of promotional content or links in code comments
121+
- Suspicious file additions (e.g., cryptocurrency miners, malware)
122+
- Mass link injection across multiple files
123+
- AI-generated code comments with promotional content
124+
- If spam, link spam, or suspicious patterns are detected:
125+
- Use the `add-labels` safe output to add appropriate labels (`spam`, `link-spam`, `ai-generated`)
126+
- **If no warnings or issues are found** and the PR appears legitimate, use the `add-labels` safe output to add the `ai-inspected` label
127+
128+
## Important Guidelines
129+
130+
- Be conservative with detections to avoid false positives
131+
- Consider the repository context when evaluating relevance
132+
- Technical discussions may naturally contain links to resources, documentation, or related issues
133+
- New contributors may have less polished writing - this doesn't necessarily indicate AI generation
134+
- Provide clear reasoning for each detection in your analysis
135+
- Only take action if you have high confidence in the detection

0 commit comments

Comments
 (0)