Skip to content

smoke-codex.md

Latest commit

 

History

History
85 lines (77 loc) · 3.75 KB

smoke-codex.md

File metadata and controls

85 lines (77 loc) · 3.75 KB
description Smoke test workflow that validates Codex engine functionality by testing AWF firewall capabilities
true
schedule workflow_dispatch pull_request reaction
every 12h
types
opened
synchronize
reopened
hooray
permissions
contents issues pull-requests discussions
read
read
read
read
name Smoke Codex
engine codex
strict true
imports
shared/gh.md
shared/mcp/tavily.md
shared/reporting.md
shared/github-queries-safe-input.md
network
allowed
defaults
github
playwright
tools
cache-memory github playwright edit bash
true
allowed_domains
github.com
*
sandbox
mcp
container
ghcr.io/github/gh-aw-mcpg
safe-outputs
add-comment create-issue add-labels hide-comment messages
hide-older-comments max
true
2
expires close-older-issues
2h
true
allowed
smoke-codex
footer run-started run-success run-failure
> 🔮 *The oracle has spoken through [{workflow_name}]({run_url})*
🔮 The ancient spirits stir... [{workflow_name}]({run_url}) awakens to divine this {event_type}...
✨ The prophecy is fulfilled... [{workflow_name}]({run_url}) has completed its mystical journey. The stars align. 🌟
🌑 The shadows whisper... [{workflow_name}]({run_url}) {status}. The oracle requires further meditation...
timeout-minutes 15

Smoke Test: Codex Engine Validation

IMPORTANT: Keep all outputs extremely short and concise. Use single-line responses where possible. No verbose explanations.

Test Requirements

  1. GitHub MCP Testing: Review the last 2 merged pull requests in ${{ github.repository }}
  2. Safe Inputs GH CLI Testing: Use the safeinputs-gh tool to query 2 pull requests from ${{ github.repository }} (use args: "pr list --repo ${{ github.repository }} --limit 2 --json number,title,author")
  3. Playwright Testing: Use the playwright tools to navigate to https://github.com and verify the page title contains "GitHub" (do NOT try to install playwright - use the provided MCP tools)
  4. Tavily Web Search Testing: Use the Tavily MCP server to perform a web search for "GitHub Agentic Workflows Firewall" and verify that results are returned with at least one item
  5. File Writing Testing: Create a test file /tmp/gh-aw/agent/smoke-test-codex-${{ github.run_id }}.txt with content "Smoke test passed for Codex at $(date)" (create the directory if it doesn't exist)
  6. Bash Tool Testing: Execute bash commands to verify file creation was successful (use cat to read the file back)
  7. Discussion Interaction Testing:
    • Use the github-discussion-query safe-input tool with params: limit=1, jq=".[0]" to get the latest discussion from ${{ github.repository }}
    • Extract the discussion number from the result (e.g., if the result is {"number": 123, "title": "...", ...}, extract 123)
    • Use the add_comment tool with discussion_number: <extracted_number> to add a mystical, oracle-themed comment stating that the smoke test agent was here
  8. Build AWF: Run npm ci && npm run build to verify the agent can successfully build the AWF project. If the command fails, mark this test as ❌ and report the failure.

Output

Add a very brief comment (max 5-10 lines) to the current pull request with:

  • PR titles only (no descriptions)
  • ✅ or ❌ for each test result
  • Overall status: PASS or FAIL

Use the add_comment tool to add a mystical oracle-themed comment to the latest discussion (using the discussion_number you extracted in step 7) - be creative and use mystical language like "🔮 The ancient spirits stir..."

If all tests pass:

  • Use the add_labels safe-output tool to add the label smoke-codex to the pull request