Skip to content

[hyper-pm] GHA-PR-LLM-AC-005: Orchestrate AC review and post advisory PR comment #259

@nicnocquee

Description

@nicnocquee

Summary

Wire GitHub data loading, AC extraction, and the LLM adapter into one pipeline: build a prompt that requests JSON matching an agreed schema (criterion id, status, rationale, evidence paths), validate the response, and post an accessible markdown advisory comment mapping each criterion to satisfied/partial/not found/not applicable with explicit uncertainty.

Scope

  • In scope: End-to-end path for same-repo PRs with linked issues; synthetic test harness or fixtures proving mapping quality; GitHub API retry for rate limits (once + backoff per PRD); LLM timeout/5xx single retry; fallback PR comment when validation fails (no raw dump).
  • Out of scope: Merging duplicate bot comments (GHA-PR-LLM-AC-006), diff/token truncation limits (GHA-PR-LLM-AC-007) except minimal stub if needed.

Acceptance criteria

  • Given a synthetic PR fixture that satisfies 2 of 3 stated acceptance criteria, when the action runs in a test harness, then output marks those two as satisfied or partial with evidence pointers (paths or PR areas) and the third as not found or partial with rationale.
  • Each criterion in the user-visible output includes status, 1–3 sentence rationale, and pointers; uncertain cases state uncertainty explicitly.
  • PR comment uses markdown headings and lists for structure (PRD §6 accessibility).
  • JSON schema for LLM output is defined and validated before posting; on failure, PR receives “automated check failed; see logs” (or equivalent) without raw model text.
  • GitHub API rate limit: one retry with backoff, then clear job failure; LLM 5xx/timeout: one retry, then advisory failure path.

Dependencies

  • Depends on: GHA-PR-LLM-AC-001, GHA-PR-LLM-AC-002, GHA-PR-LLM-AC-003, GHA-PR-LLM-AC-004
  • Blocks: GHA-PR-LLM-AC-006, GHA-PR-LLM-AC-007

Notes

Document the exact JSON schema in repo docs (PRD rubric follow-up). Prompt injection mitigations: treat issue/PR text as untrusted; instruct model to ignore embedded instructions; sanitize rendered comment.

Stacked delivery

N/A — v1 is advisory-only; no user-facing feature flag required beyond optional repo enablement documented in rollout.

PRD: https://github.com/hyperjumptech/mediapulse/blob/prds/pr-github-llm-ticket-acceptance.prd.md

{
  "hyper_pm_id": "01KP1FZ5ZZVQ9RWMKHDYRESDME",
  "type": "ticket",
  "parent_ids": {
    "epic": "01KP1JW0X7SZ4SYS4N68M8ZDWA",
    "story": "01KP1JW25FFFPZAVZHQV4RZYNJ"
  },
  "depends_on": [
    "01KP1FZ601BJV71KX0Z84K466J",
    "01KP1FZ600JR9ZC7GE427GG527"
  ]
}

Metadata

Metadata

Assignees

Labels

Type

No type
No fields configured for issues without a type.

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions