Skip to content

Commit 5553157

Browse files
authored
feat(integrations): execute GitHub issue import + task traceability (#565)
## Summary - POST /api/v2/integrations/github/import turns selected GitHub issues into CodeFRAME tasks (title + body + labels footer), linked via github_issue_number + external_url; PRs/missing-issues/fetch-failures mapped to 422/404/502, malformed repo 409. - URL-keyed dedup backed by a UNIQUE(workspace_id, external_url) index (atomic across concurrent imports); two-phase import with rollback on mid-create DB error. - Auto-close on DONE fired from core tasks.update_status (web UI + CLI + agent/batch), targeting each task's source repo from external_url; off the caller's path (event loop in server, non-daemon thread in CLI). - Frontend: GitHubIssueBadge, import flow in TaskBoardView (progress + in-modal error + summary), badge + auto-close checkbox in TaskDetailModal. ## Validation - Tests: All passing — 2949 v2 backend (CI + local), 955 frontend, npm build clean, ruff + eslint clean - Test mutation check: tests assert outcomes (DB state, dispatch targets, rendered DOM) - Internal review: claude-review passed (advisory); 2 findings applied - Cross-family review: codex (14 iterative passes — every Critical/Major fixed or rebutted) + CodeRabbit (no Critical/Major; docstring-coverage advisory only) - Final feedback triage: all addressed; rebuttals documented in commit messages + PR body - Demo: all 4 acceptance criteria verified with outcome evidence (real persisted DB rows + rendered badge DOM) Closes #565
1 parent 4010f38 commit 5553157

21 files changed

Lines changed: 2368 additions & 76 deletions

CLAUDE.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -36,9 +36,11 @@ If you are an agent working in this repo: **do not improvise architecture**. Fol
3636

3737
### Current Focus: Phase 4A
3838

39-
**Phase 5.5 is in progress** — GitHub Issues import. Repo connection via PAT (#563) is **complete**: Settings → **Integrations** tab connects a GitHub repo with a Personal Access Token. Backend `POST/DELETE/GET /api/v2/integrations/github/{connect,disconnect,status}` (`ui/routers/github_integrations_v2.py`). Validation is headless in `core/github_connect_service.py` (httpx; verifies token, repo visibility, and issues-read access; typed errors → 401/404/403). The PAT is stored machine-wide via `CredentialManager` (`CredentialProvider.GIT_GITHUB`, the #555 pattern) and **never returned in any response**; non-secret repo metadata persists per-workspace in `.codeframe/github_integration.json` (`core/github_integration_config.py`). Frontend: `GitHubIntegrationCard` + `integrationsApi`.
39+
**Phase 5.5 is complete** — GitHub Issues import. Repo connection via PAT (#563) is **complete**: Settings → **Integrations** tab connects a GitHub repo with a Personal Access Token. Backend `POST/DELETE/GET /api/v2/integrations/github/{connect,disconnect,status}` (`ui/routers/github_integrations_v2.py`). Validation is headless in `core/github_connect_service.py` (httpx; verifies token, repo visibility, and issues-read access; typed errors → 401/404/403). The PAT is stored machine-wide via `CredentialManager` (`CredentialProvider.GIT_GITHUB`, the #555 pattern) and **never returned in any response**; non-secret repo metadata persists per-workspace in `.codeframe/github_integration.json` (`core/github_integration_config.py`). Frontend: `GitHubIntegrationCard` + `integrationsApi`.
4040

41-
Issue **browse** (#564) is **complete**: `GET /api/v2/integrations/github/issues?page&per_page&search&label` on the same router lists the connected repo's **open** issues (PRs filtered out) — repo from `.codeframe/github_integration.json`, PAT from `CredentialManager`, **409** when not connected. Headless fetch in `core/github_issues_service.py` (`list_issues`): plain `/repos/{o}/{r}/issues` by default, routes to `/search/issues` for free-text search, `labels=` filter, `Link`-header pagination, 60s in-process TTL cache, typed errors → 401/403/502. Frontend: `GitHubIssueImportModal` (paginated list, debounced search, label filter, multi-select that persists across pages, select-all-on-page, Import-Selected gated on ≥1) + `integrationsApi.getIssues`; an **Import from GitHub** button on `/tasks` (`TaskBoardView`) shown only when connected. The import action itself is the remaining 5.5 work (#565).
41+
Issue **browse** (#564) is **complete**: `GET /api/v2/integrations/github/issues?page&per_page&search&label` on the same router lists the connected repo's **open** issues (PRs filtered out) — repo from `.codeframe/github_integration.json`, PAT from `CredentialManager`, **409** when not connected. Headless fetch in `core/github_issues_service.py` (`list_issues`): plain `/repos/{o}/{r}/issues` by default, routes to `/search/issues` for free-text search, `labels=` filter, `Link`-header pagination, 60s in-process TTL cache, typed errors → 401/403/502. Frontend: `GitHubIssueImportModal` (paginated list, debounced search, label filter, multi-select that persists across pages, select-all-on-page, Import-Selected gated on ≥1) + `integrationsApi.getIssues`; an **Import from GitHub** button on `/tasks` (`TaskBoardView`) shown only when connected.
42+
43+
Issue **import + traceability** (#565) is **complete**: `POST /api/v2/integrations/github/import` (same router) turns selected issues into tasks — title verbatim, body as description (+ a best-effort `**Labels:**` footer), linked via `github_issue_number` + `external_url`; PRs are rejected (`NotAnIssueError`→422), missing issues 404, fetch failures 502, malformed saved repo 409. Import is two-phase (fetch+dedupe all, then create) with rollback on a mid-create DB error; dedup is keyed on the full issue URL and backed by a `UNIQUE(workspace_id, external_url)` index (atomic across concurrent imports). Issue ops live in `core/github_issues_service.py` (`get_issue`, `close_issue`). **Auto-close**: marking an opted-in imported task DONE closes the linked issue — fired from core `tasks.update_status` so the web UI, CLI, and agent/batch paths all trigger it; the close targets the task's *source* repo parsed from `external_url` (not the live connection) and runs off the caller's path (event loop in the server, non-daemon thread in CLI). `TaskResponse` exposes the three traceability fields; `PATCH /api/v2/tasks/{id}` accepts `auto_close_github_issue` (persist-first + rollback-on-rejected-transition, with late opt-in on already-DONE tasks). Frontend: `GitHubIssueBadge`, import wiring in `TaskBoardView` (progress, in-modal error, summary banner), badge + auto-close checkbox in `TaskDetailModal`, `integrationsApi.importIssues` + `tasksApi.updateGitHubSettings`. **Known limitation**: auto-close uses the single machine-wide GIT_GITHUB PAT, so closing an older imported repo's issue after reconnecting to a different repo may fail if that PAT lacks access.
4244

4345
**Phase 5.4 is complete** — PRD stress-test web UI: trigger + streaming (#561). Backend: `GET /api/v2/prd/stress-test` SSE endpoint streams `goals_extracted`, `goal_analyzed`, `complete`, and `error` events from `core/prd_stress_test.py:stress_test_prd_stream()`, resolving the LLM provider via the standard chain and applying the standard rate limit. Frontend: `useStressTestStream` hook manages the SSE connection and event accumulation; `StressTestModal` renders the streaming progress and is opened via a "Stress Test" button on the `/prd` page (enabled only when a PRD exists). Results rendering + refinement (#562) is **complete**: the `complete` SSE event now carries structured, severity-tagged `ambiguities` (`Ambiguity.severity` is `"blocking"`/`"warning"`); `StressTestModal` shows a results view of `AmbiguityCard`s (question text, severity badge, answer textarea) with an "X of Y answered" progress indicator and a **[Refine PRD]** button (disabled until every blocking ambiguity is answered). Refine posts to `POST /api/v2/prd/stress-test/refine`, which folds the answers into a new PRD version via `resolve_ambiguities_into_prd` (offloaded with `asyncio.to_thread`) and `prd.create_new_version`, then `mutatePrd` reflects it in the editor.
4446

codeframe/core/github_issues_service.py

Lines changed: 218 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -37,6 +37,23 @@
3737

3838
_TIMEOUT = 15.0
3939

40+
41+
class NotAnIssueError(Exception):
42+
"""The requested number refers to a pull request, not an issue (#565).
43+
44+
Intentionally NOT a ``GitHubConnectError`` subclass: callers map it to a
45+
client error (the caller sent a PR number), not a GitHub upstream failure.
46+
"""
47+
48+
49+
class IssueNotFoundError(Exception):
50+
"""The requested issue number does not exist in the repo (404) (#565).
51+
52+
Intentionally NOT a ``GitHubConnectError`` subclass: a missing/stale issue
53+
number is a client error (bad payload), not a GitHub upstream failure, so
54+
callers map it to a 4xx rather than a 502.
55+
"""
56+
4057
# Parse the ``page=N`` query param out of a Link header's rel="last" URL.
4158
_LAST_PAGE_RE = re.compile(r'[?&]page=(\d+)[^>]*>;\s*rel="last"')
4259

@@ -190,6 +207,207 @@ async def _list_issues(
190207
return issues, total
191208

192209

210+
class GitHubIssueDetail(TypedDict):
211+
number: int
212+
title: str
213+
body: str
214+
labels: list[str]
215+
html_url: str
216+
217+
218+
async def get_issue(
219+
pat: str,
220+
repo: str,
221+
number: int,
222+
*,
223+
client: Optional[httpx.AsyncClient] = None,
224+
) -> GitHubIssueDetail:
225+
"""Fetch a single issue's details for import (issue #565).
226+
227+
Unlike the list endpoint, this returns the issue ``body`` so the importer
228+
can populate the task description.
229+
230+
Args:
231+
pat: GitHub Personal Access Token.
232+
repo: Repository in ``owner/repo`` format.
233+
number: Issue number to fetch.
234+
client: Optional httpx client (injected by tests). When ``None`` a
235+
short-lived client is created and closed internally.
236+
237+
Returns:
238+
``{number, title, body, labels, html_url}`` — ``body`` is normalized to
239+
``""`` when GitHub returns null.
240+
241+
Raises:
242+
ValueError: if ``repo`` is not a valid ``owner/repo`` string.
243+
InvalidTokenError: GitHub returned 401.
244+
InsufficientScopeError: the token cannot read issues (403).
245+
GitHubConnectError: any other non-success response or network error.
246+
"""
247+
owner, name = parse_repo(repo)
248+
249+
own_client = client is None
250+
if own_client:
251+
client = httpx.AsyncClient(timeout=_TIMEOUT)
252+
try:
253+
try:
254+
resp = await client.get(
255+
f"{GITHUB_API_BASE}/repos/{owner}/{name}/issues/{number}",
256+
headers=_headers(pat),
257+
)
258+
except httpx.HTTPError as exc:
259+
logger.warning("GitHub get issue failed: %s", type(exc).__name__)
260+
raise GitHubConnectError("Could not reach GitHub. Try again later.")
261+
262+
# A 404 on /issues/{n} is ambiguous: the issue may genuinely not exist,
263+
# OR the repo/token became inaccessible (renamed/deleted repo, rotated
264+
# token). Probe the repo to tell a client typo (-> IssueNotFoundError,
265+
# 404) apart from a broken integration (-> connect/auth error) so callers
266+
# get the right recovery path. The probe only runs on the 404 path.
267+
if resp.status_code == 404:
268+
try:
269+
repo_resp = await client.get(
270+
f"{GITHUB_API_BASE}/repos/{owner}/{name}", headers=_headers(pat)
271+
)
272+
except httpx.HTTPError as exc:
273+
logger.warning("GitHub repo probe failed: %s", type(exc).__name__)
274+
raise GitHubConnectError("Could not reach GitHub. Try again later.")
275+
if repo_resp.status_code == 401:
276+
raise InvalidTokenError("Invalid GitHub token.")
277+
if repo_resp.status_code == 403:
278+
raise InsufficientScopeError(
279+
"Token lacks access to this repository."
280+
)
281+
if repo_resp.status_code == 404:
282+
raise GitHubConnectError(
283+
f"Repository '{repo}' is no longer accessible."
284+
)
285+
if repo_resp.status_code >= 400:
286+
# Rate limit / 5xx / other failure on the probe — a real upstream
287+
# problem, NOT a missing issue. Surface it as such so the caller
288+
# retries rather than blaming the issue number.
289+
raise GitHubConnectError(
290+
f"GitHub repo check returned status {repo_resp.status_code}."
291+
)
292+
# Repo probe succeeded (2xx) → the issue itself genuinely does not
293+
# exist. (A 3xx would also land here, but GitHub answers repo lookups
294+
# with 2xx/4xx, not redirects, for this endpoint.)
295+
if repo_resp.status_code >= 300:
296+
raise GitHubConnectError(
297+
f"GitHub repo check returned status {repo_resp.status_code}."
298+
)
299+
raise IssueNotFoundError(f"Issue #{number} was not found in '{repo}'.")
300+
_raise_for_status(resp.status_code, context="get issue")
301+
302+
raw = resp.json()
303+
if not isinstance(raw, dict):
304+
raw = {}
305+
# The issues endpoint also returns pull requests (a PR is an issue with a
306+
# ``pull_request`` member). Reject them so the import stays consistent
307+
# with ``list_issues`` (which excludes PRs) and never links a PR as an
308+
# issue.
309+
if "pull_request" in raw:
310+
raise NotAnIssueError(f"#{number} is a pull request, not an issue.")
311+
labels_raw = raw.get("labels") or []
312+
labels = [
313+
(lbl.get("name") if isinstance(lbl, dict) else str(lbl))
314+
for lbl in labels_raw
315+
]
316+
labels = [n for n in labels if n]
317+
return {
318+
"number": int(raw.get("number", number)),
319+
"title": str(raw.get("title") or ""),
320+
"body": str(raw.get("body") or ""),
321+
"labels": labels,
322+
"html_url": str(raw.get("html_url") or ""),
323+
}
324+
finally:
325+
if own_client:
326+
await client.aclose()
327+
328+
329+
async def close_issue(
330+
pat: str,
331+
repo: str,
332+
number: int,
333+
*,
334+
comment: Optional[str] = None,
335+
timeout: float = _TIMEOUT,
336+
client: Optional[httpx.AsyncClient] = None,
337+
) -> bool:
338+
"""Close a GitHub issue, optionally posting a comment first (issue #565).
339+
340+
Args:
341+
pat: GitHub Personal Access Token.
342+
repo: Repository in ``owner/repo`` format.
343+
number: Issue number to close.
344+
comment: Optional comment body to post before closing.
345+
timeout: HTTP timeout in seconds for the (self-created) client. Auto-close
346+
passes a short value so a hung close never stalls a caller for long.
347+
client: Optional httpx client (injected by tests). When ``None`` a
348+
short-lived client is created and closed internally.
349+
350+
Returns:
351+
``True`` when the issue was closed.
352+
353+
Raises:
354+
ValueError: if ``repo`` is not a valid ``owner/repo`` string.
355+
InvalidTokenError: GitHub returned 401.
356+
InsufficientScopeError: the token cannot write issues (403).
357+
GitHubConnectError: any other non-success response or network error.
358+
"""
359+
owner, name = parse_repo(repo)
360+
361+
own_client = client is None
362+
if own_client:
363+
client = httpx.AsyncClient(timeout=timeout)
364+
try:
365+
headers = _headers(pat)
366+
base = f"{GITHUB_API_BASE}/repos/{owner}/{name}/issues/{number}"
367+
368+
if comment:
369+
# Best-effort: the comment is cosmetic. A failure to post it (locked
370+
# issue, repo with commenting disabled, transient error) must NOT
371+
# prevent the close itself, which is the operation that matters.
372+
try:
373+
cresp = await client.post(
374+
f"{base}/comments", json={"body": comment}, headers=headers
375+
)
376+
if cresp.status_code >= 400:
377+
logger.warning(
378+
"GitHub issue comment returned %s; closing anyway.",
379+
cresp.status_code,
380+
)
381+
except httpx.HTTPError as exc:
382+
logger.warning(
383+
"GitHub issue comment failed (%s); closing anyway.",
384+
type(exc).__name__,
385+
)
386+
387+
try:
388+
resp = await client.patch(
389+
base, json={"state": "closed"}, headers=headers
390+
)
391+
except httpx.HTTPError as exc:
392+
logger.warning("GitHub close issue failed: %s", type(exc).__name__)
393+
raise GitHubConnectError("Could not reach GitHub. Try again later.")
394+
395+
_raise_for_status(resp.status_code, context="close issue")
396+
# A redirect (3xx) — e.g. a moved/renamed/transferred repo — means the
397+
# PATCH was NOT applied (httpx does not follow redirects by default), so
398+
# the issue is still open. Treat it as a failure rather than reporting a
399+
# silent success.
400+
if resp.status_code >= 300:
401+
raise GitHubConnectError(
402+
f"GitHub close returned status {resp.status_code}; "
403+
"issue was not closed (repository may have moved)."
404+
)
405+
return True
406+
finally:
407+
if own_client:
408+
await client.aclose()
409+
410+
193411
async def _search_issues(
194412
client: httpx.AsyncClient,
195413
headers: dict[str, str],

0 commit comments

Comments
 (0)