Skip to content

WIP: Add GitLab repository workspace support#1547

Draft
simple-agent-manager[bot] wants to merge 16 commits into
sam/gitlab-platform-config-wipfrom
sam/gitlab-repo-workspace-wip
Draft

WIP: Add GitLab repository workspace support#1547
simple-agent-manager[bot] wants to merge 16 commits into
sam/gitlab-platform-config-wipfrom
sam/gitlab-repo-workspace-wip

Conversation

@simple-agent-manager

@simple-agent-manager simple-agent-manager Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

WIP / stacking

Stack layer 2 on #1545 (sam/gitlab-platform-config-wip) at foundation SHA 570f4a5090fe05e5b382b585cd2ea8be5ec0d2be.

  • DRAFT PR. DO NOT MARK READY. DO NOT MERGE.
  • DO NOT DEPLOY TO STAGING, DO NOT RUN STAGING VERIFICATION, AND DO NOT MUTATE STAGING. Other agents are actively using staging.
  • Skip /do Phase 6 by explicit user instruction and document that staging was intentionally skipped.
  • Stop /do Phase 7 after updating the draft PR and observing/reporting CI; never merge.
  • Do not commit a task file directly to main. Keep task records/state on this existing feature branch because the stack is intentionally unmerged.
  • Avoid force-push unless absolutely unavoidable; preserve linear stacked ancestry with normal branch merges.
  • Do not alter or merge PR WIP: Add GitLab platform OAuth config #1545; consume its published branch head only.
  • Do not create, update, or deploy any staging environment.

Refs durable idea 01KV7ZFD6HZS5N7J45VA798KN1, mission 2b4af813-b907-4033-9145-b2087078d08b, and SAM task 01KX92JQ08DTCX277PY5DYKAHS / continuation task 01KX94W93DT9Q021HM5TB24VQ8.

Summary

  • Adds GitLab as a repository provider across shared/API/web contracts and project onboarding.
  • Stores GitLab repository identity in additive D1 sidecar migration 0091_project_gitlab_repositories.sql.
  • Re-verifies GitLab Developer-or-higher access at project/task/workspace token boundaries.
  • Supports GitLab branches, recursive tree, files, raw files, and compare through provider-neutral repo browsing.
  • Threads GitLab clone/host/path metadata through TaskRunner, node-agent, VM persistence/recovery, bootstrap, and credential helper.
  • Returns GitLab OAuth credentials only through the authenticated workspace callback path; no static GitLab token environment variable.
  • Keeps GitHub and Artifacts behavior on their existing paths.
  • Refresh repair: fails TaskRunner dispatch permanently when required GitLab sidecar metadata is missing.

Foundation merge / conflicts

Normal merge commit: e06e820b2d830361bbc7d51fceaa64bd8815afc3.

Resolved five conflicts:

  • apps/api/src/auth.ts: kept foundation auth updates and GitLab repository scopes.
  • apps/api/src/routes/mcp/instruction-tools.ts: kept foundation conversation/task context resolution.
  • apps/web/tests/playwright/project-onboarding-wizard-audit.spec.ts: kept refreshed audit harness plus GitLab mocks/scenarios.
  • packages/vm-agent/internal/server/server.go: kept foundation container git-command runner plus GitLab MR response handling.
  • packages/vm-agent/internal/server/workspaces.go: kept foundation standalone/runtime refactor plus GitLab provider clone/host/path request metadata.

The stale layer-2 migration number collided with the refreshed base's 00880090; it was renumbered to 0091.

Validation

Continuation run (2026-07-11, HEAD c26162c4a):

  • pnpm typecheck (api + web + shared) — pass.
  • pnpm lint (api + web + shared) — 0 errors (pre-existing warnings only).
  • Exhaustive pnpm --filter @simple-agent-manager/api test404/404 files, 5,922/5,922 tests pass (prior stale provider-count assertion now repaired; new GitLab route + service tests included).
  • New apps/api/tests/unit/routes/project-gitlab-creation.test.ts — 9/9 (route-level GitLab POST /api/projects behavioral coverage).
  • verifyGitLabProjectAccess service error branches — <Developer 403, GitLab 404 → not-found, group-access inheritance.
  • pnpm --filter @simple-agent-manager/web test -- project-onboarding — 40/40.
  • pnpm build (shared + api) — pass.
  • go test -race ./internal/server/... ./internal/bootstrap/... ./internal/persistence/... ./internal/config/... on Go 1.24+ — pass (incl. new TestIsAllowedCredentialPathForWorkspaceFallsBackToConfig).
  • go vet ./internal/server/... ./internal/bootstrap/... — clean.
  • Playwright onboarding audit — 15/15 across iPhone SE (375×667), iPhone 14 (390×844), Desktop (1280×800), including the GitLab provider/project/branch path with the corrected /api/gitlab/branches bare-array mock and overflow assertions.
  • Quality gates: migration-safety, do-migration-safety, file-sizes, source-contract-tests, wrangler-bindings, ast-checks — pass.
  • Remote CI / SonarCloud / CodSpeed — observing after this update. NOTE: the main CI workflow only triggers for PRs targeting main; this stacked PR targets sam/gitlab-platform-config-wip, so the main CI workflow does not register. SonarCloud + CodSpeed run and are green.

Staging Verification

Intentionally skipped by explicit user instruction.

  • Staging deployment green — NOT RUN; forbidden for this task.
  • Live staging Playwright — NOT RUN; forbidden for this task.
  • Existing staging workflows — NOT RUN; staging must not be mutated.
  • New feature on staging — NOT RUN; staging must not be mutated.
  • Real staging VM — NOT RUN; staging must not be mutated.
  • Local mobile/desktop mocked Playwright evidence captured (15/15).

No staging environment was created, updated, deployed, or queried for verification.

Data flow trace

  1. UI selector: ProjectOnboardingWizard / GitLabProjectSelector selects numeric GitLab project and branch.
  2. Shared request: CreateProjectRequest.gitlabProjectId crosses POST /api/projects.
  3. API: verifyGitLabProjectAccess resolves configured host, user OAuth token, Developer access, immutable numeric project ID, namespace path, clone URL, and default branch.
  4. D1: project row plus project_gitlab_repositories sidecar persist provider identity.
  5. Spawn gates: requireRepositoryUserAccess / requireRepositoryOwnerAccess re-verify identity.
  6. TaskRunner: resolveWorkspaceGitSource loads sidecar metadata and passes provider/clone/host/path to createWorkspaceOnNode; missing metadata fails permanently.
  7. VM: workspace metadata persists/reloads; bootstrap clones HTTPS and configures credential.useHttpPath=true.
  8. Credential callback: VM helper requests /git-credential?host=&path=; VM calls authenticated control-plane /:id/git-token; API re-verifies exact GitLab identity before returning oauth2 credentials. Verified consistent camelCase field mapping across node-agent.ts → Go createWorkspaceRequest json tags → shared CreateWorkspaceAgentRequestSchema.
  9. Repo browser: provider factory selects GitLabRepoBrowser for branches/tree/file/raw/compare.
  10. Completion: VM selects GitLab MR behavior only for repoProvider=gitlab; GitHub PR and Artifacts paths remain unchanged.

Untested gaps / explicit later layers

  • Live GitLab OAuth clone/push/MR behavior is not exercised because staging was explicitly forbidden.
  • Later security layer: fail-closed VM credential exchange when host/path are omitted on the request side; per-user/provider OAuth refresh serialization; stale-host/account token lifecycle.
  • Later discovery/UX layer: full pagination/server search, account link/re-auth, polished connection/onboarding UX, GitLabProjectSelector ARIA combobox + keyboard navigation + no-results / provider-switch reset polish.
  • Later MR layer: control-plane provider service, comprehensive change-request output, RepositoryHost URL validation, and a dedicated external HTTP client for GitLab MR API calls (host is trusted platform config, so this is defense-in-depth for the MR path).
  • GitLab OAuth scope configurability (kept at parity with the existing hardcoded GitHub scopes) and per-request GitLab config/D1 lookup de-duplication are deferred optimizations.
  • GitLab member/invite routes and webhook/task trigger support remain outside this stack layer.

Specialist Review Evidence

  • All local reviewers completed and in-scope findings addressed. (Prior session could not launch local subagents due to a bwrap sandbox failure; the 2026-07-11 continuation re-ran all 8 successfully.)
  • needs-human-review retained — see rationale below (draft stacked layer with documented cross-layer deferrals awaiting a human decision to advance the stack; NOT because any reviewer failed to run).
Reviewer Status Outcome
task-completion-validator ADDRESSED Flagged missing route-level GitLab POST /api/projects coverage. Added project-gitlab-creation.test.ts (9 scenarios).
test-engineer ADDRESSED Same route gap + service error branches. Added the route test and verifyGitLabProjectAccess <Developer/404/group-access tests. Verified VM create-workspace field mapping is consistent camelCase (no snake/camel mismatch).
cloudflare-specialist ADDRESSED Flagged unbounded gitlabFetch. Added GITLAB_API_TIMEOUT_MS (default 30s) via shared fetchWithTimeout. Migration 0091 confirmed additive/safe with correct FK + indexes.
constitution-validator ADDRESSED Principle XI: unbounded GitLab fetch fixed (see above). GitLab host derives from platform config (verified, no hardcoded gitlab.com). OAuth scope configurability deferred at GitHub parity.
security-auditor ADDRESSED Fixed VM credential path gate fail-open for unregistered workspaces (config fallback, mirrors host gate) + regression test. Confirmed callback-JWT auth, live re-verification, no token logging, provider/dialect consistency. Later security-layer items deferred + documented.
go-specialist ADDRESSED Same fail-open fix + test. MR-path RepositoryHost validation and dedicated external HTTP client deferred to the later MR layer (host is trusted platform config).
ui-ux-specialist ADDRESSED Fixed the Playwright /api/gitlab/branches mock (bare array matching the real route). Data path GitLab→API verified complete. ARIA/keyboard/no-results/provider-reset polish deferred to the later UX layer (matches existing selector pattern).
doc-sync-validator ADDRESSED Removed stale "future" from security.md GitLab row; documented required read_user+api GitLab OAuth scopes in self-hosting.mdx; documented GITLAB_API_TIMEOUT_MS in .env.example.

needs-human-review rationale

All eight required reviewers ran and every in-scope CRITICAL/HIGH finding was fixed in this branch. The label is retained (not removed) because this is an intentionally unmerged, draft stacked layer that a human must review before advancing the stack, and because several findings are legitimately deferred to explicit later stack layers (MR hardening, OAuth scope configurability, onboarding UX/ARIA polish). This draft must not be marked ready or merged.

Exceptions

  • Scope: /do Phase 6 and Phase 7 merge/readiness/deploy steps.
  • Rationale: explicit user instruction; other agents are actively using staging; this is an intentionally unmerged stacked draft.
  • Expiration: until a later explicit human instruction changes these constraints.

Agent Preflight (Required)

  • Preflight completed before code changes.

Classification

  • external-api-change
  • cross-component-change
  • business-logic-change
  • public-surface-change
  • docs-sync-change
  • security-sensitive-change
  • ui-change
  • infra-change

External References

  • Verified GitLab REST API contracts against the official documentation (Context7 unavailable in this runner; used official GitLab docs): Projects, Repository Branches/Tree/Files/Compare, Merge Requests, and OAuth scopes — https://docs.gitlab.com/ee/api/ and https://docs.gitlab.com/ee/integration/oauth_provider.html
  • GitLab API behavior is wrapped behind runtime-validated service helpers; every call is now bounded by GITLAB_API_TIMEOUT_MS.
  • No new external API assumption was introduced during this refresh/review; the continuation concentrated on Phase 5 specialist review, in-scope fixes, and local behavioral validation.

Codebase Impact Analysis

  • packages/shared: repository provider and VM-agent request contracts.
  • apps/api: BetterAuth scopes, D1 migration/schema, GitLab service/routes (timeout added), project/access/token boundaries, repo browser, TaskRunner/node metadata.
  • apps/web: provider onboarding, project/branch selector, API client, unit and visual tests (branches mock corrected).
  • packages/vm-agent: clone normalization, credential helper (path gate fail-open fixed), host/path checks, persistence/recovery, completion/MR behavior.
  • GitHub and Artifacts paths were explicitly retained and regression-tested.

Documentation & Specs

  • Branch-local task record updated at tasks/active/2026-07-08-gitlab-repo-workspace-wip.md with the Phase 5 continuation outcomes.
  • PR body is the durable validation/review record.
  • security.md, self-hosting.mdx, and .env.example updated for GitLab repository access and the new timeout env var. Further public connection/onboarding UX documentation is deferred to the focused discovery/UX layer.

Constitution & Risk Check

  • Principle XI: GitLab API calls are now bounded by GITLAB_API_TIMEOUT_MS; deployment host derives from platform GitLab configuration; repo-browser limits use configured values.
  • Credential isolation: GitLab tokens are callback-fetched, never persisted in workspace environment variables, and identity is re-verified before exchange. VM credential path gate no longer fails open for unregistered workspaces.
  • Additive migration safety: unique 0091 migration, foreign key cascade, project/user/host/project indexes.
  • Primary residual risk is no live GitLab provider test because staging is explicitly forbidden; the draft and human-review label preserve that risk visibly.

@codspeed-hq

codspeed-hq Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Merging this PR will not alter performance

✅ 6 untouched benchmarks


Comparing sam/gitlab-repo-workspace-wip (e8d43d0) with sam/gitlab-platform-config-wip (cd034b0)

Open in CodSpeed

raphaeltm added 4 commits July 8, 2026 14:36
…into sam/gitlab-repo-workspace-wip

# Conflicts:
#	apps/api/src/auth.ts
#	apps/api/src/routes/mcp/instruction-tools.ts
#	apps/web/tests/playwright/project-onboarding-wizard-audit.spec.ts
#	packages/vm-agent/internal/server/server.go
#	packages/vm-agent/internal/server/workspaces.go
@simple-agent-manager simple-agent-manager Bot added the needs-human-review Agent could not complete all review gates — human must approve before merge label Jul 11, 2026
raphaeltm and others added 10 commits July 11, 2026 18:32
Re-ran all 8 /do Phase 5 local reviewers (prior session could not launch
them). Applied in-scope fixes:

- Add direct GitLab POST /api/projects route behavioral test (9 scenarios:
  happy/400/401/403/404, non-default branch exist/absent, 409 duplicate,
  sidecar-insert rollback).
- Bound every GitLab API call with a configurable timeout
  (GITLAB_API_TIMEOUT_MS, default 30s) via shared fetchWithTimeout —
  Principle XI (no unbounded fetch on the core project-creation path).
- Fix VM credential path gate fail-open: isAllowedCredentialPathForWorkspace
  now falls back to process config RepoProvider/RepositoryPath for
  unregistered workspaces (mirrors the host gate; preserves standalone mode)
  instead of returning true. Adds a regression test.
- Add verifyGitLabProjectAccess error-branch tests (<Developer 403, 404
  not-found, group-access inheritance).
- Fix Playwright onboarding audit /api/gitlab/branches mock to return a bare
  array matching the real route contract.
- Doc sync: drop stale "future" from security.md GitLab row; document
  required read_user+api GitLab OAuth scopes in self-hosting.mdx and the new
  GITLAB_API_TIMEOUT_MS in .env.example.

Deferred to later stack layers (documented in task file / PR): MR-layer host
validation + dedicated external HTTP client, OAuth scope configurability,
GitLabProjectSelector ARIA/keyboard polish, per-request lookup dedup.

Refs idea 01KV7ZFD6HZS5N7J45VA798KN1, mission 2b4af813-b907-4033-9145-b2087078d08b.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…session_snapshots

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GitLab vends a broad, unexpiring user OAuth token (unlike GitHub's
repo-scoped installation tokens), so the credential exchange must not
fall back to empty-allow behavior:

- Request-side gate: gitlab-bound workspaces must supply both host and
  path query params or the vm-agent returns 204 without ever contacting
  the control plane.
- Response-side gate: a provider=gitlab response is withheld unless the
  caller supplied host+path AND the control plane resolved a non-empty
  repositoryPath, on top of the existing exact host/path match checks.

GitHub/Artifacts keep empty-allow (gh wrapper sends host with no path).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
GitLab refresh tokens are rotating and single-use — concurrent
BetterAuth getAccessToken calls replay a consumed refresh token and
revoke the whole token family upstream (same class as the codex
staging refresh-revocation incident; .claude/rules/44 + 45).

Adds GitLabUserAccessTokenLock (sibling of GitHubUserAccessTokenLock,
wrangler DO migration v17) and routes all GitLab token lookups DO-first
with a direct fallback only when the binding is absent. Includes a
rule-45 concurrency regression test (verified to fail with the mutex
bypassed) and service-level DO-first routing tests.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Bound tryCreateGitLabMergeRequest with GitExecTimeout context so a hung
  GitLab API cannot stall task completion indefinitely
- Redact token material from logged GitLab error bodies via
  redactTaskCallbackDiagnosticText
- Fail closed when a gitlab-bound workspace has an empty bound repository
  path (both runtime-map and config-fallback branches)
- Normalize requested host casing in the git-credential helper shell script
  (tr lower) and Go-side allowed-host comparison (hostnames are
  case-insensitive per RFC 4343)
- Remove dead gitrepo.HostMatches
- Tests: 409->existing-MR fallback with param/auth assertions, empty-path
  fail-closed variants, mixed-case host acceptance + wrong-host rejection
  exercised through the rendered shell helper with a curl shim

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
…n expiry

- 403 when the stored project GitLab repo binding userId does not match the
  workspace owner (fail-closed before any token mint or upstream verify)
- Thread the GitLab access-token expiry through to the git-token response
  (result-object token variants; string wrappers preserve existing exports)
- Warn-log when the GITLAB_USER_ACCESS_TOKEN_LOCK DO binding is absent and
  refresh serialization falls back to unserialized direct lookup
- Delete project_gitlab_repositories rows in the project-delete D1 batch
  (FK cascade is not enforced for ALTER TABLE-added tables)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Adds merge-blocking tests identified by the test-engineer review:
- git-token GitLab branch: GITLAB_REAUTH_REQUIRED propagates as 401
  without re-verifying upstream access; missing repository metadata
  fails closed with 403 before any token is minted
- getGitLabOAuthConfig returns null for every partial-config case
  (missing host / client id / client secret) and normalizes a host
  with a path down to its origin
- new platform-config-validation suite covering the GitLab host rules:
  https required, http localhost carve-out, credential/path/query/
  fragment rejection, invalid URL, and client id/secret length checks

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
HIGH-1 (drop 'api' OAuth scope): REJECTED. GitLab's REST API (merge
request creation, verifyGitLabProjectAccess project lookups) requires
the 'api' scope — read_repository/write_repository only cover
git-over-HTTP and read_api cannot create merge requests.

HIGH-3 (rate limit on token vending): DEFERRED with tracked follow-up.
The endpoint requires a workspace callback JWT plus the fail-closed
userId binding, and the per-user DO mutex already eliminates the
concurrent-refresh token-family-revocation vector. A naive per-call
limit would throttle legitimate git credential fetches; the correct
refresh-aware limit is scoped in
tasks/backlog/2026-07-12-gitlab-token-lock-rate-limit.md.

Co-Authored-By: Claude <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
3.9% Duplication on New Code (required ≤ 3%)

See analysis details on SonarQube Cloud

Fixes the SonarCloud new-duplication gate on PR #1547 (3.9% > 3%):
- GitHubUserAccessTokenLock and GitLabUserAccessTokenLock now extend a
  shared abstract UserAccessTokenLock (behavior and log keys unchanged;
  wrangler-bound class names preserved).
- The two tryCreateGitLabMergeRequest tests share a
  newGitLabMRTestServer helper instead of duplicating Server/runtime
  construction.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@sonarqubecloud

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

needs-human-review Agent could not complete all review gates — human must approve before merge

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant