Skip to content

ELM migrations: pipelines rewiring + cutover review/approve + TSG doc#1501

Open
bhuvanshahMSFT wants to merge 58 commits into
Azure:masterfrom
bhuvanshahMSFT:elm-integration-1p
Open

ELM migrations: pipelines rewiring + cutover review/approve + TSG doc#1501
bhuvanshahMSFT wants to merge 58 commits into
Azure:masterfrom
bhuvanshahMSFT:elm-integration-1p

Conversation

@bhuvanshahMSFT

@bhuvanshahMSFT bhuvanshahMSFT commented Jun 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds pipeline rewiring CLI, refines the cutover review/approve flow, and ships an end-to-end ELM migrations TSG.

What changed

  • pipelines command group: list, submit, update, retry, delete (preview).
  • create: --enable-boards-github-connection, --enable-auto-discover-pipelines, --pipeline-service-connection-id.
  • list: --include-all (replaces deprecated --include-inactive).
  • Table formatters for pipeline entries and cutover review.
  • New doc doc/elm_migrations_tsg.md: end-to-end guide + troubleshooting (install latest CLI + extension, full command/param reference, stages/statuses, pitfalls).

Validation

  • pylint 10.00/10; flake8 clean.
  • Migration unit tests pass.
  • Wheel builds.

bhuvanshah7 and others added 30 commits April 22, 2026 15:00
- Add --github-token and ELM_GITHUB_TOKEN fallback

- Run GitHub device flow when token is not provided

- Keep target-owner-user-id optional for FF-off compatibility

- Add deviceFlowConfig endpoint fallback support

- Expand migration auth edge-case tests

- Update ELM docs for token/device-flow create behavior
- Translate generic 409 TF400898 during migration create into a clear active-migration message

- Keep non-conflict errors unchanged

- Add regression tests for conflict mapping and pass-through behavior
- Add troubleshooting entry in migrations.md

- Add pitfall row and dedicated 409 section in elm_migrations_tsg.md
- Show PAT guidance when both device-flow config endpoints return 404

- Validate interval/expires_in as positive integers and fail with explicit invalid response errors

- Add regression tests for new fallback and validation behavior
- Map 401/403 to generic app/service-unavailable message

- Preserve PAT fallback guidance in message

- Add unit test coverage for HTTP 401 handling
- Enforce target repository format as https://host/org/repo client-side

- Prefer PreCheckIssueType/validation issue messages from response body for CLI errors

- Keep non-TF400898 409 handling unchanged

- Add regression tests for new validation and error-detail extraction
Add is_preview for migrations command groups and align help/docs with preview and limited-availability messaging.
When a service connection is supplied via --service-endpoint-id, GitHub auth is handled server-side by the service endpoint. The CLI must not call /_apis/migrations/deviceFlowConfig, which fails (400) against GHES when the ELM GitHub App is not reachable from the caller's network.

- Gate _resolve_github_user_token on service_endpoint_id

- Reject --service-endpoint-id and --github-token together (ambiguous)

- Update help text to document precedence

- Add tests covering: SE-only, SE+token mutex, SE ignores env token, SE+whitespace token, SE+409 conflict message, SE+all optional fields
…led cutover date

The ELM service silently ignores 'null' for scheduledCutoverDate and only treats DateTimeOffset.MinValue ('0001-01-01T00:00:00+00:00') as the clear sentinel. Previously, 'az devops migrations cutover cancel' returned 200 OK but the server left the field set, leading users to believe the cancel had taken effect when it had not.

Verified empirically via 'az devops invoke' against a live stuck migration: sending the MinValue sentinel cleared scheduledCutoverDate to null in the response, while sending null was a silent no-op.

Updated unit test to assert the sentinel value is sent.
Demo User added 22 commits May 19, 2026 14:23
Adopt v1.0.5 changes: _MIGRATION_STATUS_VALUES enum (active=0, completed=1, failed=2, paused=3), 'paused' replaces 'suspended', new skip-validation policies (SourceRepositoryContainsLfsObjects=512, SourceRepositoryNotReadOnly=1024, BoardsGitHubConnectionProvisioning=2048), MaxRepoSize policy removed. Preserves local cutover review/approve flow, service-endpoint-id support, cutover-cancel guard, scheduled-date sentinel.
- Treat 404 as plain 'not found' (drop feature_not_enabled_message plumbing).
- Repository-mapping DTO now uses camelCase (sourceRepositoryId / targetRepository).
- Remove obsolete 404-feature-disabled test; simplify the remaining 404 test.
…eline-service-connection-id opt-in flags

Adds two server-contract opt-in flags to `migrations create` for pipeline rewiring:

- `--enable-auto-discover-pipelines` sets configOptions.enableAutoDiscoverPipelines=true so the server walks the source repo and creates clone definitions for every pipeline referencing it at cutover.
- `--pipeline-service-connection-id` pre-attaches the project-scoped GitHub service connection used by all rewired pipelines. Required for full auto-discovery; optional in manual mode (subsequent `pipelines submit` calls only need `--pipeline-ids`).
Per the design doc (Tfs/.../az-cli-pipelines-design.md), pipelines submit
in manual mode should accept --pipeline-ids alone when a service connection
was already attached via:
  - migrations create --pipeline-service-connection-id, or
  - pipelines update --service-connection-id

Previously the CLI hard-rejected submit without --service-connection-id,
contradicting the documented workflow. The check is removed; the server is
now the source of truth and returns a meaningful 400 when no SC is in
context. --service-connection-id remains accepted and is included in the
payload only when supplied. Help text updated to reflect optional usage.
Service endpoint (sync credential) and gitHubUserToken (user-identity verification) are independent. Drop the either/or gate and stop forcing github_token=None when SE is provided. The CLI now forwards an explicit --github-token or ELM_GITHUB_TOKEN env var alongside --service-endpoint-id, but does NOT trigger device flow on the SE path so non-interactive flows aren't broken. Required for FF Git.EnterpriseLiveMigration.RequireGitHubUserToken.
- create_migration: --pipeline-service-connection-id is dropped with a
  warning (server discards it; SC is a submit-time concept).
- _send_request: translate 400 'EnsureBranchExists' -> 'run submit first'.
- delete_pipeline_rewiring: translate 409 -> 'not in terminal stage'.
- Update test to assert SC is dropped + warning emitted.
…eConnectionId

Probe showed server bug (field accepted but not pre-attached for later
submit), but design contract requires the field. Keep CLI honoring the
design so customers benefit automatically when the server side is fixed.
The EnsureBranchExists and 409-delete translations from 207bf95 are kept.
…able

- pipelines acknowledge now requires confirmation (use -y to skip) since the server has no revoke API; recovery requires migration abandon+recreate.

- _transform_pipeline_entry_row falls back to yamlFilename when server returns name=null (pending server-side hydration UX fix).

- --acknowledge-ids / --pipeline-ids (acknowledge) help text documents irrevocability.
approve_cutover now accepts --pipelines-verified and posts pipelinesVerified:true; accept-failures is optional with a clear error when neither flag is given. list_migrations wires --include-all (includeInactiveMigrations) with --include-inactive kept as deprecated alias. Removes the dead 'pipelines acknowledge' command in favor of the cutover-level verification flow. Adds/updates unit tests (110 passing).
…ew create flags, include-all; fix stages/statuses
Demo User added 4 commits June 9, 2026 15:46
Complete _ACTIVE_STAGES so the stage-only fallback in _is_migration_active recognizes every in-flight stage, not just a subset. readyForCutover and reviewForCutover are cutover-gate stages where the migration is still active; terminal detection remains status-based and is unaffected. Adds a unit test covering all in-progress stages.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants