You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
TME test config, mage record command, and re-recorded PreflightQuota cassettes (#7779)
* Support azd config defaults.test.* keys for functional test config
Update cliConfig.init() resolution order to check test-specific azd config
keys (defaults.test.subscription, defaults.test.tenant, defaults.test.location)
before falling back to general defaults (defaults.subscription, defaults.location).
This enables a one-time setup for developers:
azd config set defaults.test.subscription <TME-SUB-ID>
azd config set defaults.test.tenant <TME-TENANT-ID>
azd config set defaults.test.location eastus2
No more manual env var exports per session.
Fixes#7777
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* docs: add mage record and defaults.test.* config documentation
Update CONTRIBUTING.md, AGENTS.md, recording guide, and environment
variables docs to document the new mage record command and
defaults.test.* azd config keys for test subscription configuration.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* exclude stale recordings from preflight playback
VsServer, Deploy_SlotDeployment, and Up_Down_ContainerAppJob recordings are stale and require TME subscription access to re-record. Exclude them from automatic playback so preflight passes for contributors without TME access. Tracked in #7780 and #7014.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* revert: restore original PreflightQuota recordings
The cassettes I re-recorded in 519138c used my personal subscription ID
(25fd0362-...) instead of the TME subscription (4d042dc6-...) that CI
expects. This caused 'requested interaction not found' errors on all 3
build platforms because azd in CI generates URLs with the TME sub but the
cassette URLs use my personal sub.
Reverting these 6 cassettes back to the originals from fe57152 (which
were recorded against TME and worked in CI).
Note: re-recording these requires TME subscription access (see #7780).
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* address review feedback: simplify mage record, clarify maintainer workflow
- magefile.go: remove manual go build; delegate to azdcli.NewCLI's buildRecordOnce (addresses weikanglim feedback)
- AGENTS.md / CONTRIBUTING.md / recording-functional-tests-guide.md: clarify re-recording is a core-maintainer workflow; any subscription works
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address Copilot PR review feedback
- magefile.go: only apply excludedPlaybackTests in playback mode so `mage record -filter=<name>` can re-record stale tests (the whole point of excluding them was to enable re-recording)
- magefile.go: include opts.mode in 'no tests match filter' message
- cli_test.go: clarify config fallbacks only apply when CI is unset
- environment-variables.md: note AZD_TEST_TENANT_ID has no defaults.tenant global fallback; clarify CI-gating
- recording-functional-tests-guide.md: same doc clarifications
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* Re-record PreflightQuota cassettes against TME subscription
All 6 preflight quota tests re-recorded and verified in both
record and playback modes.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
* fix: address wbreza review feedback
- Extract configFallback helper in cli_test.go to remove DRY violation
across subscription/tenant/location config lookups
- Add opts.mode validation in runFunctionalTests to make the
record/playback contract explicit
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
---------
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Rajesh Kamal <rajeshkamal@microsoft.com>
Copy file name to clipboardExpand all lines: cli/azd/AGENTS.md
+4Lines changed: 4 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -75,6 +75,10 @@ When writing tests, prefer table-driven tests. Use testify/mock for mocking.
75
75
76
76
> **Tip**: The `/azd-preflight` Copilot skill runs all these checks and auto-fixes issues. See `.github/skills/azd-preflight/`.
77
77
78
+
Additional mage targets:
79
+
80
+
-`mage record` — re-record functional test cassettes against a live Azure subscription. Accepts an optional `-filter=TestName` flag to re-record specific tests. Typically only core maintainers need to run this; external contributors can rely on playback mode (the default) which requires no Azure access. Requires `azd auth login` and a configured test subscription (see `docs/recording-functional-tests-guide.md`).
Copy file name to clipboardExpand all lines: cli/azd/CONTRIBUTING.md
+21Lines changed: 21 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -59,6 +59,27 @@ mage preflight
59
59
60
60
> **Tip**: If you're using GitHub Copilot, the `/azd-preflight` skill runs `mage preflight` and auto-fixes any issues it discovers.
61
61
62
+
### Re-recording functional test cassettes
63
+
64
+
> **Note**: Re-recording is typically a core-maintainer workflow and requires an Azure subscription. External contributors can run the tests in the default playback mode, which replays stored cassettes and needs no Azure access.
65
+
66
+
Re-record stale functional test recordings against a live Azure subscription:
67
+
68
+
```bash
69
+
cd cli/azd
70
+
mage record # re-record all playback tests
71
+
mage record -filter=Test_CLI_Quota # re-record only matching tests
72
+
```
73
+
74
+
Core maintainers configure the test subscription and tenant once via `azd config`:
75
+
76
+
```bash
77
+
azd config set defaults.test.subscription <SUBSCRIPTION_ID>
78
+
azd config set defaults.test.tenant <TENANT_ID>
79
+
```
80
+
81
+
These values are stored in your user-level azd config and persist across sessions. You can also set them via environment variables (`AZD_TEST_AZURE_SUBSCRIPTION_ID`, `AZD_TEST_TENANT_ID`), which take precedence. Any Azure subscription you have access to works; the `defaults.test.*` namespace is kept separate from the top-level `defaults.*` so test config does not affect regular `azd` usage. See the [recording guide](./docs/recording-functional-tests-guide.md) for details.
> **Note**: Re-recording is typically a core-maintainer workflow. External contributors generally do not need to re-record cassettes; the default playback mode replays stored interactions and requires no Azure access. Open an issue if a cassette appears stale.
266
+
267
+
### Using `mage record` (Recommended)
268
+
269
+
The easiest way to re-record tests is the `mage record` target:
270
+
271
+
```bash
272
+
cd cli/azd
273
+
mage record -filter=Test_CLI_MyNewFeature # re-record a specific test
274
+
mage record # re-record all playback tests
275
+
```
276
+
277
+
`mage record` handles building the `azd-record` binary, setting `AZURE_RECORD_MODE=record`, and running the test with a 30-minute timeout.
278
+
279
+
#### Configuring the Test Subscription
280
+
281
+
Tests need an Azure subscription and tenant. Any subscription you have access to works — the `defaults.test.*` keys are a separate namespace so test configuration does not affect regular `azd` defaults. Configure them once with `azd config` (persists across sessions):
282
+
283
+
```bash
284
+
azd config set defaults.test.subscription <SUBSCRIPTION_ID>
285
+
azd config set defaults.test.tenant <TENANT_ID>
286
+
```
287
+
288
+
The resolution order is: environment variable → `defaults.test.*` config → `defaults.*` config.
289
+
Note: tenant only falls back to `defaults.test.tenant` (no `defaults.tenant` global fallback).
290
+
Config fallbacks are only consulted when the `CI` environment variable is unset.
0 commit comments