Skip to content

[ARM] az deployment: Expose --validation-level parameter at all scopes for create, validate, and what-if subcommands#31747

Merged
zhoxing-ms merged 10 commits intoAzure:devfrom
jeskew:add-validation-level
Jul 29, 2025
Merged

[ARM] az deployment: Expose --validation-level parameter at all scopes for create, validate, and what-if subcommands#31747
zhoxing-ms merged 10 commits intoAzure:devfrom
jeskew:add-validation-level

Conversation

@jeskew
Copy link
Copy Markdown
Contributor

@jeskew jeskew commented Jul 1, 2025

Related command
az deployment

Description

This PR adds a new parameter validation_level to the az deployment commands. This parameter is present on the SDK models used by the CLI but needs to be manually passed through in the az deployment commands.

Testing Guide

The parameter can be used on the create, validate, and what-if subcommands at all scopes:

> az deployment group validate --template-file "..." --validation-level template

> az deployment sub what-if --location eastus --template-file "..." --validation-level providerNoRbac

History Notes

[ARM] az deployment: Expose --validation-level parameter at all scopes for create, validate, and what-if subcommands


This checklist is used to make sure that common guidelines for a pull request are followed.

Copilot AI review requested due to automatic review settings July 1, 2025 12:34
@azure-client-tools-bot-prd
Copy link
Copy Markdown

azure-client-tools-bot-prd bot commented Jul 1, 2025

️✔️AzureCLI-FullTest
️✔️acr
️✔️latest
️✔️3.12
️✔️3.9
️✔️acs
️✔️latest
️✔️3.12
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.12
️✔️3.9
️✔️ams
️✔️latest
️✔️3.12
️✔️3.9
️✔️apim
️✔️latest
️✔️3.12
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.12
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️aro
️✔️latest
️✔️3.12
️✔️3.9
️✔️backup
️✔️latest
️✔️3.12
️✔️3.9
️✔️batch
️✔️latest
️✔️3.12
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.12
️✔️3.9
️✔️billing
️✔️latest
️✔️3.12
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.12
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.12
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️compute_recommender
️✔️latest
️✔️3.12
️✔️3.9
️✔️computefleet
️✔️latest
️✔️3.12
️✔️3.9
️✔️config
️✔️latest
️✔️3.12
️✔️3.9
️✔️configure
️✔️latest
️✔️3.12
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.12
️✔️3.9
️✔️container
️✔️latest
️✔️3.12
️✔️3.9
️✔️containerapp
️✔️latest
️✔️3.12
️✔️3.9
️✔️core
️✔️latest
️✔️3.12
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.12
️✔️3.9
️✔️databoxedge
️✔️latest
️✔️3.12
️✔️3.9
️✔️dls
️✔️latest
️✔️3.12
️✔️3.9
️✔️dms
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.12
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.12
️✔️3.9
️✔️find
️✔️latest
️✔️3.12
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.12
️✔️3.9
️✔️identity
️✔️latest
️✔️3.12
️✔️3.9
️✔️iot
️✔️latest
️✔️3.12
️✔️3.9
️✔️keyvault
️✔️latest
️✔️3.12
️✔️3.9
️✔️lab
️✔️latest
️✔️3.12
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️maps
️✔️latest
️✔️3.12
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.12
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.12
️✔️3.9
️✔️mysql
️✔️latest
️✔️3.12
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.12
️✔️3.9
️✔️network
️✔️latest
️✔️3.12
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.12
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.12
️✔️3.9
️✔️profile
️✔️latest
️✔️3.12
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.12
️✔️3.9
️✔️redis
️✔️latest
️✔️3.12
️✔️3.9
️✔️relay
️✔️latest
️✔️3.12
️✔️3.9
️✔️resource
️✔️latest
️✔️3.12
️✔️3.9
️✔️role
️✔️latest
️✔️3.12
️✔️3.9
️✔️search
️✔️latest
️✔️3.12
️✔️3.9
️✔️security
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.12
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.12
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.12
️✔️3.9
️✔️sql
️✔️latest
️✔️3.12
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.12
️✔️3.9
️✔️storage
️✔️latest
️✔️3.12
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.12
️✔️3.9
️✔️telemetry
️✔️latest
️✔️3.12
️✔️3.9
️✔️util
️✔️latest
️✔️3.12
️✔️3.9
️✔️vm
️✔️latest
️✔️3.12
️✔️3.9

@azure-client-tools-bot-prd
Copy link
Copy Markdown

azure-client-tools-bot-prd bot commented Jul 1, 2025

⚠️AzureCLI-BreakingChangeTest
⚠️resource
rule cmd_name rule_message suggest_message
⚠️ 1006 - ParaAdd deployment create cmd deployment create added parameter validation_level
⚠️ 1006 - ParaAdd deployment group create cmd deployment group create added parameter validation_level
⚠️ 1006 - ParaAdd deployment group validate cmd deployment group validate added parameter validation_level
⚠️ 1006 - ParaAdd deployment group what-if cmd deployment group what-if added parameter validation_level
⚠️ 1006 - ParaAdd deployment mg create cmd deployment mg create added parameter validation_level
⚠️ 1006 - ParaAdd deployment mg validate cmd deployment mg validate added parameter validation_level
⚠️ 1006 - ParaAdd deployment mg what-if cmd deployment mg what-if added parameter validation_level
⚠️ 1006 - ParaAdd deployment sub create cmd deployment sub create added parameter validation_level
⚠️ 1006 - ParaAdd deployment sub validate cmd deployment sub validate added parameter validation_level
⚠️ 1006 - ParaAdd deployment sub what-if cmd deployment sub what-if added parameter validation_level
⚠️ 1006 - ParaAdd deployment tenant create cmd deployment tenant create added parameter validation_level
⚠️ 1006 - ParaAdd deployment tenant validate cmd deployment tenant validate added parameter validation_level
⚠️ 1006 - ParaAdd deployment tenant what-if cmd deployment tenant what-if added parameter validation_level
⚠️ 1006 - ParaAdd deployment validate cmd deployment validate added parameter validation_level

@yonzhan
Copy link
Copy Markdown
Collaborator

yonzhan commented Jul 1, 2025

Thank you for your contribution! We will review the pull request and get back to you soon.

@github-actions
Copy link
Copy Markdown

github-actions bot commented Jul 1, 2025

The git hooks are available for azure-cli and azure-cli-extensions repos. They could help you run required checks before creating the PR.

Please sync the latest code with latest dev branch (for azure-cli) or main branch (for azure-cli-extensions).
After that please run the following commands to enable git hooks:

pip install azdev --upgrade
azdev setup -c <your azure-cli repo path> -r <your azure-cli-extensions repo path>

@microsoft-github-policy-service microsoft-github-policy-service bot added the Auto-Assign Auto assign by bot label Jul 1, 2025
@microsoft-github-policy-service microsoft-github-policy-service bot added the ARM az resource/group/lock/tag/deployment/policy/managementapp/account management-group label Jul 1, 2025

This comment was marked as outdated.

@jeskew
Copy link
Copy Markdown
Contributor Author

jeskew commented Jul 1, 2025

@zhoxing-ms I would appreciate some guidance on the linter failure. I've added help text to the CLIArgumentType constructor for the parameter's type, and the same help text is provided to every added invocation of argument_context::argument. Where am I missing?

Comment thread src/azure-cli/azure/cli/command_modules/resource/_params.py Outdated
@zhoxing-ms
Copy link
Copy Markdown
Contributor

image

Please check the az deployment create and az deployment validate commands

@jeskew jeskew requested review from Copilot and zhoxing-ms July 3, 2025 18:52
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR exposes the new validation_level parameter on az deployment commands across all scopes and adds end-to-end tests to validate its behavior.

  • Adds a DeploymentTestsWithValidationLevel test class covering resource-group, subscription, management-group, and tenant scopes.
  • Updates recordings for tenant-scope deployment tests to include validationLevel in request/response bodies.

Reviewed Changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated no comments.

File Description
src/azure-cli/azure/cli/command_modules/resource/tests/latest/test_resource.py Added DeploymentTestsWithValidationLevel for all scopes
src/azure-cli/azure/cli/command_modules/resource/tests/latest/recordings/test_tenant_deployment_with_validation_level.yaml New tenant-scope recording including validationLevel payloads
Comments suppressed due to low confidence (1)

src/azure-cli/azure/cli/command_modules/resource/tests/latest/test_resource.py:1278

  • Consider adding a negative test case for an invalid --validation-level value to ensure that the CLI properly rejects unsupported inputs.
class DeploymentTestsWithValidationLevel(ScenarioTest):

@jeskew
Copy link
Copy Markdown
Contributor Author

jeskew commented Jul 21, 2025

@zhoxing-ms is this PR still on track to be in the next release?

Comment thread src/azure-cli/azure/cli/command_modules/resource/_params.py Outdated
Co-authored-by: Xing Zhou <Zhou.Xing@microsoft.com>
@zhoxing-ms zhoxing-ms changed the title [ARM] Add validation_level parameter for az deployment commands [ARM] az deployment: Expose --validation-level parameter at all scopes for create, validate, and what-if subcommands Jul 28, 2025
@zhoxing-ms
Copy link
Copy Markdown
Contributor

/azp run

zhoxing-ms
zhoxing-ms previously approved these changes Jul 28, 2025
@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 3 pipeline(s).

@zhoxing-ms
Copy link
Copy Markdown
Contributor

Could you please resolve these CI issues?

@zhoxing-ms
Copy link
Copy Markdown
Contributor

zhoxing-ms commented Jul 28, 2025

@jeskew Please note that Azure CLI will freeze the code on 07/29/2025 07:00 UTC for the upcoming release. If you want to catch this release train, please resolve these comments ASAP, otherwise this PR has to be postponed to next sprint.

@jeskew
Copy link
Copy Markdown
Contributor Author

jeskew commented Jul 28, 2025

@zhoxing-ms The test that is failing in CI is failing on the dev branch as well. It does not look like it is related to this PR.

image

I have tried re-recording it, but it appears to be failing due to a CRLF/LF mismatch. Re-recording on Windows does not seem to address the issue:

I was only able to get this test to pass by changing the line endings in the JSON scenario file with dos2unix and then re-recording, but I am not sure if this will have the desired effect in CI since git autocrlf is enabled by default. Please rerun the tests. I'm very keen to get this change in the next CLI release and would hate to get delayed by a month due to an unrelated test failure on the dev branch.

@jeskew
Copy link
Copy Markdown
Contributor Author

jeskew commented Jul 28, 2025

/azp run

@azure-pipelines
Copy link
Copy Markdown

Commenter does not have sufficient privileges for PR 31747 in repo Azure/azure-cli

@jeskew
Copy link
Copy Markdown
Contributor Author

jeskew commented Jul 28, 2025

@yanzhudd @yonzhan @jsntcy @zhoxing-ms Could you rerun the tests?

@yonzhan
Copy link
Copy Markdown
Collaborator

yonzhan commented Jul 28, 2025

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 3 pipeline(s).

@yonzhan
Copy link
Copy Markdown
Collaborator

yonzhan commented Jul 28, 2025

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 3 pipeline(s).

@zhoxing-ms
Copy link
Copy Markdown
Contributor

E vcr.errors.CannotOverwriteExistingCassetteException: Can't overwrite existing cassette ('/mnt/vss/_work/1/s/src/azure-cli/azure/cli/command_modules/resource/tests/latest/recordings/test_create_template_specs.yaml') in your current record mode ('once').
E No match for the request (<Request (PUT) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/cli_test_template_specs000001/providers/Microsoft.Resources/templateSpecs/cli-test-create-template-spec000002?api-version=2022-02-01>) was found.
E Found 1 similar requests with 1 different matcher(s) :
E
E 1 - (<Request (PUT) https://management.azure.com/subscriptions/00000000-0000-0000-0000-000000000000/resourceGroups/cli_test_template_specs000001/providers/Microsoft.Resources/templateSpecs/cli-test-create-template-spec000002?api-version=2021-05-01>)..)
E Matchers succeeded : ['method', 'scheme', 'host', 'port', 'path']
E Matchers failed :
E _custom_request_query_matcher - assertion failure :
E None

Actually, I need to clarify that this is not a issue from dev branch code, but rather because the dev branch recently upgraded the SDK version with new api-version for template_stpecs
PR link: #31684
image

@jeskew Therefore, you need to first pull the code of the latest dev branch, and then reinstall the corresponding Python SDK azure-mgmt-resource-templatespecs==1.0.0b1 using azdev setup. Then, when recording YAML files locally, you will use the latest api-version after bumping the SDK to solve the CI issue

@zhoxing-ms
Copy link
Copy Markdown
Contributor

zhoxing-ms commented Jul 29, 2025

Please note that Azure CLI will freeze the code on 07/29/2025 07:00 UTC for the upcoming release. If you want to catch this release train, please resolve these comments ASAP, otherwise this PR has to be postponed to next sprint.

@jeskew So this is essentially a recording YAML issue caused by not installing the latest version of Python SDK locally, and currently there may be 3 and a half hours left to solve this CI issue for you, otherwise we won't be able to catch up with the release train of this sprint and will have to postpone it to the next sprint

@jeskew jeskew force-pushed the add-validation-level branch from f4c597e to 8fd6723 Compare July 29, 2025 04:48
@jeskew
Copy link
Copy Markdown
Contributor Author

jeskew commented Jul 29, 2025

@zhoxing-ms OK, I removed my commit that rerecorded the test since the recording on dev should be correct and this PR does not altar the code used by that test. Please rerun the tests.

@yonzhan
Copy link
Copy Markdown
Collaborator

yonzhan commented Jul 29, 2025

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 3 pipeline(s).

@yonzhan
Copy link
Copy Markdown
Collaborator

yonzhan commented Jul 29, 2025

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 3 pipeline(s).

@zhoxing-ms zhoxing-ms merged commit 8c13278 into Azure:dev Jul 29, 2025
48 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ARM az resource/group/lock/tag/deployment/policy/managementapp/account management-group Auto-Assign Auto assign by bot

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants