Skip to content

[ARM] az deployment: Fix bicep template size inflation with differential template handling#31990

Merged
zhoxing-ms merged 12 commits intoAzure:devfrom
vhvb1989:fix/bicep-template-size-inflation-clean
Sep 1, 2025
Merged

[ARM] az deployment: Fix bicep template size inflation with differential template handling#31990
zhoxing-ms merged 12 commits intoAzure:devfrom
vhvb1989:fix/bicep-template-size-inflation-clean

Conversation

@vhvb1989
Copy link
Copy Markdown
Member

Description

Fixes bicep template size inflation that causes false "template too large" errors for deployments near the 4MB limit by implementing differential template processing based on file type.

Problem

The Azure CLI was applying JsonCTemplatePolicy string escaping to all template types, causing significant size inflation for bicep templates. This led to deployment failures for templates under 4MB due to the inflated request payload size.

Root Cause Analysis

  1. Universal JsonCTemplatePolicy application: All template files underwent string escaping regardless of type
  2. Size inflation: Escaped JSON strings (with backslash-quote, backslash-n, backslash-backslash) became ~20% larger than original content
  3. False 4MB limit errors: Templates under Azure's actual limit failed due to processing overhead

Solution Approach

Implemented differential template handling based on file type:

Bicep Files (.bicep)

  • Use JSON objects directly (template_obj)
  • Skip JsonCTemplatePolicy to prevent size inflation
  • Maintain compact representation without string escaping

ARM Template Files (.json)

  • Continue using string content (template_content)
  • Apply JsonCTemplatePolicy for JSONC compatibility
  • Preserve existing behavior and validation

URI-based Deployments

  • Use template_link without local processing
  • No size inflation issues
  • Server-side template handling

Key Code Changes

_prepare_deployment_properties_unmodified(): Now uses differential handling where bicep files use JSON objects directly, ARM files use string content, and URI deployments use template links.

_deploy_arm_template_core_unmodified(): JsonCTemplatePolicy is now only applied to ARM template files, not bicep files, preventing unnecessary string escaping.

Impact

  • Size reduction: ~20% smaller request payloads for bicep templates
  • Fixes deployment failures: Templates under 4MB now deploy successfully
  • Maintains compatibility: ARM templates and URI deployments unchanged
  • Preserves functionality: All existing features and validation remain intact

Testing

  • Bicep template deployments: No size inflation, successful deployments
  • ARM template deployments: Existing behavior preserved
  • URI-based deployments: Unaffected by changes
  • All existing tests pass with new implementation

Related Issues

Fixes #31989

Type of Change

[✓] Bug fix (non-breaking change which fixes an issue)
[ ] New feature (non-breaking change which adds functionality)
[ ] Breaking change (fix or feature that would cause existing functionality to not work as expected)
[ ] This change requires a documentation update

- Bicep files now use JSON objects directly to prevent Azure SDK string escaping size inflation
- ARM template files continue using string content with JsonCTemplatePolicy for JSONC compatibility
- URI-based deployments use template_link without local processing
- Conditional JsonCTemplatePolicy application only for ARM template files
- Fixes false '4MB template too large' errors for bicep templates under the actual limit

This implements a bicep-specific solution rather than modifying JsonCTemplatePolicy globally,
ensuring ~20% size reduction for bicep templates while maintaining full ARM template compatibility.
- Extract _process_template_file helper function to handle bicep/ARM template processing
- Reduces branches in _prepare_deployment_properties_unmodified
- Maintains all existing functionality while improving code organization
- Extract _get_template_for_deployment helper function
- Fix trailing whitespace issue
- Reduces branch count in main function for pylint compliance
- Replace elif statements with separate if statements after return
- Improves code clarity and follows pylint best practices
- Remove trailing whitespace from blank lines
- Final style cleanup for CI compliance
Copilot AI review requested due to automatic review settings August 22, 2025 22:34
@azure-client-tools-bot-prd
Copy link
Copy Markdown

azure-client-tools-bot-prd bot commented Aug 22, 2025

️✔️AzureCLI-FullTest
️✔️acr
️✔️latest
️✔️3.12
️✔️3.13
️✔️acs
️✔️latest
️✔️3.12
️✔️3.13
️✔️advisor
️✔️latest
️✔️3.12
️✔️3.13
️✔️ams
️✔️latest
️✔️3.12
️✔️3.13
️✔️apim
️✔️latest
️✔️3.12
️✔️3.13
️✔️appconfig
️✔️latest
️✔️3.12
️✔️3.13
️✔️appservice
️✔️latest
️✔️3.12
️✔️3.13
️✔️aro
️✔️latest
️✔️3.12
️✔️3.13
️✔️backup
️✔️latest
️✔️3.12
️✔️3.13
️✔️batch
️✔️latest
️✔️3.12
️✔️3.13
️✔️batchai
️✔️latest
️✔️3.12
️✔️3.13
️✔️billing
️✔️latest
️✔️3.12
️✔️3.13
️✔️botservice
️✔️latest
️✔️3.12
️✔️3.13
️✔️cdn
️✔️latest
️✔️3.12
️✔️3.13
️✔️cloud
️✔️latest
️✔️3.12
️✔️3.13
️✔️cognitiveservices
️✔️latest
️✔️3.12
️✔️3.13
️✔️compute_recommender
️✔️latest
️✔️3.12
️✔️3.13
️✔️computefleet
️✔️latest
️✔️3.12
️✔️3.13
️✔️config
️✔️latest
️✔️3.12
️✔️3.13
️✔️configure
️✔️latest
️✔️3.12
️✔️3.13
️✔️consumption
️✔️latest
️✔️3.12
️✔️3.13
️✔️container
️✔️latest
️✔️3.12
️✔️3.13
️✔️containerapp
️✔️latest
️✔️3.12
️✔️3.13
️✔️core
️✔️latest
️✔️3.12
️✔️3.13
️✔️cosmosdb
️✔️latest
️✔️3.12
️✔️3.13
️✔️databoxedge
️✔️latest
️✔️3.12
️✔️3.13
️✔️dls
️✔️latest
️✔️3.12
️✔️3.13
️✔️dms
️✔️latest
️✔️3.12
️✔️3.13
️✔️eventgrid
️✔️latest
️✔️3.12
️✔️3.13
️✔️eventhubs
️✔️latest
️✔️3.12
️✔️3.13
️✔️feedback
️✔️latest
️✔️3.12
️✔️3.13
️✔️find
️✔️latest
️✔️3.12
️✔️3.13
️✔️hdinsight
️✔️latest
️✔️3.12
️✔️3.13
️✔️identity
️✔️latest
️✔️3.12
️✔️3.13
️✔️iot
️✔️latest
️✔️3.12
️✔️3.13
️✔️keyvault
️✔️latest
️✔️3.12
️✔️3.13
️✔️lab
️✔️latest
️✔️3.12
️✔️3.13
️✔️managedservices
️✔️latest
️✔️3.12
️✔️3.13
️✔️maps
️✔️latest
️✔️3.12
️✔️3.13
️✔️marketplaceordering
️✔️latest
️✔️3.12
️✔️3.13
️✔️monitor
️✔️latest
️✔️3.12
️✔️3.13
️✔️mysql
️✔️latest
️✔️3.12
️✔️3.13
️✔️netappfiles
️✔️latest
️✔️3.12
️✔️3.13
️✔️network
️✔️latest
️✔️3.12
️✔️3.13
️✔️policyinsights
️✔️latest
️✔️3.12
️✔️3.13
️✔️privatedns
️✔️latest
️✔️3.12
️✔️3.13
️✔️profile
️✔️latest
️✔️3.12
️✔️3.13
️✔️rdbms
️✔️latest
️✔️3.12
️✔️3.13
️✔️redis
️✔️latest
️✔️3.12
️✔️3.13
️✔️relay
️✔️latest
️✔️3.12
️✔️3.13
️✔️resource
️✔️latest
️✔️3.12
️✔️3.13
️✔️role
️✔️latest
️✔️3.12
️✔️3.13
️✔️search
️✔️latest
️✔️3.12
️✔️3.13
️✔️security
️✔️latest
️✔️3.12
️✔️3.13
️✔️servicebus
️✔️latest
️✔️3.12
️✔️3.13
️✔️serviceconnector
️✔️latest
️✔️3.12
️✔️3.13
️✔️servicefabric
️✔️latest
️✔️3.12
️✔️3.13
️✔️signalr
️✔️latest
️✔️3.12
️✔️3.13
️✔️sql
️✔️latest
️✔️3.12
️✔️3.13
️✔️sqlvm
️✔️latest
️✔️3.12
️✔️3.13
️✔️storage
️✔️latest
️✔️3.12
️✔️3.13
️✔️synapse
️✔️latest
️✔️3.12
️✔️3.13
️✔️telemetry
️✔️latest
️✔️3.12
️✔️3.13
️✔️util
️✔️latest
️✔️3.12
️✔️3.13
️✔️vm
️✔️latest
️✔️3.12
️✔️3.13

@azure-client-tools-bot-prd
Copy link
Copy Markdown

Hi @vhvb1989,
Since the current milestone time is less than 7 days, this pr will be reviewed in the next milestone.

@azure-client-tools-bot-prd
Copy link
Copy Markdown

azure-client-tools-bot-prd bot commented Aug 22, 2025

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@yonzhan
Copy link
Copy Markdown
Collaborator

yonzhan commented Aug 22, 2025

Thank you for your contribution! We will review the pull request and get back to you soon.

@github-actions
Copy link
Copy Markdown

The git hooks are available for azure-cli and azure-cli-extensions repos. They could help you run required checks before creating the PR.

Please sync the latest code with latest dev branch (for azure-cli) or main branch (for azure-cli-extensions).
After that please run the following commands to enable git hooks:

pip install azdev --upgrade
azdev setup -c <your azure-cli repo path> -r <your azure-cli-extensions repo path>

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR fixes a bicep template size inflation issue that was causing false "template too large" errors for deployments near the 4MB limit. The fix implements differential template processing based on file type to prevent unnecessary string escaping.

  • Implements differential handling where bicep files use JSON objects directly while ARM templates continue using string content
  • Removes JsonCTemplatePolicy application for bicep files to prevent ~20% size inflation from string escaping
  • Refactors template preparation logic into reusable helper functions for better maintainability

Tip: Customize your code reviews with copilot-instructions.md. Create the file or learn how to get started.

Comment thread src/azure-cli/azure/cli/command_modules/resource/custom.py
Comment thread src/azure-cli/azure/cli/command_modules/resource/custom.py Outdated
Comment thread src/azure-cli/azure/cli/command_modules/resource/custom.py Outdated
The condition 'and not (template_file and is_bicep_file(template_file))' is no longer needed because:
- JsonCTemplatePolicy only affects string content, not JSON objects
- Bicep files use JSON objects (template_obj) directly
- ARM templates use string content (template_content) where policy applies
- Therefore, JsonCTemplatePolicy can always be applied safely
Clarifies that _deploy_arm_template_core_unmodified is resource-group-specific,
so the hardcoded 'resourceGroup' scope is intentional and correct.
- Added TestTemplateSizeOptimization class with 8 focused unit tests
- Tests validate differential handling between bicep and ARM templates
- Bicep templates use template objects (size optimized)
- ARM templates use template content (backward compatible)
- URI deployments use template links (unchanged behavior)
- All tests pass and existing functionality preserved
- Validates size optimization benefits while maintaining compatibility
@yanzhudd
Copy link
Copy Markdown
Contributor

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 3 pipeline(s).

@zhoxing-ms zhoxing-ms changed the title [ARM/Bicep] Fix bicep template size inflation with differential template handling [ARM] az deployment: Fix bicep template size inflation with differential template handling Aug 27, 2025
@zhoxing-ms
Copy link
Copy Markdown
Contributor

@vhvb1989 Could you please recording some yaml files for the related tests in live mode? For how to record tests, please refer to this guideline https://github.com/Azure/azure-cli/blob/dev/doc/authoring_tests.md#recording-tests

@zhoxing-ms
Copy link
Copy Markdown
Contributor

@vhvb1989 Please resolve these CI issues?

@vhvb1989
Copy link
Copy Markdown
Member Author

/azp run

@azure-pipelines
Copy link
Copy Markdown

Commenter does not have sufficient privileges for PR 31990 in repo Azure/azure-cli

@yonzhan
Copy link
Copy Markdown
Collaborator

yonzhan commented Aug 29, 2025

/azp run

@azure-pipelines
Copy link
Copy Markdown

Azure Pipelines successfully started running 3 pipeline(s).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ARM/Bicep] az deployment: Fix bicep template size inflation causing 4MB limit errors

5 participants