Remove SAM translator dependency, validate SAM templates via schemas#4491
Open
kddejong wants to merge 16 commits into
Open
Remove SAM translator dependency, validate SAM templates via schemas#4491kddejong wants to merge 16 commits into
kddejong wants to merge 16 commits into
Conversation
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #4491 +/- ##
==========================================
- Coverage 93.57% 93.55% -0.03%
==========================================
Files 466 468 +2
Lines 15230 15250 +20
Branches 2945 2953 +8
==========================================
+ Hits 14252 14267 +15
- Misses 599 601 +2
- Partials 379 382 +3
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
2e7370a to
56e6be1
Compare
Replace the samtranslator-based transform with direct SAM schema validation:
- Add SAM resource schemas (11 types) decomposed from the official SAM JSON Schema
- Add Globals section schema and minimal Globals merger
- Register SAM types in all 46 provider modules
- Inject synthetic implicit resources (ServerlessRestApi, ServerlessHttpApi, {Id}Role)
- Skip circular dependency checks for SAM resources (SAM splits them during transform)
- Fix Ref class to handle schemas without primaryIdentifier
- Add Globals as valid template-level property
Remove samtranslator dependency:
- Delete _sam.py transform wrapper
- Delete ManagedPolicies.json and update script
- Remove aws-sam-translator from requirements/base.txt
- Remove samtranslator mypy override
Known gaps documented in docs/sam-schema-gaps.md:
- Generated resources with suffixed logical IDs (partially mitigated)
- Runtime deprecation rules don't fire on SAM types yet
- SAM event source validation not available without transform
- 3 SAM types without confirmed CFN mappings
Keyword expansion: - Runtime deprecation rules (E2531, E2533, W2531) now fire on AWS::Serverless::Function and Globals/Function/Runtime - Lambda extension schema rules (E3673-E3677) now fire on AWS::Serverless::Function properties New rule E3715 (GlobalsTransform): - Validates Globals section requires the SAM transform - Validates Globals structure against schema (section names, property names per section) - Stripped $ref from globals schema for standalone validation
Schema generation:
- update_sam_schemas.py now generates PassThroughProp patches from
CFN schema property mappings (174 ops across 6 SAM types) and
applies them before writing schemas so they are baked into the
hash files at rest
- Inline $ref pointers from CFN definitions (depth 3) so patches
are self-contained (FilterCriteria, DestinationConfig, etc.)
- Strip description fields to match CFN schema conventions
- Auto-clean stale SAM schema files on regeneration
Manual patches (not auto-generated):
- Add State property to EventBridgeRuleEventProperties for
Function and StateMachine (missing from SAM schema source)
Resource configuration:
- Allow Connectors as a resource-level attribute (SAM embedded
connector syntax)
Context:
- Inject synthetic {Id}.Version and {Id}.Alias resources for
Functions with AutoPublishAlias/DeploymentPreference
Maintenance:
- Add update_sam_schemas.py to maintenance workflow
- Remove samtranslator version pinning and
update_serverless_aws_policies.py
- Remove backup.json
Fix E3720 rule ID conflict with BlockDeviceMappingVirtualName
…d Serverless dir - Add IgnoreGlobals to resource configuration schema (accepts '*' string or list of property names) - Add timeout=30 to urlopen in update_sam_schemas.py - Remove orphaned src/cfnlint/data/Serverless/ directory
New rule E3064 (ServerlessTransformAttributes) validates that Connectors and IgnoreGlobals resource attributes are only used when the AWS::Serverless-2016-10-31 transform is declared.
- Inline $ref pointers when copying readOnlyProperties from CFN schemas (fixes dangling SnapStartResponse $ref) - Skip SAM schemas from CFN provider schema validation (SAM schemas don't have primaryIdentifier or CFN definition naming) - Build keywords from SAM schemas for rule keyword validation - Add Globals, Connectors, IgnoreGlobals to found keywords list
Upstream main added E3720 for DbInstanceKmsKeyStorageEncrypted in aws-cloudformation#4480. Renaming GlobalsTransform to the next available ID.
- test_sam_globals.py: 16 tests for merge_globals (100% coverage) - test_globals_transform.py: 4 tests for E3722 rule (100% coverage) - test_serverless_transform_attributes.py: 4 tests for E3065 rule (100% coverage) - test_inject_sam_implicit.py: 15 tests for implicit resource injection (95% coverage)
…obals integration test
…and readOnlyProperties Remove blanket permissive GetAtt for all AWS::Serverless:: types. SAM types with CFN mappings now use the mapped type's readOnlyProperties for GetAtt validation and primaryIdentifier for Ref validation. Connector correctly errors on GetAtt since it has no primary resource.
… permissions, Api Domain/UsagePlan, implicit API stages
fdb75ab to
aefb4dc
Compare
aefb4dc to
054499d
Compare
…cript - Fix SAM_TO_CFN_TYPE mapping: AWS::Lambda::CapacityProvider (nonexistent) → AWS::ECS::CapacityProvider - Remove bogus schema file and duplicate provider module entries - Replace exec() with ast.literal_eval for parsing provider modules - Fix _inline_refs to recurse into lists (e.g. anyOf/oneOf arrays) - Add warning logs to _resolve_cfn_property on failure - Escalate readOnlyProperties lookup failure to error level Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Replace the
aws-sam-translatorruntime dependency with direct JSON Schema validation of SAM resource types. SAM templates are now validated against schemas derived from the SAM project's own schema definitions, using the same provider schema infrastructure as CloudFormation resources.Changes
scripts/update_sam_schemas.py): Downloads the SAM JSON Schema, decomposes it into per-resource-type schemas, generates PassThrough property patches from CFN schemas, and updates provider modules with SAM type entries._sam_globals.py): MergesGlobalssection properties into SAM resources at validation time, respectingIgnoreGlobals.AWS::Serverless::ConnectorandIgnoreGlobalsusage requires the SAM transform declaration.AWS::Serverless::Functionpaths.module_names). GetAtt on the SAM resource itself is validated againstreadOnlyPropertiesfrom the mapped CFN type.primaryIdentifierandreadOnlyPropertiesfrom their mapped type (e.g., Function → Lambda::Function, GraphQLApi → AppSync::GraphQLApi).aws-sam-translatordependency,_sam.pytransform module,ManagedPolicies.json.Behavioral Notes
!Ref MyFunctionRoleand!GetAtt MyFunctionRole.Arnpass via the wildcard mechanism. Attribute validation is skipped for sub-resources (same as Modules).Testing
Note
Commit message for 9d6ef6b incorrectly says E3064 — the actual rule ID is E3065 (E3064 is taken by VpcEndpointPrivateDnsDuplicate).