test: add CFN synthesis tests covering 24 plugin scenarios by sid88in · Pull Request #704 · sid88in/serverless-appsync-plugin

sid88in · 2026-05-24T20:53:45Z

test: add CFN synthesis tests covering 24 plugin scenarios

What

Adds a second-tier test suite that exercises the plugin against real
serverless.yml configurations and asserts on the generated
CloudFormation, complementing the existing 287 unit tests. Includes 24
runnable example projects under examples/ that double as user-facing
documentation.

Why

Unit tests catch logic and validation bugs but can't tell you whether
the plugin actually produces working CloudFormation across the matrix
of features users combine in practice. This adds that layer without
requiring AWS credentials or real deployments — sls package does the
synthesis locally and we assert on the template.

Side benefit: the project had no examples/ folder. Users had to read
the test fixtures or piece together docs to learn how to configure a
feature. These examples are copy-paste-ready and guaranteed to stay
current with the plugin's actual behavior — if they break, CI fails.

Coverage

24 fixtures, 68 assertions:

Auth (6): basic-api-key, cognito-userpools, iam-auth, oidc-auth,
lambda-authorizer, multi-auth

Resolvers (3): lambda-resolvers-js (JS code with @aws-appsync/utils),
lambda-resolvers-vtl (VTL templates), pipeline-resolvers

Data sources (6): datasource-http (with both unauthenticated and
IAM-signed variants), datasource-none, datasource-eventbridge,
datasource-opensearch, datasource-rds, plus DynamoDB and Lambda
covered transitively through other fixtures

Advanced features (9): caching (API + per-resolver),
waf (rules + association), logging-xray, custom-domain
(CloudFormation-managed), introspection-disabled
(with queryDepthLimit and resolverCountLimit), substitutions,
environment-variables, api-keys-multiple, tags,
visibility-private, schema-multiple-files

Infrastructure

e2e/helpers/synthesize.ts — runs sls package via a child process,
returns the parsed CFN template, cleans up temp dirs
e2e/helpers/assertions.ts — readable assertion helpers
(expectAuthenticationType, expectDataSourceOfType,
expectResourceWithProperties, etc.)
jest.e2e.config.ts + jest.e2e.setup.ts — separate Jest config
(so unit-test feedback stays fast); globalSetup creates the
examples/node_modules/serverless-appsync-plugin symlink so example
projects resolve the plugin from the current source tree, without
needing a per-example npm install
examples/README.md indexes all 24 examples for users browsing the repo
e2e/README.md explains the testing strategy for contributors

CI

New e2e job in .github/workflows/ci.yml runs after the unit-test
matrix on every PR and push to master or alpha. Single Node version
(22) since CFN output doesn't vary by Node version — runs in ~70 seconds.

Real plugin behaviors caught while building the fixtures

These are the kind of "silent gotcha" issues the suite is meant to
surface — found and used to refine the examples themselves:

API_KEY auth silently produces zero API keys if apiKeys: array
isn't specified
HTTP data sources with AWS_IAM authorization require explicit
iamRoleStatements
OpenSearch data sources can't auto-derive the ARN from Fn::GetAtt
— they need an explicit domain: field naming the CloudFormation
resource

How to run locally

# Unit tests (unchanged)
npm run test

# CFN synthesis tests
npm run test:e2e

# Everything
npm run test:all

# Single fixture
npx jest --config jest.e2e.config.ts basic-api-key

Verified

✅ npm run test — 287 unit tests pass
✅ npm run test:e2e — 68 synthesis assertions across 25 suites pass
✅ npm run lint — clean (0 errors, 0 warnings)
✅ npm run build — clean
✅ Full suite runs in ~80 seconds locally

What's not in this PR (intentionally)

No real-AWS deploy tests (Tier 2) — separate follow-up work that
needs a dedicated AWS test account and OIDC setup
No merged-api example — the plugin doesn't support MERGED API
type yet; that's a roadmap item, not a missed test

Follow-ups for after merge

Real-AWS Tier 2 tests for end-to-end deployment verification on a
scheduled cadence
Continue expanding fixtures as new features land (e.g. AppSync Events
API once supported, MERGED APIs once supported)
Consider linking from the main README to the examples/ folder

Adds a second-tier test suite that exercises the plugin against real serverless.yml configurations and asserts on the generated CloudFormation template, complementing the existing 287 unit tests. Coverage (24 fixtures, 68 assertions): Auth (6): basic-api-key, cognito-userpools, iam-auth, oidc-auth, lambda-authorizer, multi-auth Resolvers (3): lambda-resolvers-js (JS code + esbuild bundling), lambda-resolvers-vtl (VTL templates), pipeline-resolvers Data sources (6): datasource-http (incl. IAM-signed variant), datasource-none, datasource-eventbridge, datasource-opensearch, datasource-rds, plus DynamoDB and Lambda covered transitively Advanced features (9): caching (API + per-resolver), waf (rules, associations), logging-xray, custom-domain (CloudFormation-managed), introspection-disabled (with queryDepth/resolverCount limits), substitutions, environment-variables, api-keys-multiple, tags, visibility-private, schema-multiple-files Infrastructure: - e2e/helpers/synthesize.ts — runs sls package via subprocess, returns parsed CFN template, cleans up temp dirs - e2e/helpers/assertions.ts — readable assertion helpers (expectAuthenticationType, expectDataSourceOfType, etc.) - jest.e2e.config.ts + jest.e2e.setup.ts — separate config; globalSetup creates the examples/node_modules/serverless-appsync-plugin symlink so example projects resolve the plugin from the current source tree - Each example is a copy-paste-ready user project doubling as a test fixture, eliminating the docs/tests duplication problem - examples/README.md indexes all 24 examples for users - e2e/README.md explains the testing strategy for contributors CI: new 'e2e' job in .github/workflows/ci.yml runs after the unit-test matrix on every PR and push to master/alpha. Single Node version (22) since CFN output doesn't vary by Node version; full suite takes ~70s. Real plugin behaviors caught while building the fixtures (used to refine the examples): - API_KEY auth silently produces zero API keys if 'apiKeys:' is omitted - HTTP data sources with AWS_IAM authorization require explicit iamRoleStatements - OpenSearch data source can't auto-derive ARN from Fn::GetAtt; needs 'domain:' field naming the CloudFormation resource Verified: npm run test:all passes (287 unit + 68 e2e), npm run lint clean.

AlexHladin · 2026-05-25T08:27:44Z

+    needs: tests
+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v5


@sid88in let's switch to the latest v6 version

good catch! bumped

AlexHladin · 2026-05-25T08:28:59Z

+    steps:
+      - name: Checkout code
+        uses: actions/checkout@v5
+      - uses: actions/setup-node@v5


@sid88in this one, also could be switched to v6

good catch! bumped

AlexHladin · 2026-05-25T08:30:00Z

+        uses: actions/checkout@v5
+      - uses: actions/setup-node@v5
+        with:
+          node-version: 22


@sid88in the plugin supports 20, 22, 24, and 26 versions, just curious why 22?

CFN output is identical across Node versions. Running 4 Node versions for the synthesis tests would be 4x the CI minutes for zero added signal. The unit-test matrix already verifies the plugin works across Node 20/22/24/26 — those tests exercise the actual plugin logic. The e2e job tests the output of that logic, which doesn't change based on Node.
That said, 22 is somewhat arbitrary — could have been any of the four. For now I will keep it 22

AlexHladin · 2026-05-25T08:32:50Z

+    | 'API_KEY'
+    | 'AWS_IAM'
+    | 'AMAZON_COGNITO_USER_POOLS'
+    | 'OPENID_CONNECT'
+    | 'AWS_LAMBDA',


@sid88in, how about export this type from here, and use here?

good catch fixed

AlexHladin · 2026-05-25T08:37:03Z

+
+provider:
+  name: aws
+  runtime: nodejs20.x


@sid88in nodejs20 is not supported by AWS, only 22 and 24

good catch node 20x is EOL April 30, moved to 22

- Bump actions/checkout and actions/setup-node to v6 (latest stable) in both the tests and e2e jobs of .github/workflows/ci.yml - Bump all example runtimes from nodejs20.x to nodejs22.x. Node 20 reached EOL on April 30 2026; Lambda console removed it then and function updates will be blocked from Sep 30 2026. Examples should reflect the current recommended runtime so users copying them get a supported runtime by default. - Export AuthenticationType union from src/types/common.ts (derived from the existing Auth discriminated union via Auth['type']) and use it in e2e/helpers/assertions.ts instead of duplicating the literal string union. Keeps the e2e helper and the source of truth in sync automatically if a new auth type is ever added. Re: 'why Node 22 for the e2e job and not the full 20/22/24/26 matrix?' The CFN output produced by serverless package is identical across Node versions (no Node-specific code paths in synthesis). The existing unit-test matrix already verifies plugin logic across all four Node versions; running the synthesis tests across the same matrix would 4x CI minutes for zero added signal. 22 specifically matches what the Release workflow uses, keeping CI consistent.

sid88in · 2026-05-26T00:40:08Z

@AlexHladin please review the CR again. addressed comments - pls verify

sid88in requested a review from AlexHladin May 24, 2026 20:54

AlexHladin reviewed May 25, 2026

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

test: add CFN synthesis tests covering 24 plugin scenarios#704

test: add CFN synthesis tests covering 24 plugin scenarios#704
sid88in wants to merge 2 commits into
masterfrom
chore/cfn-synthesis-tests

sid88in commented May 24, 2026 •

edited

Loading

Uh oh!

AlexHladin May 25, 2026

Uh oh!

sid88in May 26, 2026

Uh oh!

AlexHladin May 25, 2026

Uh oh!

sid88in May 26, 2026

Uh oh!

AlexHladin May 25, 2026

Uh oh!

sid88in May 26, 2026 •

edited

Loading

Uh oh!

AlexHladin May 25, 2026

Uh oh!

sid88in May 26, 2026

Uh oh!

AlexHladin May 25, 2026

Uh oh!

sid88in May 26, 2026

Uh oh!

sid88in commented May 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

sid88in commented May 24, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

test: add CFN synthesis tests covering 24 plugin scenarios

What

Why

Coverage

Infrastructure

CI

Real plugin behaviors caught while building the fixtures

How to run locally

Verified

What's not in this PR (intentionally)

Follow-ups for after merge

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sid88in May 26, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sid88in commented May 26, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

sid88in commented May 24, 2026 •

edited

Loading

sid88in May 26, 2026 •

edited

Loading