Skip to content

Add post-build smoke E2E against built .zip artifact #241

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Mwalek merged 2 commits into from
May 15, 2026
Merged

Add post-build smoke E2E against built .zip artifact #241

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ad3a446
Add post-build smoke E2E against built .zip artifact
Mwalek May 15, 2026
154e40c
Address CodeRabbit review on .circleci/config.yml
Mwalek May 15, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
64 changes: 56 additions & 8 deletions .circleci/config.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,54 @@ jobs:
else
echo "Skipping translation pipeline."
fi
- run:
name: Collect build artifacts
command: |
mkdir -p .release
cp gf-entries-in-excel-*.zip .release/
- store_artifacts:
path: .release
destination: release
- persist_to_workspace:
root: /home/circleci
paths:
- plugin/.release
- plugin/gf-entries-in-excel-*.zip

run_post_build_smoke:
<<: *default_job_config
machine:
image: default
docker_layer_caching: true
resource_class: xlarge
steps:
- attach_workspace:
at: /home/circleci
- run:
name: Post-build smoke
# Unzips .release/*.zip, restores dev deps via gktools' Docker composer,
# boots wp-env with retry, runs `playwright test --grep=@smoke` against
# the built artifact. See @gravitykit/e2e-bootstrap README for details.
command: |
npx @gravitykit/e2e-bootstrap smoke \
--composer-cmd "npx @gravitykit/gktools composer install"
Comment on lines +145 to +151

@coderabbitai coderabbitai Bot May 15, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟡 Minor | ⚡ Quick win

🧩 Analysis chain

🏁 Script executed:

#!/bin/bash
# Description: Check if `@gravitykit/e2e-bootstrap` smoke command handles retries internally

# Check the e2e-bootstrap package README or source for retry logic
gh repo clone gravitykit/e2e-bootstrap /tmp/e2e-bootstrap-check 2>/dev/null || true
if [ -d /tmp/e2e-bootstrap-check ]; then
  echo "=== Checking for retry logic in e2e-bootstrap ==="
  rg -i "retry|attempt" /tmp/e2e-bootstrap-check --type md --type js -C 3
fi

Repository: GravityKit/GravityExport-Lite

Length of output: 55

🏁 Script executed:

# Search for e2e-bootstrap references in the current repo to understand how it's used
rg "e2e-bootstrap" . -A 2 -B 2

Repository: GravityKit/GravityExport-Lite

Length of output: 2884

🏁 Script executed:

# Check the CircleCI config for dependency versions or references
cat -n .circleci/config.yml | sed -n '140,160p'

Repository: GravityKit/GravityExport-Lite

Length of output: 991

🌐 Web query:

@gravitykit/e2e-bootstrap smoke command retry wp-env setup

💡 Result:

The @gravitykit/e2e-bootstrap package is used in conjunction with wp-env to facilitate end-to-end (E2E) testing workflows, particularly in CI/CD environments like GitHub Actions [1]. There is no native "smoke command" or built-in "retry" flag within @gravitykit/e2e-bootstrap itself for wp-env setup. Instead, retry logic is typically implemented at the CI/CD workflow level (e.g., in.circleci/config.yml or GitHub Actions YAML) [1]. To implement a robust retry mechanism for wp-env setup, the following pattern is recommended: 1. Implement a loop in your CI configuration that attempts to run the setup command (e.g., npm run tests:e2e:setup) multiple times [1]. 2. Ensure that each retry attempt starts from a clean state. Simply deleting the local metadata directory (e.g., /home/circleci/.wp-env) is often insufficient because stale Docker containers or volumes may persist [1]. 3. The retry loop should explicitly invoke a teardown routine (such as wp-env stop or a custom npm run tests:e2e:teardown task) upon failure before attempting to clear the environment and retry [1]. For example, in a CI environment, you might structure your loop to: - Run the setup command. - If it fails, execute the teardown command. - Remove the.wp-env directory. - Sleep briefly. - Retry the setup [1]. This approach ensures that subsequent attempts do not inherit corrupted or stale configurations from failed previous runs [1].

Citations:

The comment claiming "boots wp-env with retry" is inaccurate; the @gravitykit/e2e-bootstrap smoke command does not implement retry logic internally.

Unlike run_e2e_tests (lines 62-75) which has an explicit retry loop, this job lacks retry handling for wp-env setup failures. The e2e-bootstrap package does not provide built-in retry functionality—retry logic must be implemented at the CI/CD level with proper teardown and cleanup on failure.

Consider adding a retry loop similar to run_e2e_tests, or update the comment to remove the misleading claim about retries.

🤖 Prompt for AI Agents 
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In @.circleci/config.yml around lines 146 - 152, The comment over the
"Post-build smoke" job is misleading because the npx `@gravitykit/e2e-bootstrap`
smoke command does not perform retries; either implement CI-level retry behavior
similar to the existing run_e2e_tests job or update the comment to remove the
"boots wp-env with retry" claim. To fix, locate the "Post-build smoke" job in
.circleci/config.yml where the command uses npx `@gravitykit/e2e-bootstrap` smoke
and either add a shell retry loop with proper teardown/cleanup that mirrors
run_e2e_tests' retry pattern, or change the job comment to accurately describe
that no retry is performed and retries must be handled at the CI level.

- store_artifacts:
path: tests/E2E/results
- store_artifacts:
path: tests/E2E/report
- store_test_results:
path: tests/E2E/results/junit.xml

publish_release:
<<: *default_job_config
steps:
- attach_workspace:
at: /home/circleci
- run:
name: Configure git
command: |
git config user.email "support@gravitykit.com"
git config user.name "GravityKit - CI"
- run:
name: Create GitHub release
command: npx @gravitykit/gktools release
Expand All @@ -127,14 +175,6 @@ jobs:
if ! git log -1 --pretty=%B | grep -iq "\[skip notify\]"; then
npx @gravitykit/gktools announce
fi
- run:
name: Collect build artifacts
command: |
mkdir -p .release
cp gf-entries-in-excel-*.zip .release/ 2>/dev/null || true
- store_artifacts:
path: .release
destination: release

workflows:
version: 2
Expand All @@ -148,3 +188,11 @@ workflows:
<<: *context
requires:
- prepare
- run_post_build_smoke:
<<: *context
requires:
- build_package_release
- publish_release:
<<: *context
requires:
- run_post_build_smoke
11 changes: 7 additions & 4 deletions package-lock.json
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

2 changes: 1 addition & 1 deletion package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
"tests:e2e:run": "playwright test --config=tests/E2E/setup/playwright.config.js"
},
"devDependencies": {
"@gravitykit/e2e-bootstrap": "^1.0.0",
"@gravitykit/e2e-bootstrap": "^1.1.0",
"@playwright/test": "1.56.1",
"@wordpress/env": "^10.0.0",
"grunt": "^1.5.3",
Expand Down
2 changes: 1 addition & 1 deletion tests/E2E/tests/activation.spec.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
const { test, expect } = require('@playwright/test');

test.describe('GravityExport Lite — Activation Smoke Test', () => {
test.describe('GravityExport Lite — Activation Smoke Test @smoke', () => {

test('Plugin activates without fatal PHP errors', async ({ page } ) => {
await page.goto('/wp-admin/plugins.php');
Expand Down
Loading