Skip to content

checkly-pw-test.mdx

Latest commit

 

History

History
397 lines (264 loc) · 10.7 KB

checkly-pw-test.mdx

File metadata and controls

397 lines (264 loc) · 10.7 KB
title checkly pw-test
description Run Playwright tests with Checkly monitoring features
sidebarTitle checkly pw-test

Available since CLI v6.1.0.

The checkly pw-test command runs Playwright tests on Checkly's cloud infrastructure and supports standard Playwright CLI arguments. This command records test sessions by default.

Before using `checkly pw-test`, ensure you have:
  • An initialized Checkly CLI project
  • Valid Checkly account authentication (run npx checkly login if needed)
  • Playwright tests in your project
  • A playwright.config.ts or playwright.config.js file.

Basic Usage

Use -- to separate Checkly flags from Playwright test options.

npx checkly pw-test [checkly options] -- [playwright options]

Define checkly pw-test specific options before the -- separator:

Option Description
--config The Checkly CLI configuration file. If not passed, uses the checkly.config.ts|js file in the current directory.
--create-check Create a Checkly check from the Playwright test.
--env, -e Env vars to be passed to the test run. Default: []
--env-file dotenv file path to be passed. For example --env-file="./.env"
--installCommand Override the command used to install dependencies before running tests.
--location, -l The location to run the checks at.
--private-location The private location to run checks at.
--[no-]record Record test results in Checkly as a test session with full logs, traces and videos.
--reporter A list of custom reporters for the test output.
--stream-logs Stream logs from the test run to the console.
--test-session-name A name to use when storing results in Checkly
--timeout A timeout (in seconds) to wait for checks to complete.
--verbose Always show the full logs of the checks.

Define Playwright test runner flags after the -- separator:

Option Description
--project Select Playwright projects
--grep Filter tests by pattern
--grep-invert Exclude tests by pattern

The --reporter and --headed options are not supported. Find more Playwright options in the Playwright test runner docs.

Checkly Command Options

Specify a configuration file to use instead of the checkly.config.ts or checkly.config.js in the current directory.

Usage:

npx checkly pw-test --config="./checkly.staging.config.ts"

Add a new Playwright Check Suite with your Playwright configuration to your checkly.config.

Usage:

npx checkly pw-test --create-check

Examples:

Running this command:

 npx checkly pw-test --create-check -- --project="Mobile Chrome" --grep="@critical"

Adds a new Playwright Check Suite to your checkly.config.ts:

const config = defineConfig({
  projectName: "Playwright Project",
  logicalId: "playwright-project",

  checks: {
    playwrightConfigPath: "./playwright.config.ts",
    playwrightChecks: [
      {
        logicalId: "playwright-check-project-mobile-chrome-grep-critical",
        name: 'Playwright Test: "--project=Mobile Chrome" --grep=@critical',
        testCommand:
          'npx playwright test "--project=Mobile Chrome" --grep=@critical',
        locations: ["eu-central-1"],
        frequency: 10,
      },
    ],
  },
})

If there's no existing checkly.config.ts file, the --create-check option will create one.

Pass environment variables to the test run. Can be specified multiple times to set multiple variables.

Usage:

npx checkly pw-test --env API_KEY=123 --env BASE_URL=https://example.com
npx checkly pw-test -e NODE_ENV=production -e DEBUG=true

Load environment variables from a dotenv file.

Usage:

npx checkly pw-test --env-file="./.env"

Override the command used to install dependencies before running tests. By default, Checkly runs npm install --dev. Use this to customize the install step, for example to skip lifecycle scripts or use a different package manager.

Usage:

npx checkly pw-test --installCommand='npm install --no-scripts'
npx checkly pw-test --installCommand='pnpm install --frozen-lockfile'

Specify the geographic location where the checks should run. This determines which Checkly data center executes your tests.

Usage:

npx checkly pw-test --location="us-east-1"
npx checkly pw-test -l="eu-west-1"

Run checks at a specific private location. Private locations allow you to test internal applications or services behind a firewall.

Usage:

npx checkly pw-test --private-location="office-network"
npx checkly pw-test --private-location="staging-vpc"

The pw-test automatically records your test results as a test session with full logs, traces, and videos. Use --no-record to disable recording.

Usage:

npx checkly pw-test --record
npx checkly pw-test --no-record

Specify a custom reporter for the test output. Available options: list, dot, ci, github, json

Usage:

npx checkly pw-test --reporter="json"
npx checkly pw-test --reporter="github"

Logs appear in real time as tests run.

Usage:

npx checkly pw-test --stream-logs

Provide a custom name for the test session when storing results in Checkly. This helps identify and organize test runs.

Usage:

npx checkly pw-test --test-session-name="Release v1.2.3 tests"
npx checkly pw-test --test-session-name="Daily regression suite"

Set a timeout (in seconds) to wait for checks to complete.

Usage:

npx checkly pw-test --timeout="300"
npx checkly pw-test --timeout="1200"

The current maximum timeout is 1200 seconds (20 minutes).

Show full logs for every check, including passing ones.

Usage:

npx checkly pw-test --verbose

Common Playwright Test Options

The pw-test command reuses your playwright.config settings. To overwrite these values from the command line, Playwright Check Suites supports most Playwright test runner options.

Select specific Playwright projects to run. This allows you to run only a subset of your configured projects from your Playwright configuration.

Usage:

npx checkly pw-test -- --project="chromium"
npx checkly pw-test -- --project="firefox"

You can only run projects specified in your playwright.config file.

Filter tests to run based on a pattern match against test titles. Only tests matching the pattern will be executed.

Usage:

npx checkly pw-test -- --grep="@smoke"
npx checkly pw-test -- --grep="login|authentication"

Exclude tests from running based on a pattern match against test titles. Tests matching the pattern will be skipped.

Usage:

npx checkly pw-test -- --grep-invert="@slow"
npx checkly pw-test -- --grep-invert="flaky|unstable"

Playwright Configuration vs. Applied Command Line Options

Playwright Check Suites and the pw-test command read and parse your existing playwright.config to apply your configuration.

To differentiate between your local testing setup and the Checkly monitoring environment, you can rely on the set CI environment variable or override specific settings using command line options.

export default defineConfig({
  retries: process.env.CI ? 2 : 0,
  workers: process.env.CI ? 1 : undefined,
  use: {
    trace: 'on-first-retry',
  },
  projects: [
    {
      name: 'chromium',
      use: { ...devices['Desktop Chrome'] },
    },
    // ...
  ]
});

If you run npx checkly pw-test without additional arguments, the test suite will:

  • run with 2 retries
  • use 1 worker
  • run all defined projects including chromium

All these options can be further customized by passing command line arguments to pw-test.

# Run with 4 workers, only in chromium, and 3 retries
npx checkly pw-test -- --workers=4 --project="chromium" --retries=3

Examples

Validate smoke tagged tests from us-east-1

npx checkly pw-test --location="us-east-1" -- --grep="@smoke"

Debug production issues

npx checkly pw-test --verbose -- tests/checkout.spec.ts

Test from different locations

npx checkly pw-test --location="ap-southeast-1"
npx checkly pw-test --location="eu-west-1"

Run critical tests only

npx checkly pw-test --env ENVIRONMENT=production -- --grep="@critical"

Run tests in multiple browsers

npx checkly pw-test -- --project=chromium --project=firefox

Run a specific test file

npx checkly pw-test -- test.spec.ts

Key Features

  • Your Playwright configuration applies automatically (traces, videos, screenshots)
  • Test sessions are recorded by default with full logs, traces, and videos
  • View all artifacts in Checkly's UI

Related Commands