Skip to content

Testify/playwright#1552

Open
mkugelstad wants to merge 22 commits into
masterfrom
testify/playwright
Open

Testify/playwright#1552
mkugelstad wants to merge 22 commits into
masterfrom
testify/playwright

Conversation

@mkugelstad
Copy link
Copy Markdown

This PR introduce Playwright support in EvoMaster’s JavaScript test generator. The changes include:

  • Implemented Playwright request generation.

  • Fixed header serialization, request body handling, and test initialization issues in generated Playwright tests.

  • Removed duplicate Playwright generator entries to ensure a single, consistent code path.

  • Added Playwright dependencies to BlackBox test modules and updated BlackBoxUtils.kt with correct base/relative paths for Playwright execution.

  • Multiple merges from master to keep the branch aligned with upstream changes.

All modifications should be isolated to Playwright‑specific code paths to avoid regressions in other generators.

May Kristin Ugelstad and others added 21 commits March 4, 2026 13:52
HttpWsTestCaseWriter still work in progress.
…r cookie and token handling, and for GraphQl.
# Conflicts:
#	core/src/main/kotlin/org/evomaster/core/output/service/HttpWsTestCaseWriter.kt
…javascript and hopefully fix playwright syntax error.
…and add function for running PLaywright tests.

HttpsWsTestCaseWriter.kt: Remove await request handling for Playwright and hopefully fix header handling for response.
RestTestCaseWriter.kt: Add await for playwright
SpringTestBase.kt: Add playwright support
Add playwright.config
…errors for generating playwright output: ApiTestCaseWriter.kt: update json response size assertions, CookieWriter.kt: update assignment of cookie value, httpWsTestCaseWriter.kt: update header and add verb handling, RestTestCaseWriter.kt: update verb handling incl. fetch method, TokenWriter.kt: update token handling
@mkugelstad mkugelstad requested a review from arcuri82 May 20, 2026 11:06
Copy link
Copy Markdown
Collaborator

@arcuri82 arcuri82 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thx for the PR! see my comments

private const val GENERATED_FOLDER_NAME = "generated"

const val baseLocationForJavaScript = "$JS_BASE_PATH/$GENERATED_FOLDER_NAME"
const val baseLocationForPlaywright = "$JS_BASE_PATH/$GENERATED_FOLDER_NAME/playwright"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why the need for a new folder location instead of using same baseLocationForJavaScript? also, if really need new folder, should had written like:

const val baseLocationForPlaywright = "$baseLocationForJavaScript/playwright"


fun relativePath(folderName: String) = "$GENERATED_FOLDER_NAME/$folderName"

fun relativePathPlaywright(folderName: String) = "$GENERATED_FOLDER_NAME/playwright/$folderName"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

see previous comment. this function might not be needed

"$folderRelativePath/"
}

val npx = if (isWindows()) "npx.cmd" else "npx"
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we have fun npm() and fun mvn(). for consistency should introduce fun npx()

runTestsCommand(command, JS_BASE_PATH, "NPM")
}

fun runPlaywrightTests(folderRelativePath: String) {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do you need this function and not using runNpmTests()? aren't tests generated for JS_JEST_PLAYWRIGHT valid jest tests? why do they require to run npx playwright instead of using npm test?
add comments to clarify and explain the reason

folderRelativePath
} else {
"$folderRelativePath/"
}
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

code here seems duplicated from runNpmTests(). if indeed this function is needed (see previous comment) than should refactor (eg a single function with possible more parameters) to avoid code duplication

}

if (format.isJavaScript()) {
if (format.isJavaScript()) { // add to code block - ok for playwright?
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is the comment still valid?

}

if (format.isJavaScript()){
if (format.isJavaScript() && !format.isPlaywright()){
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

are you making sure of disabling redirect in Playwright? if so, could leave brief comment here stating where it is done

if (config.testTimeout > 0) {

if (config.testTimeout > 0 && !format.isPlaywright()) {
lines.add("jest.setTimeout(${config.testTimeout * 1000});")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aren't JS Playwright tests using Jest?

if (config.outputFormat.isPlaywright()) {
lines.add("const $baseUrlOfSut = \"${BlackBoxUtils.targetUrl(config, sampler)}\";")
} else {
lines.add("const $baseUrlOfSut = \"${BlackBoxUtils.targetUrl(config, sampler)}\";")
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

aren't these 2 branches doing the same?

* Return true if the output format is Playwright.
* Playwright is currently only supported for JavaScript (or TypeScript).
*/
fun isPlaywright() = this.name.endsWith("_playwright", true) // Testing new playwright imp
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is comment // Testing new playwright still relevant?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants