Conversation
There was a problem hiding this comment.
The ReporterRuntime.stopStep method doesn't stop stage steps. It's not a problem when we're handling API steps (lambda/log) because the stage steps are stopped by #handleStopStepMessage. But some integrations call stopStep directly to introduce framework-specific steps. If such a step contains a stage, it breaks the resulting step structure. CodeceptJS, Cucumber.js, and Playwright are affected.
A sidenote: Cypress is not affected because it handles steps differently (at browser side).
Case 1: CodeceptJS
codecept.conf.js:
exports.config = {
name: "codeceptjs",
tests: "./specs/*.spec.js",
output: "./output",
plugins: {
allure: {
resultsDir: "./allure-results",
require: "allure-codeceptjs",
},
},
helpers: {
MyHelper: {
require: "./helpers.js",
},
},
};helpers.js:
const Helper = require("@codeceptjs/helper");
const { stage } = require("allure-js-commons");
module.exports = class extends Helper {
async load() {
await stage("Loading the data 1");
await stage("Loading the data 2");
await Promise.resolve();
}
async navigate() {
await stage("Open the tab");
await stage("Open the dialog");
await Promise.resolve();
}
};./specs/sample.spec.js:
Feature("sample feature");
Scenario("sample test", async ({ I }) => {
I.load();
I.navigate();
});
Case 2: Cucumber.js
cucumber.js:
module.exports = {
default: {
format: ["allure-cucumberjs/reporter:./out/ignore"],
},
};features/sample.feature:
Feature: sample feature
Scenario: sample scenario
Given foo
When bar
Then bazfeatures/support/sample.js:
const { Given, When, Then } = require("@cucumber/cucumber");
const { stage } = require("allure-js-commons");
Given("foo", async function () {
await stage("Loading the data");
});
When("bar", async function () {});
Then("baz", async function () {});features/support/world.js:
require("allure-cucumberjs")
Case 3: Playwright
In theory, Playwright is also affected, but in practice, this effect is masked by a step synchronization issue. Gonna create a separate issue for this (if not yet exists).
Edited: #1444.
import { test } from '@playwright/test';
import { stage, step } from "allure-js-commons";
test("steps", async () => {
await step("outer", async () => {
await stage("inner stage");
});
await step("sibling", async () => {});
});
| import { logStep, stage, step } from "allure-js-commons"; | ||
|
|
||
| test("steps", async () => { | ||
| stage("stage 1"); |
There was a problem hiding this comment.
| stage("stage 1"); | |
| await stage("stage 1"); |
| await logStep("a"); | ||
| await step("b", async () => { | ||
| await logStep("b 1"); | ||
| stage("b 2"); |
There was a problem hiding this comment.
| stage("b 2"); | |
| await stage("b 2"); |
| await logStep("b 2 nested"); | ||
| }); | ||
|
|
||
| stage("stage 2"); |
There was a problem hiding this comment.
| stage("stage 2"); | |
| await stage("stage 2"); |
| test("steps", async () => { | ||
| stage("stage 1"); | ||
| await logStep("a"); | ||
| await step("b", async () => { |
There was a problem hiding this comment.
WDYT about removing this lambda step and asserting the steps structure directly? This will simplify the test significantly. Basically, the current test splits this lambda step away anyway, but in a more complex way.
Or just assert the current (invalid) structure with the lambda step as the first step. This way, the test will fail if/when we fix the synchronization issue with Playwright steps, so we don't forget to update it.
In any case, I'll mention this test in an issue for Playwright steps to fix once the issue is resolved.
Context
This PR adds
stage(name)as a new runtime API for integrations.stage()is a lightweight way to describe the major phases of a test without wrapping the code instep()callbacks. It is intended for cases where a test naturally flows through preparation, SUT execution, and verification, and where values created in one phase should remain in normal local variables for the next phase.Instead of restructuring the test just to create reporting boundaries,
stage()lets us mark a phase and treat everything that follows as part of that stage until the next stage starts or the current execution context ends.In the report, this produces three top-level steps:
prepare datasubmit orderverify resultAny runtime steps logged between stage boundaries become nested under the active stage. Nested stages are also supported, so a
stage()started inside astep()becomes a child of that step.High-level behavior:
stage()is syntax sugar for semantic test phasesChecklist