Skip to content

e2e(FR-2465): review and rewrite failing session E2E tests#6473

Merged
graphite-app[bot] merged 1 commit intomainfrom
04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests
Apr 17, 2026
Merged

e2e(FR-2465): review and rewrite failing session E2E tests#6473
graphite-app[bot] merged 1 commit intomainfrom
04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests

Conversation

@ironAiken2
Copy link
Copy Markdown
Contributor

Resolves #6416 (FR-2465)

Summary

Session E2E tests were failing due to environment version mismatches and code issues. This PR fixes the root causes:

  • TOML duplicate key error (test-util.ts): The test server's config.toml contains debug = true twice under [general]. Added a deduplicateTomlKeys() pre-processor to strip duplicate keys before passing to @iarna/toml parser, which is strict about duplicates.

  • Feature flag timing (session-scheduling-history-modal.spec.ts): page.evaluate() to set backendaiclient._features['session-scheduling-history'] was called before backendaiclient was ready. Fixed by adding waitForFunction to confirm client readiness post-navigation before evaluating.

  • selectDatetimeProperty scope bug (session-scheduling-history-modal.spec.ts): The helper referenced page from a closure that was not in scope at the describe block level. Fixed by adding page: Page as an explicit parameter and updating all 6 call sites.

  • Version-aware datetime filter tests (session-scheduling-history-modal.spec.ts): Created At / Updated At filter options require feat/FR-2302 (added after WebUI v26.3.0). Tests now skip gracefully with explanatory messages when running against an older server build.

  • Version-aware cluster mode warning tests (session-cluster-mode.spec.ts): The "Multi-node with size 1" warning requires ClusterModeFormItems.tsx from feat/FR-2381 (not in v26.3.0). Added version-aware skip logic to 3 tests.

  • Dependency tests requiring running agents (session-dependency.spec.ts): Tests that create sessions and wait for RUNNING state cannot pass on the test server (no available agents). Marked with test.fixme with explanatory message.

  • Coverage report update (E2E_COVERAGE_REPORT.md): Updated test status to reflect current fixme/skip state.

Test plan

  • e2e/utils/test-util.ts - deduplicateTomlKeys() handles the config.toml duplicate key issue
  • e2e/session/session-scheduling-history-modal.spec.ts - selectDatetimeProperty(page, modal, name) signature is correct; datetime tests skip gracefully on v26.3.0 server
  • e2e/session/session-cluster-mode.spec.ts - cluster mode warning tests skip gracefully when FR-2381 feature unavailable
  • e2e/session/session-dependency.spec.ts - tests requiring RUNNING sessions are properly marked fixme

🤖 Generated with Claude Code

Copilot AI review requested due to automatic review settings April 7, 2026 04:10
@github-actions github-actions Bot added the size:L 100~500 LoC label Apr 7, 2026
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR stabilizes Session-related Playwright E2E tests in Backend.AI WebUI by addressing strict TOML parsing failures, fixing test runtime ordering issues, and making tests resilient to feature/version differences across server/WebUI builds.

Changes:

  • Added a TOML pre-processor to tolerate duplicate keys when fetching/parsing config.toml in E2E utilities.
  • Updated Session E2E specs to (a) wait for backendaiclient readiness before toggling feature flags, and (b) skip tests gracefully when features are unavailable on older builds.
  • Marked dependency tests as fixme where the CI/test server cannot satisfy prerequisites, and updated the E2E coverage report accordingly.

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.

Show a summary per file
File Description
e2e/utils/test-util.ts Pre-processes fetched config.toml to remove duplicate keys before strict TOML parsing.
e2e/session/session-scheduling-history-modal.spec.ts Fixes feature-flag timing, fixes helper scoping via explicit page param, and adds version-aware skipping for datetime filter options.
e2e/session/session-dependency.spec.ts Marks tests that require RUNNING sessions as fixme due to lack of available agents on the test server.
e2e/session/session-cluster-mode.spec.ts Adds version-aware skip logic for the FR-2381 warning that isn’t present on older builds.
e2e/E2E_COVERAGE_REPORT.md Updates coverage statuses/date to reflect the new skip/fixme state.

Comment thread e2e/session/session-dependency.spec.ts Outdated
Comment thread e2e/E2E_COVERAGE_REPORT.md Outdated
@ironAiken2 ironAiken2 force-pushed the 04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests branch 2 times, most recently from 3e3d741 to 200d70f Compare April 13, 2026 07:25
Copy link
Copy Markdown
Contributor Author

ironAiken2 commented Apr 13, 2026


How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

  • flow:merge-queue - adds this PR to the back of the merge queue
  • flow:hotfix - for urgent changes, fast-track this PR to the front of the merge queue

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has required the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

@ironAiken2 ironAiken2 changed the base branch from main to graphite-base/6473 April 13, 2026 08:05
@ironAiken2 ironAiken2 force-pushed the 04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests branch from 200d70f to 7df7e6b Compare April 13, 2026 08:05
@ironAiken2 ironAiken2 changed the base branch from graphite-base/6473 to 04-07-e2e_fr-2472_review_and_rewrite_failing_serving_e2e_tests April 13, 2026 08:05
@ironAiken2 ironAiken2 force-pushed the 04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests branch from 7df7e6b to 71f4b27 Compare April 13, 2026 08:21
@ironAiken2 ironAiken2 force-pushed the 04-07-e2e_fr-2472_review_and_rewrite_failing_serving_e2e_tests branch from ed90df2 to a002e8a Compare April 13, 2026 08:21
@ironAiken2 ironAiken2 changed the base branch from 04-07-e2e_fr-2472_review_and_rewrite_failing_serving_e2e_tests to graphite-base/6473 April 13, 2026 10:16
@ironAiken2 ironAiken2 force-pushed the 04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests branch from 71f4b27 to 78069d7 Compare April 13, 2026 10:17
@ironAiken2 ironAiken2 changed the base branch from graphite-base/6473 to 04-07-e2e_fr-2472_review_and_rewrite_failing_serving_e2e_tests April 13, 2026 10:17
@ironAiken2 ironAiken2 force-pushed the 04-07-e2e_fr-2472_review_and_rewrite_failing_serving_e2e_tests branch from 3ab17bb to e7c518c Compare April 13, 2026 10:27
@ironAiken2 ironAiken2 force-pushed the 04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests branch from 78069d7 to abfd2b8 Compare April 13, 2026 10:27
@ironAiken2 ironAiken2 force-pushed the 04-07-e2e_fr-2472_review_and_rewrite_failing_serving_e2e_tests branch from e7c518c to 095a68e Compare April 13, 2026 11:48
@ironAiken2 ironAiken2 force-pushed the 04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests branch from abfd2b8 to 604a3aa Compare April 13, 2026 11:48
@ironAiken2 ironAiken2 force-pushed the 04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests branch from 604a3aa to daf4814 Compare April 13, 2026 11:57
@ironAiken2 ironAiken2 changed the title fix(FR-2465): review and rewrite failing session E2E tests e2e(FR-2465): review and rewrite failing session E2E tests Apr 14, 2026
@ironAiken2 ironAiken2 force-pushed the 04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests branch from daf4814 to d44c53c Compare April 15, 2026 04:41
@ironAiken2 ironAiken2 force-pushed the 04-07-e2e_fr-2472_review_and_rewrite_failing_serving_e2e_tests branch from 095a68e to a3f2e0f Compare April 15, 2026 04:41
Copy link
Copy Markdown
Contributor

@agatha197 agatha197 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

    at /Users/sujinkim/lablup/webui1/e2e/session/session-template-modal.spec.ts:245:33
  ✘  24 …l – Real Session History › Created session appears in Recent History and can be re-launched from template @session @functional (3.7m)


  1) [chromium] › e2e/session/session-template-modal.spec.ts:255:9 › Session Template Modal – Real Session History › Created session appears in Recent History and can be re-launched from template @session @functional 

    TimeoutError: page.waitForURL: Timeout 30000ms exceeded.
    =========================== logs ===========================
    waiting for navigation until "load"
      navigated to "http://localhost:9082/session/start?step=4&formValues=%7B%22sessionType%22%3A%22interactive%22%2C%22sessionName%22%3A%22e2e-template-1776266201926-ce1wlm%22%2C%22environments%22%3A%7B%22manual%22%3A%22cr.backend.ai%2Fstable%2Fpython%3A3.9-ubuntu20.04%40x86_64%22%7D%2C%22resourceGroup%22%3A%22default%22%2C%22allocationPreset%22%3A%22minimum-required%22%2C%22resource%22%3A%7B%22accelerator%22%3A0%2C%22cpu%22%3A0%2C%22mem%22%3A%220.125g%22%2C%22shmem%22%3A%2264m%22%7D%2C%22num_of_sessions%22%3A1%2C%22cluster_mode%22%3A%22multi-node%22%2C%22cluster_size%22%3A1%2C%22hpcOptimization%22%3A%7B%22autoEnabled%22%3Atrue%7D%2C%22mount_id_map%22%3A%7B%7D%2C%22vfoldersNameMap%22%3A%7B%7D%2C%22mount_ids%22%3A%5B%5D%2C%22enabledAutomaticShmem%22%3Atrue%2C%22agent%22%3A%22auto%22%2C%22envvars%22%3A%5B%5D%7D"
    ============================================================

      276 |       // Without this wait, navigateTo(page, 'session/start') could fire before
      277 |       // the async startSession API resolves, leaving localStorage empty.
    > 278 |       await page.waitForURL(/\/session(?!\/start)/, { timeout: 30_000 });
          |                  ^
      279 |
      280 |       // Verify session appears in session list
      281 |       const sessionRow = page.locator('tr').filter({ hasText: sessionName });
        at /Users/sujinkim/lablup/webui1/e2e/session/session-template-modal.spec.ts:278:18

    Test timeout of 180000ms exceeded while running "afterEach" hook.

      237 |     let createdSessionName: string | null = null;
      238 |
    > 239 |     test.afterEach(async ({ page }) => {
          |          ^
      240 |       // Cleanup: terminate created session if it exists
      241 |       if (createdSessionName) {
      242 |         try {
        at /Users/sujinkim/lablup/webui1/e2e/session/session-template-modal.spec.ts:239:10

    Error Context: test-results/session-session-template-m-66061-e-re-launched-from-template-chromium/error-context.md

  1 failed
    [chromium] › e2e/session/session-template-modal.spec.ts:255:9 › Session Template Modal – Real Session History › Created session appears in Recent History and can be re-launched from template @session @functional 

Can you check again? I tested with pnpm exec playwright test e2e/session/

@ironAiken2 ironAiken2 force-pushed the 04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests branch from d44c53c to 23bbbbd Compare April 16, 2026 01:34
@ironAiken2 ironAiken2 force-pushed the 04-07-e2e_fr-2472_review_and_rewrite_failing_serving_e2e_tests branch from a3f2e0f to ceae911 Compare April 16, 2026 01:34
@ironAiken2
Copy link
Copy Markdown
Contributor Author

Addressed feedback from @agatha197: Fixed the session-template-modal.spec.ts test failure. Replaced the page.waitForURL(/\/session(?!\/start)/) (which timed out at 30s) with the proven pattern from session-creation.spec.ts — waiting for the session row to be visible (60s timeout). This implicitly confirms both navigation and pushSessionHistory completion without relying on URL pattern matching.

@ironAiken2 ironAiken2 requested a review from agatha197 April 16, 2026 01:40
@ironAiken2 ironAiken2 force-pushed the 04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests branch from e49b8e1 to f7d9cad Compare April 16, 2026 09:38
@ironAiken2 ironAiken2 force-pushed the 04-07-e2e_fr-2472_review_and_rewrite_failing_serving_e2e_tests branch from ceae911 to ae01446 Compare April 16, 2026 09:38
Copy link
Copy Markdown
Contributor

@agatha197 agatha197 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ironAiken2 ironAiken2 force-pushed the 04-07-e2e_fr-2472_review_and_rewrite_failing_serving_e2e_tests branch from ae01446 to dd58370 Compare April 17, 2026 02:15
@ironAiken2 ironAiken2 force-pushed the 04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests branch from f7d9cad to b7d00b7 Compare April 17, 2026 02:15
@ironAiken2 ironAiken2 changed the base branch from 04-07-e2e_fr-2472_review_and_rewrite_failing_serving_e2e_tests to graphite-base/6473 April 17, 2026 02:44
@ironAiken2 ironAiken2 force-pushed the 04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests branch from b7d00b7 to 06a5d4b Compare April 17, 2026 02:44
@ironAiken2 ironAiken2 changed the base branch from graphite-base/6473 to main April 17, 2026 02:44
@graphite-app
Copy link
Copy Markdown

graphite-app Bot commented Apr 17, 2026

Merge activity

Resolves #6416 (FR-2465)

## Summary

Session E2E tests were failing due to environment version mismatches and code issues. This PR fixes the root causes:

- **TOML duplicate key error** (`test-util.ts`): The test server's `config.toml` contains `debug = true` twice under `[general]`. Added a `deduplicateTomlKeys()` pre-processor to strip duplicate keys before passing to `@iarna/toml` parser, which is strict about duplicates.

- **Feature flag timing** (`session-scheduling-history-modal.spec.ts`): `page.evaluate()` to set `backendaiclient._features['session-scheduling-history']` was called before `backendaiclient` was ready. Fixed by adding `waitForFunction` to confirm client readiness post-navigation before evaluating.

- **`selectDatetimeProperty` scope bug** (`session-scheduling-history-modal.spec.ts`): The helper referenced `page` from a closure that was not in scope at the describe block level. Fixed by adding `page: Page` as an explicit parameter and updating all 6 call sites.

- **Version-aware datetime filter tests** (`session-scheduling-history-modal.spec.ts`): `Created At` / `Updated At` filter options require feat/FR-2302 (added after WebUI v26.3.0). Tests now skip gracefully with explanatory messages when running against an older server build.

- **Version-aware cluster mode warning tests** (`session-cluster-mode.spec.ts`): The "Multi-node with size 1" warning requires `ClusterModeFormItems.tsx` from feat/FR-2381 (not in v26.3.0). Added version-aware skip logic to 3 tests.

- **Dependency tests requiring running agents** (`session-dependency.spec.ts`): Tests that create sessions and wait for RUNNING state cannot pass on the test server (no available agents). Marked with `test.fixme` with explanatory message.

- **Coverage report update** (`E2E_COVERAGE_REPORT.md`): Updated test status to reflect current fixme/skip state.

## Test plan

- [x] `e2e/utils/test-util.ts` - `deduplicateTomlKeys()` handles the `config.toml` duplicate key issue
- [x] `e2e/session/session-scheduling-history-modal.spec.ts` - `selectDatetimeProperty(page, modal, name)` signature is correct; datetime tests skip gracefully on v26.3.0 server
- [x] `e2e/session/session-cluster-mode.spec.ts` - cluster mode warning tests skip gracefully when FR-2381 feature unavailable
- [x] `e2e/session/session-dependency.spec.ts` - tests requiring RUNNING sessions are properly marked fixme

🤖 Generated with [Claude Code](https://claude.com/claude-code)
@graphite-app graphite-app Bot force-pushed the 04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests branch from 06a5d4b to 99c55d0 Compare April 17, 2026 02:45
@graphite-app graphite-app Bot merged commit 99c55d0 into main Apr 17, 2026
8 checks passed
@graphite-app graphite-app Bot deleted the 04-07-fix_fr-2465_review_and_rewrite_failing_session_e2e_tests branch April 17, 2026 02:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Review and rewrite failing Session e2e tests

3 participants