Skip to content

LCORE-1472: Lcore 1472 improve ci time#1498

Merged
radofuchs merged 3 commits into
lightspeed-core:mainfrom
radofuchs:LCORE_1472_improve_ci_time
Apr 14, 2026
Merged

LCORE-1472: Lcore 1472 improve ci time#1498
radofuchs merged 3 commits into
lightspeed-core:mainfrom
radofuchs:LCORE_1472_improve_ci_time

Conversation

@radofuchs

@radofuchs radofuchs commented Apr 14, 2026

Copy link
Copy Markdown
Contributor

Description

Type of change

  • Refactor
  • New feature
  • Bug fix
  • CVE fix
  • Optimization
  • Documentation Update
  • Configuration Update
  • Bump-up service version
  • Bump-up dependent library
  • Bump-up library or tool used for development (does not change the final image)
  • CI configuration change
  • Konflux configuration change
  • Unit tests improvement
  • Integration tests improvement
  • End to end tests improvement
  • Benchmarks improvement

Tools used to create PR

Identify any AI code assistants used in this PR (for transparency and review context)

  • Assisted-by: (e.g., Claude, CodeRabbit, Ollama, etc., N/A if not used)
  • Generated by: (e.g., tool name and version; N/A if not used)

Related Tickets & Documents

  • Related Issue #
  • Closes #

Checklist before requesting a review

  • I have performed a self-review of my code.
  • PR has passed all pre-merge test jobs.
  • If it is a core feature, I have added thorough tests.

Testing

  • Please provide detailed steps to perform tests related to this code change.
  • How were the fix/results from this change verified? Please provide relevant screenshots or results.

Summary by CodeRabbit

  • Tests

    • Implemented parallelized end-to-end test execution across three groups for improved CI/CD performance.
    • Added per-feature test timing measurements to performance logs.
    • Reorganized test features with group categorization for parallel execution support.
  • Chores

    • Updated CI/CD workflow to distribute end-to-end tests across multiple execution groups.
    • Enhanced test infrastructure with configurable tag-based test filtering.

@coderabbitai

coderabbitai Bot commented Apr 14, 2026

Copy link
Copy Markdown
Contributor

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: ASSERTIVE

Plan: Pro

Run ID: 306b44c4-c2db-44e3-bb60-b443218c4734

📥 Commits

Reviewing files that changed from the base of the PR and between 258aaa7 and 3fd37d7.

📒 Files selected for processing (33)
  • .github/workflows/e2e_tests.yaml
  • Makefile
  • tests/e2e/features/authorized_noop.feature
  • tests/e2e/features/authorized_noop_token.feature
  • tests/e2e/features/authorized_rh_identity.feature
  • tests/e2e/features/conversation_cache_v2.feature
  • tests/e2e/features/conversations.feature
  • tests/e2e/features/environment.py
  • tests/e2e/features/faiss.feature
  • tests/e2e/features/feedback.feature
  • tests/e2e/features/health.feature
  • tests/e2e/features/http_401_unauthorized.feature
  • tests/e2e/features/info.feature
  • tests/e2e/features/inline_rag.feature
  • tests/e2e/features/llama_stack_disrupted.feature
  • tests/e2e/features/mcp.feature
  • tests/e2e/features/mcp_servers_api.feature
  • tests/e2e/features/mcp_servers_api_auth.feature
  • tests/e2e/features/mcp_servers_api_no_config.feature
  • tests/e2e/features/models.feature
  • tests/e2e/features/proxy.feature
  • tests/e2e/features/query.feature
  • tests/e2e/features/rbac.feature
  • tests/e2e/features/responses.feature
  • tests/e2e/features/responses_streaming.feature
  • tests/e2e/features/rest_api.feature
  • tests/e2e/features/rlsapi_v1.feature
  • tests/e2e/features/rlsapi_v1_errors.feature
  • tests/e2e/features/smoketests.feature
  • tests/e2e/features/steps/proxy.py
  • tests/e2e/features/steps/tls.py
  • tests/e2e/features/streaming_query.feature
  • tests/e2e/features/tls.feature
📜 Recent review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (7)
  • GitHub Check: Konflux kflux-prd-rh02 / lightspeed-stack-on-pull-request
  • GitHub Check: unit_tests (3.13)
  • GitHub Check: build-pr
  • GitHub Check: Pylinter
  • GitHub Check: E2E Tests for Lightspeed Evaluation job
  • GitHub Check: E2E: library mode / ci
  • GitHub Check: E2E: server mode / ci
🧰 Additional context used
📓 Path-based instructions (2)
tests/e2e/features/**/*.feature

📄 CodeRabbit inference engine (AGENTS.md)

Use behave (BDD) framework with Gherkin feature files for end-to-end tests

Files:

  • tests/e2e/features/info.feature
  • tests/e2e/features/http_401_unauthorized.feature
  • tests/e2e/features/authorized_noop_token.feature
  • tests/e2e/features/health.feature
  • tests/e2e/features/rest_api.feature
  • tests/e2e/features/models.feature
  • tests/e2e/features/feedback.feature
  • tests/e2e/features/mcp.feature
  • tests/e2e/features/conversation_cache_v2.feature
  • tests/e2e/features/conversations.feature
  • tests/e2e/features/authorized_rh_identity.feature
  • tests/e2e/features/authorized_noop.feature
  • tests/e2e/features/proxy.feature
  • tests/e2e/features/llama_stack_disrupted.feature
  • tests/e2e/features/responses.feature
  • tests/e2e/features/faiss.feature
  • tests/e2e/features/query.feature
  • tests/e2e/features/tls.feature
  • tests/e2e/features/mcp_servers_api_auth.feature
  • tests/e2e/features/rlsapi_v1_errors.feature
  • tests/e2e/features/responses_streaming.feature
  • tests/e2e/features/mcp_servers_api_no_config.feature
  • tests/e2e/features/mcp_servers_api.feature
  • tests/e2e/features/streaming_query.feature
  • tests/e2e/features/rlsapi_v1.feature
  • tests/e2e/features/rbac.feature
  • tests/e2e/features/inline_rag.feature
  • tests/e2e/features/smoketests.feature
tests/**/*.py

📄 CodeRabbit inference engine (AGENTS.md)

tests/**/*.py: Use pytest for all unit and integration tests; do not use unittest
Use pytest.mark.asyncio marker for async unit tests

Files:

  • tests/e2e/features/steps/tls.py
  • tests/e2e/features/environment.py
  • tests/e2e/features/steps/proxy.py
🧠 Learnings (10)
📓 Common learnings
Learnt from: CR
Repo: lightspeed-core/lightspeed-stack PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-04-05T12:19:36.009Z
Learning: Maintain E2E test list in `tests/e2e/test_list.txt`
Learnt from: CR
Repo: lightspeed-core/lightspeed-stack PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-04-05T12:19:36.009Z
Learning: Applies to tests/e2e/features/**/*.feature : Use behave (BDD) framework with Gherkin feature files for end-to-end tests
📚 Learning: 2026-04-05T12:19:36.009Z
Learnt from: CR
Repo: lightspeed-core/lightspeed-stack PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-04-05T12:19:36.009Z
Learning: Place E2E step definitions in `tests/e2e/features/steps/`

Applied to files:

  • tests/e2e/features/info.feature
  • tests/e2e/features/authorized_noop_token.feature
  • tests/e2e/features/health.feature
  • tests/e2e/features/rest_api.feature
  • tests/e2e/features/models.feature
  • tests/e2e/features/feedback.feature
  • tests/e2e/features/mcp.feature
  • tests/e2e/features/conversation_cache_v2.feature
  • tests/e2e/features/conversations.feature
  • tests/e2e/features/authorized_noop.feature
  • tests/e2e/features/proxy.feature
  • tests/e2e/features/llama_stack_disrupted.feature
  • tests/e2e/features/responses.feature
  • tests/e2e/features/faiss.feature
  • tests/e2e/features/query.feature
  • tests/e2e/features/tls.feature
  • tests/e2e/features/responses_streaming.feature
  • tests/e2e/features/rlsapi_v1.feature
  • tests/e2e/features/rbac.feature
  • tests/e2e/features/inline_rag.feature
  • tests/e2e/features/smoketests.feature
  • Makefile
📚 Learning: 2026-04-05T12:19:36.009Z
Learnt from: CR
Repo: lightspeed-core/lightspeed-stack PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-04-05T12:19:36.009Z
Learning: Applies to tests/e2e/features/**/*.feature : Use behave (BDD) framework with Gherkin feature files for end-to-end tests

Applied to files:

  • tests/e2e/features/info.feature
  • tests/e2e/features/authorized_noop_token.feature
  • tests/e2e/features/health.feature
  • tests/e2e/features/rest_api.feature
  • tests/e2e/features/models.feature
  • tests/e2e/features/feedback.feature
  • tests/e2e/features/mcp.feature
  • tests/e2e/features/conversation_cache_v2.feature
  • tests/e2e/features/conversations.feature
  • tests/e2e/features/authorized_rh_identity.feature
  • tests/e2e/features/authorized_noop.feature
  • tests/e2e/features/llama_stack_disrupted.feature
  • tests/e2e/features/responses.feature
  • tests/e2e/features/faiss.feature
  • tests/e2e/features/query.feature
  • tests/e2e/features/responses_streaming.feature
  • .github/workflows/e2e_tests.yaml
  • tests/e2e/features/rlsapi_v1.feature
  • tests/e2e/features/rbac.feature
  • tests/e2e/features/environment.py
  • tests/e2e/features/inline_rag.feature
  • tests/e2e/features/smoketests.feature
  • Makefile
📚 Learning: 2026-04-05T12:19:36.009Z
Learnt from: CR
Repo: lightspeed-core/lightspeed-stack PR: 0
File: AGENTS.md:0-0
Timestamp: 2026-04-05T12:19:36.009Z
Learning: Maintain E2E test list in `tests/e2e/test_list.txt`

Applied to files:

  • tests/e2e/features/info.feature
  • tests/e2e/features/health.feature
  • tests/e2e/features/rest_api.feature
  • .github/workflows/e2e_tests.yaml
  • tests/e2e/features/smoketests.feature
  • Makefile
📚 Learning: 2025-09-02T11:15:02.411Z
Learnt from: radofuchs
Repo: lightspeed-core/lightspeed-stack PR: 485
File: tests/e2e/test_list.txt:2-3
Timestamp: 2025-09-02T11:15:02.411Z
Learning: In the lightspeed-stack e2e tests, the Authorized tag is intentionally omitted from noop authentication tests because they are designed to test against the default lightspeed-stack.yaml configuration rather than the specialized noop-with-token configuration.

Applied to files:

  • tests/e2e/features/http_401_unauthorized.feature
  • tests/e2e/features/authorized_noop_token.feature
  • tests/e2e/features/conversation_cache_v2.feature
  • tests/e2e/features/conversations.feature
  • tests/e2e/features/authorized_rh_identity.feature
  • tests/e2e/features/authorized_noop.feature
  • tests/e2e/features/proxy.feature
  • tests/e2e/features/llama_stack_disrupted.feature
  • tests/e2e/features/responses.feature
  • tests/e2e/features/faiss.feature
  • tests/e2e/features/query.feature
  • tests/e2e/features/mcp_servers_api_auth.feature
  • tests/e2e/features/responses_streaming.feature
  • tests/e2e/features/streaming_query.feature
  • tests/e2e/features/rlsapi_v1.feature
📚 Learning: 2025-09-02T11:09:40.404Z
Learnt from: radofuchs
Repo: lightspeed-core/lightspeed-stack PR: 485
File: tests/e2e/features/environment.py:87-95
Timestamp: 2025-09-02T11:09:40.404Z
Learning: In the lightspeed-stack e2e tests, noop authentication tests use the default lightspeed-stack.yaml configuration, while noop-with-token tests use the Authorized tag to trigger a config swap to the specialized noop-with-token configuration file.

Applied to files:

  • tests/e2e/features/http_401_unauthorized.feature
  • tests/e2e/features/authorized_noop_token.feature
  • tests/e2e/features/conversation_cache_v2.feature
  • tests/e2e/features/conversations.feature
  • tests/e2e/features/authorized_rh_identity.feature
  • tests/e2e/features/authorized_noop.feature
  • tests/e2e/features/proxy.feature
  • tests/e2e/features/llama_stack_disrupted.feature
  • tests/e2e/features/responses.feature
  • tests/e2e/features/faiss.feature
  • tests/e2e/features/query.feature
  • tests/e2e/features/mcp_servers_api_auth.feature
  • tests/e2e/features/responses_streaming.feature
  • tests/e2e/features/streaming_query.feature
  • tests/e2e/features/rlsapi_v1.feature
📚 Learning: 2025-09-02T11:14:17.117Z
Learnt from: radofuchs
Repo: lightspeed-core/lightspeed-stack PR: 485
File: tests/e2e/features/steps/common_http.py:244-255
Timestamp: 2025-09-02T11:14:17.117Z
Learning: The POST step in tests/e2e/features/steps/common_http.py (`access_rest_api_endpoint_post`) is intentionally designed as a general-purpose HTTP POST method, not specifically for REST API endpoints, so it should not include context.api_prefix in the URL construction.

Applied to files:

  • tests/e2e/features/health.feature
  • tests/e2e/features/rest_api.feature
  • tests/e2e/features/models.feature
📚 Learning: 2026-04-13T13:39:54.963Z
Learnt from: radofuchs
Repo: lightspeed-core/lightspeed-stack PR: 1490
File: tests/e2e/features/environment.py:206-211
Timestamp: 2026-04-13T13:39:54.963Z
Learning: In lightspeed-stack e2e tests (tests/e2e/features/), `context.feature_config` is intentionally set inside Background/step functions (scenario-scoped Behave layer). The `after_scenario` restore logic in `environment.py` only restores config when `context.scenario_lightspeed_override_active` is True, which is only set by `configure_service` when an actual config switch occurs. The module-level `_active_lightspeed_stack_config_basename` in `tests/e2e/features/steps/common.py` prevents re-applying the same config in subsequent scenarios (making `scenario_lightspeed_override_active` stay False). This means the ephemeral nature of step-set context attributes is intentional — the design ensures config restore happens exactly once per actual switch, not redundantly on every scenario.

Applied to files:

  • tests/e2e/features/llama_stack_disrupted.feature
📚 Learning: 2026-04-13T13:39:54.963Z
Learnt from: radofuchs
Repo: lightspeed-core/lightspeed-stack PR: 1490
File: tests/e2e/features/environment.py:206-211
Timestamp: 2026-04-13T13:39:54.963Z
Learning: In lightspeed-stack E2E tests under tests/e2e/features, it is intentional to set context.feature_config inside Background/step functions (scenario-scoped Behave layer). The environment.py after_scenario restore logic should only restore configuration when context.scenario_lightspeed_override_active is True; this flag is set by configure_service only when a real config switch occurs (so restore does not run for scenarios without a switch). Additionally, steps/common.py’s module-level _active_lightspeed_stack_config_basename is used to prevent re-applying the same config across subsequent scenarios, ensuring scenario_lightspeed_override_active stays False after the first apply. Therefore, reviewers should not “fix” this flow as if feature_config were incorrectly scoped or if after_scenario restoration is missing—config switching and restoration are meant to happen exactly once per actual switch, not redundantly per scenario.

Applied to files:

  • tests/e2e/features/steps/tls.py
  • tests/e2e/features/environment.py
  • tests/e2e/features/steps/proxy.py
📚 Learning: 2026-04-07T09:20:26.590Z
Learnt from: radofuchs
Repo: lightspeed-core/lightspeed-stack PR: 1467
File: tests/e2e/features/steps/common.py:36-49
Timestamp: 2026-04-07T09:20:26.590Z
Learning: For Behave-based Python tests, rely on Behave’s Context layered stack for attribute lifecycle: Behave pushes a new Context layer when entering feature scope (before_feature) and again for scenario scope (before_scenario). Attributes assigned inside given/when/then steps live on the current scenario layer and are automatically removed when the scenario ends. As a result, step-set attributes should not be expected to persist across scenarios or features, and manual cleanup in after_scenario/after_feature is generally unnecessary for attributes set in step functions. Only perform manual cleanup for attributes that you set explicitly in before_feature/before_scenario, since those live on the respective feature/scenario layers.

Applied to files:

  • tests/e2e/features/steps/tls.py
  • tests/e2e/features/environment.py
  • tests/e2e/features/steps/proxy.py
🔇 Additional comments (32)
tests/e2e/features/mcp.feature (1)

1-1: E2E test distribution across groups is reasonably balanced by scenario count.

Actual scenario distribution:

  • group_1: 73 scenarios (rest_api, responses, responses_streaming, tls, faiss, mcp_servers_api_no_config, mcp_servers_api_auth, rlsapi_v1_errors, authorized_noop)
  • group_2: 89 scenarios (mcp, models, health, authorized_noop_token, conversation_cache_v2, conversations, rlsapi_v1, streaming_query, rbac)
  • group_3: 105 scenarios (info, http_401_unauthorized, feedback, query, llama_stack_disrupted, inline_rag, proxy, mcp_servers_api, authorized_rh_identity, smoketests)

The distribution ranges from 73 to 105 scenarios (~44% difference), which is reasonable for parallelization. Individual large features like responses.feature (24 scenarios), mcp.feature (30 scenarios), and http_401_unauthorized.feature (24 scenarios) are distributed across different groups, helping balance the load.

If execution time metrics show group_3 takes significantly longer despite having fewer long-running tests, consider using scenario-level tags or splitting features to balance by expected execution time rather than scenario count.

tests/e2e/features/feedback.feature (1)

1-1: Tag update looks correct.

Line 1 correctly adds the shard tag while preserving the existing feature classification tag.

As per coding guidelines: tests/e2e/features/**/*.feature: Use behave (BDD) framework with Gherkin feature files for end-to-end tests.

tests/e2e/features/llama_stack_disrupted.feature (1)

1-1: Tag composition is good.

Line 1 keeps existing behavior tags and cleanly adds shard grouping.

As per coding guidelines: tests/e2e/features/**/*.feature: Use behave (BDD) framework with Gherkin feature files for end-to-end tests.

tests/e2e/features/query.feature (1)

1-1: Looks good for shard tagging.

Line 1 is a correct feature-level tag update and preserves @Authorized.

As per coding guidelines: tests/e2e/features/**/*.feature: Use behave (BDD) framework with Gherkin feature files for end-to-end tests.

tests/e2e/features/conversations.feature (1)

1-1: Feature tag update is correct.

Line 1 properly applies shard grouping without altering existing authorization tagging semantics.

As per coding guidelines: tests/e2e/features/**/*.feature: Use behave (BDD) framework with Gherkin feature files for end-to-end tests.

tests/e2e/features/rlsapi_v1_errors.feature (1)

1-1: Good shard-tag addition.

Line 1 preserves RBAC scoping and adds a valid group tag for parallel E2E execution.

As per coding guidelines: tests/e2e/features/**/*.feature: Use behave (BDD) framework with Gherkin feature files for end-to-end tests.

tests/e2e/features/authorized_rh_identity.feature (1)

1-1: Tag update is correct.

Line 1 cleanly adds shard grouping while retaining the identity-test classification tag.

As per coding guidelines: tests/e2e/features/**/*.feature: Use behave (BDD) framework with Gherkin feature files for end-to-end tests.

tests/e2e/features/smoketests.feature (1)

1-1: Looks good.

Line 1 applies the feature-level shard tag correctly.

As per coding guidelines: tests/e2e/features/**/*.feature: Use behave (BDD) framework with Gherkin feature files for end-to-end tests.

tests/e2e/features/health.feature (1)

1-1: Shard-tag coverage and test_list consistency verified.

All E2E features have proper @e2e_group_[1-3] tags, no features have conflicting multi-group tags, and all changed features are accounted for in tests/e2e/test_list.txt.

tests/e2e/features/mcp_servers_api_no_config.feature (1)

1-1: Shard tag addition is clean and non-disruptive.

@e2e_group_1 is added correctly while preserving the existing feature tag behavior.

As per coding guidelines: tests/e2e/features/**/*.feature: Use behave (BDD) framework with Gherkin feature files for end-to-end tests.

tests/e2e/features/authorized_noop_token.feature (1)

1-1: Good shard tagging with auth semantics preserved.

Keeping @Authorized while adding @e2e_group_2 is the correct combination for this noop-with-token feature.

Based on learnings: noop-with-token tests use the @Authorized tag to trigger the specialized noop-with-token configuration.

tests/e2e/features/responses_streaming.feature (1)

1-1: Tag update looks correct for sharded execution.

@e2e_group_1 is applied cleanly and existing @Authorized behavior remains intact.

tests/e2e/features/faiss.feature (1)

1-1: LGTM on feature shard tagging.

The group tag is added correctly and the existing authorization tag remains preserved.

tests/e2e/features/steps/tls.py (1)

94-95: Comment clarification is accurate and helpful.

This improves intent clarity without changing step behavior.

tests/e2e/features/conversation_cache_v2.feature (1)

1-1: Correct shard assignment with required auth tag retained.

Adding @e2e_group_2 while preserving @Authorized is consistent for this feature.

Based on learnings: noop-with-token tests use the @Authorized tag to trigger the specialized noop-with-token configuration file.

tests/e2e/features/rbac.feature (1)

1-1: RBAC feature tagging change looks good.

@e2e_group_2 is added correctly with the existing @RBAC tag preserved.

tests/e2e/features/authorized_noop.feature (1)

1-1: Nice: shard tag added without changing noop auth behavior.

Adding only @e2e_group_1 here is correct for default noop coverage.

Based on learnings: in noop authentication tests, @Authorized is intentionally omitted to keep the default lightspeed-stack.yaml path.

tests/e2e/features/inline_rag.feature (1)

1-2: LGTM!

The @e2e_group_3 tag is correctly added for E2E test sharding. Valid Gherkin syntax.

tests/e2e/features/mcp_servers_api_auth.feature (1)

1-2: LGTM!

The @e2e_group_1 tag is correctly prepended while preserving the existing @MCPServerAPIAuth tag. Valid Gherkin syntax for multiple tags.

tests/e2e/features/streaming_query.feature (1)

1-2: LGTM!

The @e2e_group_2 tag is correctly added while preserving the @Authorized tag that triggers the noop-with-token configuration swap.

tests/e2e/features/rlsapi_v1.feature (1)

1-2: LGTM!

The @e2e_group_2 tag is correctly added while preserving the @Authorized tag. Consistent with the grouping pattern used in streaming_query.feature.

.github/workflows/e2e_tests.yaml (3)

14-16: LGTM!

The matrix expansion to include e2e_group: [1, 2, 3] correctly enables parallel E2E execution across 3 shards. Job naming clearly identifies the group for easier debugging.


192-200: LGTM!

The E2E_BEHAVE_TAG_EXPR environment variable correctly uses behave tag expression syntax (not @Skipand@e2e_group_N``), and the switch to make test-e2e-tagged aligns with the new Makefile target.


107-107: No action needed. The tests/e2e/test_list.txt file is a manifest of feature files to execute, not a location for managing test tags. Tags like @e2e_group_* are defined within the feature files themselves using Gherkin syntax, not in this list file. The current structure is correct.

			> Likely an incorrect or invalid review comment.
tests/e2e/features/mcp_servers_api.feature (1)

1-2: LGTM!

The @e2e_group_3 tag is correctly added while preserving the existing @MCP tag. Consistent grouping with other MCP-related features.

tests/e2e/features/steps/proxy.py (1)

8-12: LGTM!

The docstring update clearly documents the intentional design: restores only modify the backup file without triggering restarts, requiring explicit restart steps in feature files. This improves readability and aligns with the config switching pattern noted in learnings.

tests/e2e/features/environment.py (3)

37-38: LGTM!

Good practice using a module-level constant for the attribute name, avoiding magic strings scattered across hooks.


355-359: LGTM!

Storing the start timestamp on the feature object (rather than context) is the correct approach—it survives Behave's context layer resets between scenarios. Using time.perf_counter() is appropriate for measuring elapsed wall-clock time.


408-423: LGTM!

The timing output implementation is solid:

  • Defensive getattr check handles cases where before_feature didn't run
  • Feature label extraction gracefully falls back to feature.name
  • flush=True ensures immediate output
  • capture = False on the hook guarantees timing lines appear in CI logs regardless of test outcome
Makefile (3)

35-36: Local E2E target change is solid.

Running without the script wrapper here is a good DX improvement and still preserves execution via the canonical E2E list.

Based on learnings: Maintain E2E test list in tests/e2e/test_list.txt.


42-47: Tagged CI/local targets are well-structured.

The paired test-e2e-tagged and test-e2e-tagged-local targets keep behavior aligned while supporting CI wrapper requirements.

Based on learnings: Applies to tests/e2e/features/**/*.feature : Use behave (BDD) framework with Gherkin feature files for end-to-end tests.


40-40: Tag prefix consistency is optional, not required for correctness.

Behave's tag-expression v2 protocol (default since v1.2.7) treats e2e_group_1 and @e2e_group_1 as equivalent for exact tag matching. Adding @ prefixes is a stylistic choice for consistency if you prefer uniform formatting, but unprefixed group tags are not parser-sensitive or problematic as originally stated.


Walkthrough

This PR implements E2E test parallelization across 3 concurrent CI jobs. Features are tagged with @e2e_group_1, @e2e_group_2, or @e2e_group_3 to distribute tests. The Makefile adds configurable tag-based test targets. Feature execution timing metrics are added to monitor performance.

Changes

Cohort / File(s) Summary
CI/Build Configuration
.github/workflows/e2e_tests.yaml, Makefile
Added 3-group matrix sharding to E2E workflow; added E2E_BEHAVE_TAG_EXPR variable and new test-e2e-tagged / test-e2e-tagged-local targets for filtered tag-based test execution. Workflow now invokes make test-e2e-tagged with per-group tag filtering.
Test Environment & Observability
tests/e2e/features/environment.py
Adds per-feature wall-clock timing measurement via before_feature / after_feature hooks; stores monotonic start time on feature object, computes elapsed seconds, prints [e2e feature timing] to console, and disables hook output capture.
Proxy & TLS Test Infrastructure
tests/e2e/features/steps/proxy.py, tests/e2e/features/steps/tls.py
Modified restore_if_modified step to restore run.yaml without restarting containers; container restart control now managed explicitly via feature steps. Updated module documentation to clarify responsibility delegation.
E2E Feature Tag Distribution
tests/e2e/features/*.feature (30+ files)
Added @e2e_group_1, @e2e_group_2, or @e2e_group_3 tags to feature definitions to distribute test load across parallel CI jobs. Tags are assigned to balance test groups; existing feature logic and scenarios remain unchanged.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~35 minutes

Possibly related PRs

  • lightspeed-stack#1413 — Modifies the same E2E environment hooks (after_feature) and proxy cleanup step (restore_if_modified).
  • lightspeed-stack#1410 — Updates proxy test orchestration and E2E step definitions (tests/e2e/features/steps/proxy.py).

Suggested labels

review-effort-3, test-infrastructure, e2e-testing

Suggested reviewers

  • tisnik
  • max-svistunov
🚥 Pre-merge checks | ✅ 2 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name Status Explanation Resolution
Title check ❓ Inconclusive The title references a Jira ticket and describes the goal ('improve ci time') but lacks specificity about the main implementation approach—sharding E2E tests into three groups. Consider a more descriptive title like 'Shard E2E tests into three parallel groups to improve CI time' to clarify the primary mechanism of improvement.
✅ Passed checks (2 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Docstring Coverage ✅ Passed Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
✨ Simplify code
  • Create PR with simplified code

Warning

Review ran into problems

🔥 Problems

Timed out fetching pipeline failures after 30000ms


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@tisnik tisnik left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@radofuchs radofuchs merged commit 05607dd into lightspeed-core:main Apr 14, 2026
25 of 26 checks passed
Anxhela21 pushed a commit to Anxhela21/lightspeed-stack that referenced this pull request Apr 27, 2026
* improve timing

---------

Co-authored-by: Radovan Fuchs <rfuchs@rfuchs-thinkpadp1gen7.tpb.csb>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants