Merge pull request #85 from Integration-Automation/dev

Extended capabilities batch + MCP server

Author: JE-Chen

.github/workflows/test_dev.yml

@@ -46,6 +46,12 @@ jobs:
       matrix:
         python-version: ["3.10", "3.11", "3.12", "3.13"]
 
+    env:
+      # webdriver-manager calls api.github.com to find geckodriver/chromedriver
+      # releases; the unauthenticated quota (60/h per IP) gets hammered in CI.
+      # Pass the workflow token so requests use the 5000/h authenticated limit.
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
     steps:
     - uses: actions/checkout@v4
 

.github/workflows/test_stable.yml

@@ -46,6 +46,12 @@ jobs:
       matrix:
         python-version: ["3.10", "3.11", "3.12", "3.13"]
 
+    env:
+      # webdriver-manager hits api.github.com for geckodriver/chromedriver
+      # release lookups; share the workflow token to use the authenticated
+      # 5000/h quota instead of the 60/h per-IP limit.
+      GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
     steps:
     - uses: actions/checkout@v4
 

README.md

@@ -536,6 +536,98 @@ Companion APIs — `WR_run_for_users` (multi-user matrix), `WR_run_ab` (A/B mode
 - **HAR diff** — `WR_diff_har` / `WR_diff_har_files` show added / removed / status-changed requests between two runs.
 - **Arbitrary-script gate** — `executor.set_allow_arbitrary_script(False)` blocks `WR_execute_script` / `WR_execute_async_script` / `WR_pw_evaluate` / `WR_cdp` / `WR_pw_cdp` for untrusted action JSON.
 
+## Extended Capabilities
+
+Reliability & flake reduction:
+
+- **Adaptive retry** — `je_web_runner.utils.adaptive_retry.run_with_retry(fn, policy=...)` replays only failures the classifier marks transient / flaky / environment; real bugs short-circuit.
+- **Locator strength scorer** — `linter.locator_strength.score_locator(strategy, value)` ranks locators 0–100; `assert_strength` fails CI on fragile XPath / TAG_NAME picks.
+- **Smart wait** — `smart_wait.wait_for_fetch_idle` and `wait_for_spa_route_stable` patch `window.fetch` and `history.pushState` to detect SPA quiescence — no more `time.sleep`.
+- **Service throttler** — `throttler.throttle("payments-api")` is a file-semaphore that caps cross-shard concurrency on a shared service.
+
+Debugging & observability:
+
+- **Timeline merger** — `observability.timeline.build(spans=, console=, responses=)` merges OTel spans, console messages, and network responses into one chronologically-sorted event list.
+- **Failure bundle** — `failure_bundle.FailureBundle("login_test", error_repr).add_screenshot(...).write("bundle.zip")` packages screenshots / DOM / network / console / trace into a single replayable zip with manifest.
+- **Memory leak detector** — `memory_leak.detect_growth(driver, action, iterations=10, growth_bytes_per_iter_budget=...)` polls `performance.memory.usedJSHeapSize` and fails on linear-fit growth above budget.
+- **Playwright trace recorder** — `trace_recorder.TraceRecorder(output_dir="trace-out").start(context, name); …; .stop(context)` always writes a `.zip` viewable with `playwright show-trace`.
+- **CSP reporter** — `csp_reporter.CspViolationCollector` injects a `securitypolicyviolation` listener and exposes `assert_none()` / `assert_no_directive("script-src")`.
+
+Test data & determinism:
+
+- **Record/replay fixture** — `snapshot.fixture_record.FixtureRecorder("fx.json", mode="auto")` saves the producer's output the first time, replays it forever after.
+- **DB fixture loader** — `database.fixtures.load_fixture_file("seed.json")` + `load_into_connection(conn, fixture)` seeds testcontainers Postgres / MySQL / SQLite from a `{table: [rows]}` JSON.
+
+API & contract testing:
+
+- **API mocking** — `api_mock.MockRouter().add("GET", "/api/users/*", body={"id": 1}).attach_to_page(page)` intercepts Playwright routes; URL globs and `re:` regex patterns supported.
+- **Contract testing** — `contract_testing.validate_response(body, schema)` runs a JSON-Schema subset; `validate_against_openapi(body, doc, "/users/{id}", "GET", 200)` resolves `$ref` and checks the right schema for the response status.
+- **GraphQL helper** — `graphql.GraphQLClient("https://api/graphql").execute("{ me { id } }")`; `extract_field(payload, "me.id")` plucks values via dotted path.
+- **In-process mock services** — `mock_services.MockOAuthServer().start()` issues fake bearer tokens, `MockSmtpServer` captures sent mails, `MockS3Storage` is a memory KV.
+
+Security probes:
+
+- **Header tampering** — `header_tampering.HeaderTampering().set_header("X-Forwarded-For", "192.0.2.1").attach_to_page(page)` mutates outbound requests so testers can probe missing-CSRF / wrong-origin / stripped-auth handling.
+- **License scanner** — `license_scanner.scan_text(bundle_text)` finds SPDX identifiers and known license phrases (AGPL/GPL/MIT/Apache-2.0/MPL/ISC/BSD) so SBOM gates can `assert_allowed_licenses`.
+
+Browser & locale:
+
+- **Device emulation presets** — `device_emulation.playwright_kwargs("iPhone 15 Pro")` and `apply_to_chrome_options(opts, "Desktop 1080p")`; viewport + DPR + UA + touch in one call.
+- **Geo / TZ / locale** — `geo_locale.GeoOverride(latitude=51.5, longitude=-0.13, timezone="Europe/London", locale="en-GB")` produces both CDP commands and Playwright `new_context` kwargs.
+- **Multi-tab choreographer** — `multi_tab.TabChoreographer().open_new(driver, "side", url=...)` registers tabs by alias so action JSON can `WR_switch_tab("side")`.
+- **WebAuthn virtual authenticator** — `webauthn.enable_virtual_authenticator(driver)` uses CDP `WebAuthn.*` to simulate passkey / FIDO2 sign-in flows.
+- **Cookie consent dismisser** — `cookie_consent.ConsentDismisser().dismiss(driver)` clicks the first matching OneTrust / TrustArc / Cookiebot / Didomi / Quantcast button; selector list extensible via `register_selector`.
+
+Reporting & CI:
+
+- **PR comment poster** — `pr_comment.post_or_update_comment("owner/repo", 42, body, token=...)` is idempotent via a hidden HTML marker so retried CI runs don't pile up.
+- **Trend dashboard** — `trend_dashboard.compute_trend("ledger.json")` buckets the ledger by day; `render_html(trend)` produces a self-contained SVG line chart + table.
+
+Orchestration & developer experience:
+
+- **Action template library** — `action_templates.render_template("login_basic", {...})` substitutes `{{placeholders}}` in built-in flows (login, accept-cookies, switch-locale, close-modal).
+- **Diff-aware shard** — `sharding.diff_shard.select_for_changed(candidates, base_ref="main")` filters candidates to those touched by the current branch's `git diff`.
+- **Watch mode** — `watch_mode.watch_loop(directory, on_change=callback, interval=0.5)` re-runs a callback whenever JSON files change.
+- **Kubernetes runner** — `k8s_runner.render_job_manifests(ShardJobConfig(name_prefix="run", image=..., total_shards=8, actions_dir="/actions"))` produces one `batch/v1 Job` per shard.
+- **Per-route perf budgets** — `perf_metrics.budgets.evaluate_metrics("/checkout", {"lcp_ms": 2300}, budgets)` plus `assert_within_budget(result)` enforce route-specific thresholds.
+
+AI assistance:
+
+- **Failure RCA** — `ai_assist.llm_assist.explain_failure(test_name, error_repr, console=, network=, steps=)` asks the registered LLM for `{likely_cause, evidence, next_steps, confidence}`.
+
+## MCP Server
+
+WebRunner ships a [Model Context Protocol](https://modelcontextprotocol.io/) server so any MCP-aware client (Claude, IDE plugins, etc.) can drive WebRunner over JSON-RPC stdio.
+
+```bash
+python -m je_web_runner.mcp_server
+```
+
+The default tool list exposes:
+
+- `webrunner_lint_action`, `webrunner_locator_strength`
+- `webrunner_render_template`, `webrunner_compute_trend`
+- `webrunner_validate_response`, `webrunner_summary_markdown`
+- `webrunner_diff_shard`, `webrunner_render_k8s`, `webrunner_partition_shard`
+
+```python
+from je_web_runner.mcp_server import McpServer, Tool, build_default_tools, serve_stdio
+
+# Or build a custom server
+server = McpServer()
+for tool in build_default_tools():
+    server.register(tool)
+server.register(Tool(
+    name="my_custom_tool",
+    description="…",
+    input_schema={"type": "object", "properties": {"x": {"type": "string"}}},
+    handler=lambda args: f"hello {args['x']}",
+))
+serve_stdio(server=server)
+```
+
+The server speaks MCP `2024-11-05`: `initialize`, `tools/list`, `tools/call`, `resources/list`, `ping`, `shutdown`.
+
 ## Browser Internals
 
 ```python

docs/source/Eng/doc/extended_features/extended_features_doc.rst

@@ -262,3 +262,130 @@ registers any ``Callable[[str], str]`` and powers:
   self-healing locator flow.
 * ``generate_actions_from_prompt(request)`` — natural language → action
   JSON draft.
+* ``explain_failure(test_name, error_repr, console=, network=, steps=)``
+  — produces a JSON RCA: ``{likely_cause, evidence, next_steps,
+  confidence}``.
+
+Reliability helpers
+===================
+
+* ``adaptive_retry.run_with_retry(fn, policy=...)`` — retries only when
+  the failure classifier labels the exception transient / flaky /
+  environment; ``RetryPolicy`` exposes per-category budgets and history.
+* ``linter.locator_strength.score_locator(strategy, value)`` — scores a
+  locator on a 0–100 scale; ``score_action_locators`` runs across an
+  action JSON list.
+* ``smart_wait.wait_for_fetch_idle`` / ``wait_for_spa_route_stable`` —
+  inject window.fetch and history hooks to detect SPA quiescence.
+* ``throttler.throttle("payments-api")`` — file-semaphore for cross-shard
+  concurrency limits.
+
+Observability
+=============
+
+* ``observability.timeline.build(spans=, console=, responses=)`` —
+  merges three event sources into a chronological list.
+* ``failure_bundle.FailureBundle("test", error_repr).write("bundle.zip")``
+  — replayable zip with manifest (``screenshot`` / ``dom`` / ``console``
+  / ``network`` / ``trace`` / arbitrary text & files).
+* ``memory_leak.detect_growth(driver, action, iterations=10)`` —
+  performance.memory linear-fit slope; ``growth_bytes_per_iter_budget``
+  raises on regression.
+* ``trace_recorder.TraceRecorder().start(context, name) / .stop(context)``
+  — Playwright tracing wrapper that always emits a ``.zip``.
+* ``csp_reporter.CspViolationCollector`` — securitypolicyviolation
+  listener with ``assert_none`` / ``assert_no_directive``.
+
+Test data & determinism
+=======================
+
+* ``snapshot.fixture_record.FixtureRecorder("fx.json", mode="auto")`` —
+  record once, replay forever; modes ``record`` / ``replay`` / ``auto``.
+* ``database.fixtures.load_fixture_file("seed.json")`` +
+  ``load_into_connection(conn, fixture)`` — seed Postgres / MySQL /
+  SQLite from ``{table: [rows]}`` JSON.
+
+API & contract testing
+======================
+
+* ``api_mock.MockRouter().add(method, url_pattern, body=, status=, times=)``
+  — supports literal, glob, and ``re:`` regex URL patterns; attach to a
+  Playwright page with ``attach_to_page(page)``.
+* ``contract_testing.validate_response(body, schema)`` — JSON-Schema
+  subset (type / properties / required / items / enum / oneOf /
+  additionalProperties); ``validate_against_openapi`` resolves
+  ``$ref`` and looks up ``paths[…].responses[…]``.
+* ``graphql.GraphQLClient(endpoint).execute(query, variables=)`` +
+  ``extract_field(payload, "users[0].name")``.
+* ``mock_services`` — ``MockOAuthServer``, ``MockSmtpServer``,
+  ``MockS3Storage`` for offline CI runs.
+
+Security probes
+===============
+
+* ``header_tampering.HeaderTampering()`` — rule list + Playwright
+  ``page.route()`` integration to set / remove / append headers.
+* ``license_scanner.scan_text(bundle_text)`` — find SPDX identifiers and
+  known license phrases; ``assert_allowed_licenses(findings, allow=,
+  deny=)`` for SBOM gates.
+* ``cookie_consent.ConsentDismisser().dismiss(driver)`` — auto-click
+  OneTrust / TrustArc / Cookiebot / Didomi / Quantcast accept buttons.
+
+Browser & locale
+================
+
+* ``device_emulation`` — ``available_presets`` /
+  ``playwright_kwargs("iPhone 15 Pro")`` /
+  ``apply_to_chrome_options(opts, "Desktop 1080p")`` /
+  ``cdp_emulation_command(name)``.
+* ``geo_locale.GeoOverride`` — yields both
+  ``cdp_payloads(override)`` and ``playwright_context_kwargs(override)``.
+* ``multi_tab.TabChoreographer`` — track tabs by alias;
+  ``register_current`` / ``open_new`` / ``switch_to`` / ``with_tab`` /
+  ``close``.
+* ``webauthn.enable_virtual_authenticator(driver)`` — CDP
+  ``WebAuthn.addVirtualAuthenticator`` for passkey simulation.
+
+Reporting & CI
+==============
+
+* ``pr_comment.post_or_update_comment(repo, pr_number, body, token=)``
+  — idempotent via a hidden HTML marker.
+* ``trend_dashboard.compute_trend("ledger.json")`` +
+  ``render_html(trend)`` — daily pass-rate / duration / SVG chart.
+
+Orchestration & DX
+==================
+
+* ``action_templates.render_template("login_basic", {...})`` —
+  built-in templates: ``login_basic``, ``accept_cookies``,
+  ``switch_locale``, ``close_modal``; ``register_template`` for custom.
+* ``sharding.diff_shard.select_for_changed(candidates, base_ref="main")``
+  — git-diff-aware test selection.
+* ``watch_mode.watch_loop(directory, on_change=callback)`` — polled file
+  watcher with snapshot diff.
+* ``k8s_runner.render_job_manifests(ShardJobConfig(...))`` /
+  ``render_job_yaml(config)`` — one ``batch/v1 Job`` per shard.
+* ``perf_metrics.budgets`` — ``load_budgets("budgets.json")`` +
+  ``evaluate_metrics(route, metrics, budgets)`` +
+  ``assert_within_budget(result)``.
+
+MCP server
+==========
+
+WebRunner ships a Model Context Protocol server so MCP-aware clients can
+drive it over JSON-RPC stdio:
+
+.. code-block:: shell
+
+   python -m je_web_runner.mcp_server
+
+Default tools registered: ``webrunner_lint_action``,
+``webrunner_locator_strength``, ``webrunner_render_template``,
+``webrunner_compute_trend``, ``webrunner_validate_response``,
+``webrunner_summary_markdown``, ``webrunner_diff_shard``,
+``webrunner_render_k8s``, ``webrunner_partition_shard``.
+
+Custom tools register via ``McpServer.register(Tool(...))``; the server
+implements MCP ``2024-11-05`` (``initialize`` / ``tools/list`` /
+``tools/call`` / ``resources/list`` / ``ping`` / ``shutdown``).

docs/source/Zh/doc/extended_features/extended_features_doc.rst

@@ -189,3 +189,84 @@ WebRunner 不打包任何 LLM client。透過 ``set_llm_callable(fn)`` 註冊任
 
 * ``suggest_locator`` — 自我修復定位的 LLM 後援
 * ``generate_actions_from_prompt`` — 自然語言生成 action 草稿
+* ``explain_failure`` — 從失敗素材生成 RCA：``{likely_cause, evidence,
+  next_steps, confidence}``
+
+可靠度
+======
+
+* ``adaptive_retry.run_with_retry`` — 依 classifier 結果決定是否重試
+* ``linter.locator_strength.score_locator`` — locator 0–100 分強度評估
+* ``smart_wait.wait_for_fetch_idle`` / ``wait_for_spa_route_stable`` —
+  比 ``time.sleep`` 智慧的 SPA 等待
+* ``throttler.throttle("svc")`` — 跨 shard 的檔案信號量
+
+可觀測性
+========
+
+* ``observability.timeline.build`` — 合併 OTel span / console / 網路回應
+* ``failure_bundle.FailureBundle`` — 失敗素材打包成可重現的 zip
+* ``memory_leak.detect_growth`` — heap 線性回歸找洩漏
+* ``trace_recorder.TraceRecorder`` — Playwright tracing 包裝
+* ``csp_reporter.CspViolationCollector`` — CSP 違規監聽
+
+測試資料 / 確定性
+=================
+
+* ``snapshot.fixture_record.FixtureRecorder`` — 第一次跑記錄、之後重放
+* ``database.fixtures`` — YAML/JSON → SQLAlchemy 連線 seed
+
+API 與合約
+==========
+
+* ``api_mock.MockRouter`` — Playwright route() 上層的宣告式 mock
+* ``contract_testing`` — JSON Schema 子集 + OpenAPI ``$ref`` 解析
+* ``graphql.GraphQLClient`` — GraphQL HTTP client + ``extract_field``
+* ``mock_services`` — SMTP / OAuth / S3 in-process mock
+
+安全測試
+========
+
+* ``header_tampering.HeaderTampering`` — 改 cookie/referer/origin
+* ``license_scanner`` — SPDX / 已知授權字樣偵測
+* ``cookie_consent.ConsentDismisser`` — 自動關閉 GDPR 彈窗
+
+裝置 / 區域
+===========
+
+* ``device_emulation`` — iPhone / Pixel / iPad / Desktop 預設
+* ``geo_locale`` — geolocation / timezone / locale 一次設定
+* ``multi_tab.TabChoreographer`` — 多分頁腳本連動
+* ``webauthn.enable_virtual_authenticator`` — passkey / FIDO2 模擬
+
+報告 / CI
+=========
+
+* ``pr_comment.post_or_update_comment`` — GitHub PR 自動留言（idempotent）
+* ``trend_dashboard.compute_trend`` — ledger 日趨勢 + SVG 圖表
+
+編排 / 開發者體驗
+=================
+
+* ``action_templates`` — login_basic / accept_cookies / switch_locale /
+  close_modal 等可重用樣板
+* ``sharding.diff_shard`` — 只跑 git diff 影響到的測試
+* ``watch_mode.watch_loop`` — 檔案變動監看
+* ``k8s_runner.render_job_manifests`` — 每個 shard 一個 batch/v1 Job
+* ``perf_metrics.budgets`` — 每路由 FCP/LCP/CLS 預算
+
+MCP server
+==========
+
+提供 Model Context Protocol stdio JSON-RPC server：
+
+.. code-block:: shell
+
+   python -m je_web_runner.mcp_server
+
+預設工具：``webrunner_lint_action`` / ``webrunner_locator_strength`` /
+``webrunner_render_template`` / ``webrunner_compute_trend`` /
+``webrunner_validate_response`` / ``webrunner_summary_markdown`` /
+``webrunner_diff_shard`` / ``webrunner_render_k8s`` /
+``webrunner_partition_shard``。可透過 ``McpServer.register(Tool(...))``
+擴充自訂工具，協定版本 ``2024-11-05``。

je_web_runner/mcp_server/__init__.py

@@ -0,0 +1,9 @@
+"""WebRunner MCP server: expose WR_* actions over the Model Context Protocol."""
+from je_web_runner.mcp_server.server import (
+    McpServer,
+    McpServerError,
+    build_default_tools,
+    serve_stdio,
+)
+
+__all__ = ["McpServer", "McpServerError", "build_default_tools", "serve_stdio"]

je_web_runner/mcp_server/__main__.py

@@ -0,0 +1,6 @@
+"""Entry point so ``python -m je_web_runner.mcp_server`` starts the stdio server."""
+from je_web_runner.mcp_server.server import serve_stdio
+
+
+if __name__ == "__main__":
+    serve_stdio()