Skip to content

Commit a522ac3

Browse files
committed
Document the specialized modules in READMEs and Sphinx tree
Adds a Specialized Modules section to README.md grouping the 73 new modules by capability area (Web Platform APIs / Security / Perf / Backend Integration / AI Workflow / a11y-i18n-Visual / Governance), mirrors the same in README_zh-TW.md and README_zh-CN.md, and adds a dedicated Sphinx chapter under docs/source/{Eng,Zh}/doc/specialized_modules/ wired into the Quality & Data chapter of both language indices.
1 parent ae24937 commit a522ac3

7 files changed

Lines changed: 1437 additions & 0 deletions

File tree

README.md

Lines changed: 224 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,7 @@ WebRunner (`je_web_runner`) started as a Selenium wrapper and grew into a full a
4343
- [Observability](#observability)
4444
- [Test Orchestration](#test-orchestration)
4545
- [Quality & Security](#quality--security)
46+
- [Specialized Modules](#specialized-modules)
4647
- [Advanced WebDriverWrapper](#advanced-webdriverwrapper)
4748
- [Browser Internals](#browser-internals)
4849
- [Test Data](#test-data)
@@ -845,6 +846,229 @@ Test orchestration:
845846

846847
- **Test impact analysis**`impact_analysis.build_index("./actions")` walks every action JSON file and projects locator names, URLs, template names, and `WR_*` commands into a reverse index; `affected_action_files(index, locators=["primary_cta"])` answers "which tests touch this?" so diff-aware shards can go beyond filename matching.
847848

849+
## Specialized Modules
850+
851+
A second wave of utility modules, each in its own subpackage under
852+
`je_web_runner/utils/`, organised by capability area. Each module is
853+
fully unit-tested and ships independent of the core executor (import
854+
only what you use).
855+
856+
### Web Platform APIs
857+
858+
- **`webtransport_assert`** — HTTP/3 WebTransport datagram + stream
859+
frame recorder with count / payload / JSON-shape / stream-complete
860+
assertions (mirror of `websocket_assert` and `sse_assert`).
861+
- **`indexed_db_explorer`** — Browser-side harvest JS + typed
862+
`IdbSnapshot`; assertions cover store existence, record count, key
863+
presence, index presence, plus per-store diff.
864+
- **`file_system_access`** — JS shim mocking `showOpenFilePicker` /
865+
`showSaveFilePicker` / `showDirectoryPicker`; records every write
866+
performed against the fake handle for later assertion.
867+
- **`notifications_audit`** — Tracks `Notification.requestPermission`
868+
call timing (user-gesture check, min page age) and policy violations
869+
(re-prompt after deny, notification spam after deny, tag reuse).
870+
- **`sse_assert`** — Server-Sent Events stream recorder + chunk-buffer
871+
feed + count / data-contains / JSON-shape / strictly-increasing-id
872+
assertions.
873+
- **`websocket_assert`** — WebSocket frame recorder + count / payload /
874+
pubsub-pattern / JSON-shape assertions.
875+
- **`webrtc_assert`**`PeerSnapshot.from_dict`, `aggregate_stats`
876+
(getStats), and connected / track-present / SDP-codec / packet-loss /
877+
min-bytes assertions.
878+
- **`view_transitions`** — Instrumentation snippet for the View
879+
Transitions API + duration budget / CLS budget / group-name asserts.
880+
881+
### Security & Headers
882+
883+
- **`mixed_content_audit`** — HAR + console-message scan for HTTP
884+
resources on HTTPS pages (active vs passive vs HSTS-upgrade).
885+
- **`clickjacking_audit`** — X-Frame-Options + `frame-ancestors` parser
886+
+ iframe-probe page generator; STRICT / SAMEORIGIN / ALLOWED / MISSING
887+
verdict.
888+
- **`open_redirect_detector`** — Eight-payload probe set (`//evil`,
889+
`@userinfo`, `javascript:`, `data:`, mixed-case bypass…) +
890+
classifier (BLOCKED / ALLOWED / AMBIGUOUS).
891+
- **`sri_verify`** — Parse `<script>` / `<link rel=stylesheet>` tags →
892+
validate `integrity=` strength + crossorigin requirement +
893+
recompute hash from a caller-supplied payload provider.
894+
- **`coop_coep_audit`**`crossOriginIsolated` page-header check (COOP
895+
`same-origin` + COEP `require-corp` / `credentialless`) + per-resource
896+
CORP / CORS validator.
897+
- **`token_leak_detector`** — Scan response bodies / HAR / log lines for
898+
leaked JWTs (with header validation), AWS / GitHub / Slack / Stripe /
899+
Google / generic-bearer tokens. Deduped by token suffix.
900+
- **`consent_audit`** — Cookie catalogue (GA / FB pixel / Hotjar /
901+
LinkedIn / Mixpanel / Stripe / Intercom / CSRF / session) + pre-consent
902+
+ post-reject reintroduction detector.
903+
- **`pii_in_screenshot`** — OCR + PII regex (Luhn-validated card, SSN,
904+
ROC ID, IBAN, IPv4, phone, email) over screenshots; reuses
905+
`ocr_assert` for the OCR layer.
906+
907+
### Performance Budgets
908+
909+
- **`inp_tracker`** — Interaction-to-Next-Paint instrumentation +
910+
p98-INP + good/needs-work/poor rating per Google's thresholds.
911+
- **`hydration_check`** — SSR hydration mismatch detection (DOM diff
912+
with framework-attr/comment stripping + console-marker scan covering
913+
React / Vue / Svelte / Astro / Nuxt).
914+
- **`bundle_budget`** — HAR → per-AssetKind transfer totals (script /
915+
stylesheet / image / font / media) + breach detail + biggest-asset
916+
ranking.
917+
- **`third_party_budget`** — Vendor catalogue (GA / FB Pixel / Hotjar /
918+
Intercom / Stripe / Segment / Mixpanel / Amplitude / Sentry / etc.) +
919+
req / byte / blocking-ms / vendor-count budgets.
920+
- **`long_animation_frame`**`long-animation-frame` PerformanceObserver
921+
listener + per-script attribution (forced reflow time, pause time).
922+
- **`console_error_budget`** — JS console / unhandled-rejection budget
923+
with regex ignore patterns; Selenium and CDP adapters.
924+
925+
### Backend Integration
926+
927+
- **`grpc_tester`** — gRPC stub method wrapper + gRPC-Web framing
928+
(length-prefix encode/decode + trailer parser) + status assertions.
929+
- **`webhook_receiver`** — Stdlib threaded HTTP server (random port) +
930+
`wait_for(predicate)` polling + path / header / JSON-predicate
931+
assertion helpers. Drop-in for "did the app POST a webhook?" tests.
932+
- **`idempotency_check`** — Run a request twice + compare
933+
status / body / state / side-effect count. `ignore_body_keys`,
934+
`allow_status_change_to` for legitimate 409-on-second.
935+
- **`pagination_audit`** — Walk all pages via caller-supplied fetcher;
936+
detects duplicates across pages, cursor-loop, off-by-one totals, and
937+
sort-order violations.
938+
- **`backend_log_correlator`** — W3C traceparent → fetch matching log
939+
lines from Loki / Elasticsearch / JSON-lines file → attach to a
940+
failure bundle.
941+
- **`email_render`** — MailHog / Mailpit / `.eml` capture →
942+
cross-viewport screenshot via pluggable render driver.
943+
944+
### AI / Workflow
945+
946+
- **`failure_narrator`** — Load failure-bundle directory → LLM-driven
947+
natural-language "why this failed" summary → strict JSON envelope →
948+
markdown report. LLM client pluggable.
949+
- **`repro_minimizer`** — Classic delta-debugging (ddmin) shrinking a
950+
failing action list to its smallest still-failing subsequence.
951+
- **`locator_hardener`** — Heuristic fragility score (nth-of-type /
952+
text-xpath / hashed-class / deep-descendant) → LLM-suggested stable
953+
selectors with safety filter on the response.
954+
- **`test_categorizer`** — Regex rules over action-name patterns → auto
955+
tag: smoke / regression / perf / a11y / security / payment /
956+
data_driven / visual / api.
957+
- **`exploratory_ai`** — Agentic exploratory tester with `PageObserver`
958+
+ `ActionPlanner` protocols; ships a deterministic `RandomPlanner` as
959+
fuzz fallback, collects `BugSignal`s from observed errors.
960+
- **`story_to_actions`** — LLM-driven translation of a user story +
961+
optional Figma frame hints into validated WR action JSON; validator
962+
rejects unsafe action names and bad locator strategies.
963+
- **`session_to_test`** — rrweb / generic-event-stream → WR action JSON;
964+
auto-detects input format.
965+
- **`test_auto_repair`** — LLM-driven test rewrite from a failure bundle
966+
+ git diff context.
967+
- **`edge_case_generator`** — LLM edge-case variant generator
968+
(complement to `mutation_testing`).
969+
- **`multimodal_qa`** — Send screenshot + question to a vision LLM,
970+
parse pass/fail/uncertain verdict with confidence floor; useful for
971+
UI "is this correct?" checks beyond pixel diff.
972+
- **`prompt_drift_monitor`** — Track an app-internal LLM feature's
973+
output drift via baseline embeddings + must_include / must_exclude
974+
lexical anchors.
975+
- **`test_dedup_ai`** — Structural (canonical fingerprint) + semantic
976+
(cosine clustering with pluggable embedder) dedupe of action JSON
977+
files.
978+
- **`walkthrough_docs`** — Generate step-by-step SOP / Confluence-style
979+
docs from recorded runs.
980+
981+
### a11y / i18n / Visual
982+
983+
- **`ocr_assert`** — OCR-based text assertion (`contains` / `fuzzy` /
984+
`any`) for canvas / WebGL / image content; whitespace + accent
985+
normalisation built in.
986+
- **`screen_reader_runner`** — Walk an accessibility tree to simulate
987+
NVDA / VoiceOver reading order + flag unnamed interactive elements,
988+
heading-level skips, missing alt, generic link text.
989+
- **`pseudo_localization`** — Pseudo-localise strings
990+
(`__éxámplé strîng__`) + scan rendered page for hard-coded text
991+
leaks. Preserves `{name}` / `%d` / `<tag>` placeholders.
992+
- **`forced_colors_mode`** — CDP-features builder for the four CSS
993+
media queries (color-scheme / reduced-motion / forced-colors /
994+
contrast) + computed-style diff with "became invisible" detection.
995+
- **`visual_ai`** — aHash / dHash / pHash + SSIM-proxy for canvas /
996+
chart visual diff.
997+
998+
### Governance & Reporting
999+
1000+
- **`pr_risk_score`** — Fuses flake / impact-analysis / locator-health /
1001+
coverage signals into a 0-100 PR risk score with markdown report and
1002+
is_blocking gate.
1003+
- **`flag_matrix`** — Feature-flag combination matrix with
1004+
forbid / require constraints, pinned baselines, deterministic
1005+
sampling, and greedy smallest-failing-subset cover.
1006+
- **`chaos_hooks`** — Seeded chaos injection (offline / throttle /
1007+
mid-flow reload / tab-background) with deterministic schedule per
1008+
action list.
1009+
- **`db_snapshot`** — Per-test DB savepoint/rollback isolation with
1010+
pluggable backend protocol; ships an `InMemoryBackend` for
1011+
unit-testing the workflow itself.
1012+
- **`time_freezer`** — CDP injection script that overrides
1013+
`Date` / `Date.now` / `performance.now`; freeze or slow-motion modes
1014+
for deterministic time-bound tests.
1015+
- **`persona_runner`** — Same suite × N personas (admin / free /
1016+
enterprise / guest) matrix; summary flags persona-specific vs
1017+
file-specific regressions.
1018+
- **`git_bisect_flake`** — Ledger-only or probe-driven bisect for the
1019+
regression commit that caused a test to start failing.
1020+
- **`test_cost_estimator`** — Per-runner rate-card (Sauce / BrowserStack /
1021+
LambdaTest / GitHub Actions) × ledger minutes → USD + CO₂ estimate
1022+
per suite / runner / test.
1023+
- **`slack_digest`** — Render Slack Block-Kit + Teams Adaptive Card +
1024+
plain-text test digest with quarantine activity, top-risk PRs, cost
1025+
trend, and pass-rate delta.
1026+
- **`quarantine_age_report`** — Add fresh / lingering / stale /
1027+
abandoned tier per quarantined test + escalation alerts.
1028+
- **`test_debt_dashboard`** — Scan pytest skip / xfail / TODO + JSON
1029+
`_skip` markers + age + CODEOWNERS-derived owner mapping.
1030+
- **`sla_tracker`** — Weekly / daily bucketed % of suites finishing
1031+
under an SLA duration threshold + trend.
1032+
- **`bug_repro_stability`** — Repeat a failing probe N times → classify
1033+
deterministic / flaky / non-reproducible + error-signature grouping +
1034+
longest pass / fail streak.
1035+
- **`test_owners_map`** — CODEOWNERS parser (last-match-wins glob
1036+
semantics) + per-test override layer + unowned-test audit.
1037+
- **`failure_triage`** — AI failure root-cause analysis on failure
1038+
bundles.
1039+
- **`flake_detector`** — Time-decayed flake scoring + quarantine
1040+
registry.
1041+
- **`locator_health`** — Project-wide locator audit + upgrade
1042+
suggestions.
1043+
- **`mutation_testing`** — Action JSON mutation testing (kill-rate /
1044+
score).
1045+
- **`live_dashboard`** — Aggregated web UI: runs + flake + quarantine +
1046+
locators.
1047+
- **`test_scheduler`** — Value-density scheduler under time + cloud
1048+
budget.
1049+
1050+
### Other Specialised Modules
1051+
1052+
- **`chrome_profile`** — Persistent Chrome profile + stealth +
1053+
snapshot / sync-back.
1054+
- **`device_cloud`** — Real-device cloud (BrowserStack / Sauce /
1055+
LambdaTest) connector.
1056+
- **`otel_bridge`** — W3C traceparent injection for distributed
1057+
tracing.
1058+
- **`otp_interceptor`** — MailHog / Mailpit / IMAP / SMS OTP polling
1059+
for 2FA flows.
1060+
- **`download_verify`** — PDF / CSV / Excel / JSON / SHA256 download
1061+
assertions.
1062+
- **`openapi_to_e2e`** — OpenAPI / Swagger spec → `WR_http_*` action
1063+
JSON generator.
1064+
- **`cross_tab_sync`** — Multi-page BroadcastChannel / storage
1065+
propagation asserts.
1066+
1067+
For per-module reference also see [`CLAUDE.md`](CLAUDE.md), the
1068+
auto-generated [`docs/reference/command_reference.md`](docs/reference/command_reference.md),
1069+
and the Sphinx chapter under
1070+
`docs/source/Eng/doc/specialized_modules/`.
1071+
8481072
## Advanced WebDriverWrapper
8491073

8501074
The Selenium wrapper is now composed via mixins under

README/README_zh-CN.md

Lines changed: 87 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -48,6 +48,7 @@ WebRunner(`je_web_runner`)是一款跨平台网页自动化框架,旨在
4848
- [日志记录](#日志记录)
4949
- [支持的浏览器](#支持的浏览器)
5050
- [支持的平台](#支持的平台)
51+
- [进阶模块](#进阶模块)
5152
- [许可证](#许可证)
5253

5354
## 核心功能
@@ -784,6 +785,92 @@ WebRunner 使用轮转式文件处理器记录日志。
784785
- Ubuntu / Linux
785786
- Raspberry Pi
786787

788+
## 进阶模块
789+
790+
除核心 WebDriver / 动作执行器外,WebRunner 还提供大量专用模块,每个
791+
模块都在 `je_web_runner/utils/<area>/` 下独立成包,并附完整单元测试。
792+
按能力分类:
793+
794+
### Web 平台 API
795+
796+
- **`webtransport_assert`** — HTTP/3 WebTransport datagram + stream 断言
797+
- **`indexed_db_explorer`** — IndexedDB 快照与对象 / 索引断言
798+
- **`file_system_access`** — 模拟 `showOpenFilePicker` / `showSaveFilePicker` 并记录写入
799+
- **`notifications_audit`** — 追踪 `Notification.requestPermission` 调用时机与策略
800+
- **`sse_assert`** — Server-Sent Events 流录制 + count / data / id 断言
801+
- **`websocket_assert`** — WebSocket frame 录制 + count / payload / pubsub 断言
802+
- **`webrtc_assert`** — PeerConnection 状态 / ICE / track / RTP 断言
803+
- **`view_transitions`** — View Transitions API duration / CLS / group 断言
804+
805+
### 安全 / Headers
806+
807+
- **`mixed_content_audit`** — HTTPS 页面中的 HTTP 资源检测
808+
- **`clickjacking_audit`** — X-Frame-Options + frame-ancestors + iframe 探测
809+
- **`open_redirect_detector`** — 八种 payload 检测 ?redirect= 类漏洞
810+
- **`sri_verify`** — Subresource Integrity hash 存在性 + 正确性验证
811+
- **`coop_coep_audit`** — crossOriginIsolated COOP / COEP + 子资源 CORP/CORS 检查
812+
- **`token_leak_detector`** — 扫描 HAR / log / response 中的 JWT / AWS / GitHub token 等
813+
- **`consent_audit`** — GDPR / CCPA cookie 分类 + 同意前泄漏 / 拒绝后重新植入检测
814+
- **`pii_in_screenshot`** — Screenshot OCR + 隐私 (信用卡 / 身份证 / SSN) 扫描
815+
816+
### 性能预算
817+
818+
- **`inp_tracker`** — Interaction to Next Paint 测量 + p98 + 预算
819+
- **`hydration_check`** — SSR hydration mismatch (DOM diff + console marker)
820+
- **`bundle_budget`** — 每种资源 (script/css/image/font/media) 大小预算
821+
- **`third_party_budget`** — 第三方厂商请求数 / 大小 / 阻塞时间预算
822+
- **`long_animation_frame`** — Long Animation Frame API + 每 script 归因
823+
- **`console_error_budget`** — JS console / unhandled rejection 预算
824+
825+
### 后端集成
826+
827+
- **`grpc_tester`** — gRPC stub 调用录制 + gRPC-Web framing / trailer
828+
- **`webhook_receiver`** — 临时 HTTP server 接收 app 对外 webhook + 断言
829+
- **`idempotency_check`** — 同一请求发送两次,比对 status / body / state / 副作用
830+
- **`pagination_audit`** — 遍历所有页,检测重复 / 漏抓 / cursor loop / 排序错误
831+
- **`backend_log_correlator`** — W3C trace_id → Loki / Elasticsearch / 文件 log 拉取
832+
- **`email_render`** — MailHog / Mailpit / `.eml` 拦截 → 多 viewport 截图
833+
834+
### AI / 工作流
835+
836+
- **`failure_narrator`** — LLM 将 failure bundle 转为自然语言失败摘要
837+
- **`repro_minimizer`** — Delta-debugging (ddmin) 缩减失败 action list
838+
- **`locator_hardener`** — Fragility 评分 + LLM 建议更稳定的 selector
839+
- **`test_categorizer`** — 自动标记 smoke / regression / perf / a11y / security
840+
- **`exploratory_ai`** — Agent 式探索测试员 + 确定性 RandomPlanner
841+
- **`story_to_actions`** — 用户故事 / Figma frame → 验证后的 WR action JSON
842+
- **`session_to_test`** — rrweb / 通用 event stream → WR action JSON
843+
- **`multimodal_qa`** — Screenshot + 问题送 vision LLM 取得 pass/fail 判定
844+
- **`prompt_drift_monitor`** — 追踪 app 内 LLM 功能的输出漂移
845+
- **`test_dedup_ai`** — 结构性 + 语义 embedding 去重 action JSON
846+
847+
### 无障碍 / 国际化 / 视觉
848+
849+
- **`ocr_assert`** — Canvas / WebGL / 图片内 OCR 文本断言
850+
- **`screen_reader_runner`** — 遍历 a11y tree 模拟 NVDA / VoiceOver 朗读顺序
851+
- **`pseudo_localization`** — 字符串伪本地化 + 检测硬编码字符串
852+
- **`forced_colors_mode`** — Dark / reduced-motion / forced-colors / 高对比矩阵
853+
854+
### 治理与报告
855+
856+
- **`pr_risk_score`** — 融合 flake / impact / locator / coverage 信号为 0-100 分
857+
- **`flag_matrix`** — Feature flag 组合矩阵 + 失败最小子集分析
858+
- **`chaos_hooks`** — 种子化 chaos 注入 (离线 / throttle / 中途 reload)
859+
- **`db_snapshot`** — 每个 test 自动 DB savepoint / rollback
860+
- **`time_freezer`** — CDP 注入脚本冻结 `Date` / `performance.now`
861+
- **`persona_runner`** — 同一份 suite × N 种角色矩阵
862+
- **`git_bisect_flake`** — Ledger / probe 两种模式 bisect 出回归 commit
863+
- **`test_cost_estimator`** — 云端分钟 × 费率 → USD + CO₂ 估算
864+
- **`slack_digest`** — Slack Block Kit / Teams Card / 纯文本测试周报
865+
- **`quarantine_age_report`** — Quarantine 条目 + age + 升级 tier
866+
- **`test_debt_dashboard`** — skip / xfail / TODO 盘点 + age + 负责人
867+
- **`sla_tracker`** — 「Y% suite 在 X 分钟内跑完」周 / 日趋势
868+
- **`bug_repro_stability`** — N 次重复 + 分类 deterministic / flaky / 无法重现
869+
- **`test_owners_map`** — CODEOWNERS 解析 + 覆盖层 + 无人认领审计
870+
871+
完整模块树见 [`CLAUDE.md`](../CLAUDE.md);完整命令清单见英文版
872+
[README.md](../README.md)`docs/source/Zh/doc/specialized_modules/`
873+
787874
## 许可证
788875

789876
本项目采用 [MIT 许可证](../LICENSE)

0 commit comments

Comments
 (0)