Merge pull request #202 from Integration-Automation/dev

Merge dev: QA/test framework layer + USB passthrough subsystem

Author: JE-Chen

README.md

@@ -363,11 +363,42 @@ over a list while the body sees the current item.
        }],
    ])
 
-Comparison operators for ``AC_if_var``: ``eq``, ``ne``, ``lt``, ``le``,
-``gt``, ``ge``, ``contains``, ``startswith``, ``endswith``.
+Comparison operators for ``AC_if_var`` (and ``AC_while_var``): ``eq``,
+``ne``, ``lt``, ``le``, ``gt``, ``ge``, ``contains``, ``startswith``,
+``endswith``.
+
+``AC_while_var`` loops a ``body`` while a variable comparison holds. The
+condition is re-evaluated against the live scope before every iteration,
+so a body that mutates the variable (e.g. ``AC_inc_var``) terminates the
+loop; ``max_iter`` (default 1000) caps a condition that never turns
+false. ``AC_break`` / ``AC_continue`` work as in any loop::
+
+   executor.execute_action([
+       ["AC_set_var", {"name": "i", "value": 0}],
+       ["AC_while_var", {
+           "name": "i", "op": "lt", "value": 5,
+           "body": [["AC_inc_var", {"name": "i"}]],
+       }],
+   ])
+
+``AC_try`` adds try / catch / finally. When ``body`` raises, the
+``catch`` branch runs instead of aborting the script; ``finally`` always
+runs (on success, on a caught error, or while a ``reraise`` / loop
+break/continue propagates). The error text is exposed to ``error_var``
+for the ``catch`` branch to inspect, and ``reraise=true`` re-raises after
+cleanup::
+
+   executor.execute_action([
+       ["AC_try", {
+           "body": [["AC_click_image", {"image": "dialog_ok.png"}]],
+           "catch": [["AC_set_var", {"name": "dismissed", "value": False}]],
+           "finally": [["AC_screenshot", {"file_path": "after.png"}]],
+           "error_var": "err",
+       }],
+   ])
 
 Action-JSON commands: ``AC_set_var``, ``AC_get_var``, ``AC_inc_var``,
-``AC_if_var``, ``AC_for_each``.
+``AC_if_var``, ``AC_for_each``, ``AC_while_var``, ``AC_try``.
 
 GUI: **Variables** tab — live view of ``executor.variables`` with
 single-set, JSON seed, and clear-all controls; reflects what

README/README_zh-CN.md

@@ -0,0 +1,318 @@
+==================================
+New Features (2026-06) — QA Layer
+==================================
+
+Nine additions that turn AutoControl's automation primitives into a
+full **test framework**: assert screen state, drive scripts from data,
+detect and quarantine flaky tests, run a scored suite, emit CI-native
+reports, audit accessibility / i18n, fan a script across a device
+matrix, and assert on audio / video. Every feature ships with a
+headless Python API, an ``AC_*`` executor command, an ``ac_*`` MCP
+tool, and a Qt GUI tab — same pattern as the rest of the framework.
+
+.. contents::
+   :local:
+   :depth: 2
+
+
+Assertions
+==========
+
+Assertion DSL
+-------------
+
+Verify the screen state instead of only driving it. Each ``assert_*``
+observes the current state, returns an :class:`AssertionResult`, and
+(by default) raises ``AutoControlAssertionException`` on mismatch so a
+script / test / scheduled run fails loudly at the broken assumption::
+
+    from je_auto_control import (
+        assert_text, assert_image, assert_pixel, assert_window,
+    )
+
+    assert_text("Login successful", region=[0, 0, 800, 200])
+    assert_image("checkmark.png", threshold=0.9)
+    assert_pixel(100, 200, [0, 200, 0], tolerance=10)
+    assert_window("Settings", exists=True)
+
+``assert_text`` accepts ``regex=True`` and ``present=False`` (assert
+*absence*); every helper takes ``raise_on_fail`` and ``capture_on_fail``
+(saves a screenshot of the failing screen under
+``~/.je_auto_control/assertions/``).
+
+Executor: ``AC_assert_text / _image / _pixel / _window``.
+MCP: ``ac_assert_*``. GUI: **Assertions** tab.
+
+
+Off-screen and system assertions
+---------------------------------
+
+The DSL also verifies state that is not on the screen::
+
+    from je_auto_control import (
+        assert_clipboard, assert_process, assert_file, assert_http,
+    )
+
+    assert_clipboard("ORDER-12345", mode="contains")
+    assert_process("chrome", running=True)
+    assert_file("export.csv", min_size=1, contains="total")
+    assert_http("https://localhost:8080/health", status=200)
+
+* ``assert_clipboard`` — clipboard text by ``equals`` / ``contains`` /
+  ``regex``; ``present=False`` confirms a secret was *cleared*.
+* ``assert_process`` — a process whose name contains the argument is (or
+  is not) running, via ``psutil``.
+* ``assert_file`` — existence / substring / SHA-256 / minimum size of a
+  file; the path is ``realpath``-normalised before any I/O. Verifies a
+  download or export.
+* ``assert_http`` — an ``http``/``https`` endpoint returns a status code
+  (and optional body substring), always with an explicit ``timeout``.
+  Only ``http``/``https`` schemes are accepted; an unreachable host is a
+  failed assertion, not a crash.
+
+Executor: ``AC_assert_clipboard / _process / _file / _http``.
+MCP: ``ac_assert_clipboard / ac_assert_process / ac_assert_file /
+ac_assert_http``.
+
+
+Assertion combinators (group / OR / poll)
+-----------------------------------------
+
+Compose the eight assertion kinds with declarative *specs* — plain dicts
+like ``{"kind": "text", "text": "Saved"}`` — so the same checks are
+reachable from Python, JSON, and MCP without passing callables::
+
+    from je_auto_control import assert_all, assert_any, assert_eventually
+
+    # soft assertions: run the whole batch, collect every failure
+    assert_all([
+        {"kind": "window", "title": "Dashboard"},
+        {"kind": "text", "text": "Welcome"},
+    ])
+
+    # OR: pass when at least one spec passes (short-circuits)
+    assert_any([
+        {"kind": "text", "text": "Success"},
+        {"kind": "window", "title": "Redirecting"},
+    ])
+
+    # poll any spec until it passes or times out
+    assert_eventually({"kind": "http", "url": "http://localhost:8080/health"},
+                      timeout=30, interval=0.5)
+
+``assert_all`` (AND) never short-circuits and returns a
+:class:`GroupAssertionResult` summarising every sub-result;
+``assert_any`` (OR) stops at the first pass; ``assert_eventually``
+re-checks one spec on an interval until it holds — ideal for waiting on a
+service to come up or a download file to appear.
+
+Executor: ``AC_assert_all / AC_assert_any / AC_assert_eventually``.
+MCP: ``ac_assert_all / ac_assert_any / ac_assert_eventually``.
+
+
+Media assertions (audio / video)
+--------------------------------
+
+Assert that something actually *played* or *animated*::
+
+    from je_auto_control import assert_audio_activity, assert_video_changes
+
+    assert_audio_activity(duration_s=1.0, threshold=0.01, expect_sound=True)
+    assert_video_changes("clip.mp4", start_s=0, end_s=3, expect_motion=True)
+
+``assert_audio_activity`` records from an input device and compares the
+RMS level to a threshold (sound vs silence). ``assert_video_changes``
+measures mean frame-to-frame difference over a video segment (motion vs
+static), with an optional ``region`` crop. The numeric cores
+(``rms``, ``mean_frame_diff``, ``measure_audio_rms``,
+``video_segment_motion``) are public and pure. ``sounddevice`` /
+OpenCV are lazy dependencies.
+
+Executor: ``AC_assert_audio / AC_assert_video_changes``.
+MCP: ``ac_assert_audio / ac_assert_video_changes``. GUI: **Media
+Checks** tab.
+
+
+Data-driven execution
+=====================
+
+Feed rows from CSV / JSON / SQLite / Excel / inline literals into a
+``${var}`` script, then run the same body once per row::
+
+    from je_auto_control import load_rows
+
+    rows = load_rows({"kind": "csv", "path": "users.csv"})
+
+In a JSON action file the new ``AC_for_each_row`` block command loads a
+data source and binds each row to a variable whose columns are
+addressable as ``${row.column}``::
+
+    ["AC_for_each_row", {
+        "source": {"kind": "csv", "path": "users.csv"},
+        "as": "row",
+        "body": [
+            ["AC_type_keyboard", {"keys": "${row.username}"}],
+            ["AC_assert_text", {"text": "${row.expected}"}]
+        ]
+    }]
+
+The SQLite connector accepts a **single read-only** ``SELECT`` / ``WITH``
+statement only (multi-statement / write queries are rejected); all file
+paths are ``realpath``-validated. ``${var}`` interpolation now resolves
+dotted paths into dict keys and list indices (``${row.user}``,
+``${results.0}``) while preserving value types.
+
+Executor: ``AC_load_data`` + ``AC_for_each_row``.
+MCP: ``ac_load_data``. GUI: **Data Sources** tab.
+
+
+Flaky-test detection & quarantine
+==================================
+
+Flaky report
+------------
+
+Score intermittent failures from the SQLite run-history store. Runs are
+grouped by ``script_path`` (or ``source_id``); the report counts
+pass/fail outcomes and pass↔fail *flips* in chronological order so a
+flaky script ranks above one that is consistently green or red::
+
+    from je_auto_control import analyze_flakiness
+
+    report = analyze_flakiness(min_runs=3)
+    for entry in report.entries:
+        print(entry.key, entry.flip_rate, entry.flaky)
+
+Executor: ``AC_flaky_report``. MCP: ``ac_flaky_report``.
+GUI: **Flaky Tests** tab.
+
+
+Quarantine (closing the loop)
+-----------------------------
+
+A quarantined case name is *skipped* by the suite runner (recorded as
+``skipped`` with reason ``quarantined``) so a known-flaky case stops
+poisoning the suite's red/green status until it is fixed. The store is a
+small JSON file (mode 0600 on POSIX) that persists across restarts::
+
+    from je_auto_control import (
+        default_quarantine_store, auto_quarantine_from_flakiness,
+    )
+
+    default_quarantine_store().add("login_suite", reason="under triage")
+    auto_quarantine_from_flakiness(flip_rate_threshold=0.5)
+
+``auto_quarantine_from_flakiness`` reads the flakiness report and
+quarantines every group above the flip-rate threshold.
+
+Executor: ``AC_quarantine_add / _remove / _list / _clear / _auto``.
+MCP: ``ac_quarantine_*``. GUI: quarantine panel on the **Test Suites**
+tab.
+
+
+QA suite runner + CI reports
+============================
+
+Suite orchestration
+-------------------
+
+Turn flat action lists into scored test cases with setup / teardown,
+tags, and per-case pass/fail. A case carrying a ``data`` source expands
+to one scored case per row::
+
+    from je_auto_control import run_suite
+
+    spec = {
+        "name": "Login",
+        "setup":    [["AC_focus_window", {"title": "MyApp"}]],
+        "teardown": [["AC_close_window", {"title": "MyApp"}]],
+        "cases": [
+            {"name": "valid login", "tags": ["smoke"],
+             "actions": [["AC_assert_text", {"text": "Welcome"}]]},
+            {"name": "each user", "as": "row",
+             "data": {"kind": "csv", "path": "users.csv"},
+             "actions": [["AC_assert_text", {"text": "${row.expected}"}]]},
+        ],
+    }
+    result = run_suite(spec, tags=["smoke"])
+    print(result.passed, result.failed, result.errored, result.skipped)
+
+An ``AutoControlAssertionException`` marks a case **failed**; any other
+exception marks it **error**; a clean run is **passed**. Quarantined
+case names are recorded as **skipped**.
+
+Executor: ``AC_run_suite``. MCP: ``ac_run_suite``.
+GUI: **Test Suites** tab.
+
+
+CI-native reports (JUnit / Allure)
+----------------------------------
+
+Emit reports that Jenkins, GitHub Actions, GitLab CI, and Allure parse
+natively::
+
+    from je_auto_control import write_junit_xml, write_allure_results
+
+    write_junit_xml(result, "reports/junit.xml")
+    write_allure_results(result, "reports/allure")
+
+``AC_run_suite`` writes them inline when given ``junit_path`` /
+``allure_dir``::
+
+    ["AC_run_suite", {"spec": {...}, "junit_path": "reports/junit.xml"}]
+
+Only report *generation* happens here (never parsing untrusted XML), so
+the stdlib ``xml.etree.ElementTree`` writer is safe.
+
+
+Accessibility & i18n audit
+==========================
+
+Reuse the accessibility tree and OCR layer to *inspect* a UI for common
+accessibility / localisation defects rather than to drive it::
+
+    from je_auto_control import run_audit, contrast_ratio
+
+    report = run_audit(
+        app_name="MyApp",
+        contrast_pairs=[{"foreground": [120, 120, 120],
+                         "background": [255, 255, 255], "label": "hint"}],
+        texts=["Save chang…"],   # OCR strings to scan for truncation
+    )
+
+Checks:
+
+* **Missing labels** — interactive widgets (button, menu item, link,
+  field …) exposed through the a11y tree with no accessible name.
+* **Contrast** — WCAG 2.x relative-luminance contrast ratio with AA /
+  AAA thresholds (``contrast_ratio([0,0,0],[255,255,255]) == 21.0``).
+* **Truncation** — OCR strings ending in an ellipsis (clipped after
+  translation).
+
+Executor: ``AC_audit_accessibility / AC_audit_contrast``.
+MCP: ``ac_audit_*``. GUI: **A11y Audit** tab.
+
+
+Mobile device matrix
+====================
+
+Fan a single action list out across many Android / iOS devices **in
+parallel**, each on its own isolated executor (so runtime variable
+scopes never collide between threads). The script targets the current
+device through a bound ``${device.*}`` variable::
+
+    from je_auto_control import run_on_devices
+
+    report = run_on_devices(
+        actions=[["AC_android_tap", {"x": 100, "y": 200,
+                                     "serial": "${device.serial}"}]],
+        devices=[{"platform": "android", "serial": "emulator-5554"},
+                 {"platform": "android", "serial": "emulator-5556"}],
+        max_parallel=4,
+    )
+    print(report.passed, report.failed)
+
+A failure on one device is isolated — it never aborts the others.
+
+Executor: ``AC_run_device_matrix``. MCP: ``ac_run_device_matrix``.
+GUI: **Device Matrix** tab.