|
| 1 | +--- |
| 2 | +name: testing |
| 3 | +description: Use when writing or running end-to-end tests for a Shiny for Python (py-shiny) app - launching an app under pytest, locating and asserting on UI components (inputs, outputs, cards, navsets), simulating user interaction with Playwright, or when tempted to hand-roll CSS selectors, raw Playwright locators, or sleep() waits against Shiny-rendered HTML. |
| 4 | +compatibility: Requires the playwright, pytest, and pytest-playwright packages, plus Playwright browsers (`playwright install`). |
| 5 | +--- |
| 6 | + |
| 7 | +# Testing Shiny for Python apps with Playwright |
| 8 | + |
| 9 | +## Overview |
| 10 | + |
| 11 | +shiny ships typed **Playwright controllers** (`shiny.playwright.controller`) |
| 12 | +that know each component's DOM structure, plus pytest fixtures that launch the |
| 13 | +app in a subprocess. Use them instead of hand-written CSS selectors, raw |
| 14 | +locators, or `time.sleep()` — controller `expect_*` methods auto-wait and |
| 15 | +auto-retry. |
| 16 | + |
| 17 | +## Setup |
| 18 | + |
| 19 | +```bash |
| 20 | +pip install pytest playwright pytest-playwright |
| 21 | +playwright install chromium |
| 22 | +``` |
| 23 | + |
| 24 | +Installing shiny registers a pytest plugin, so the `local_app` fixture below |
| 25 | +is available without any conftest. |
| 26 | + |
| 27 | +## Write a test |
| 28 | + |
| 29 | +With `app.py` and the test file in the same directory: |
| 30 | + |
| 31 | +```python |
| 32 | +# app.py |
| 33 | +from shiny import App, Inputs, Outputs, Session, render, ui |
| 34 | + |
| 35 | +app_ui = ui.page_fluid( |
| 36 | + ui.input_slider("n", "N", 0, 100, 20), |
| 37 | + ui.output_text_verbatim("txt"), |
| 38 | +) |
| 39 | + |
| 40 | +def server(input: Inputs, output: Outputs, session: Session): |
| 41 | + @render.text |
| 42 | + def txt(): |
| 43 | + return f"n*2 is {input.n() * 2}" |
| 44 | + |
| 45 | +app = App(app_ui, server) |
| 46 | +``` |
| 47 | + |
| 48 | +```python |
| 49 | +# test_app.py |
| 50 | +from playwright.sync_api import Page |
| 51 | + |
| 52 | +from shiny.playwright import controller |
| 53 | +from shiny.run import ShinyAppProc |
| 54 | + |
| 55 | +def test_doubles_slider_value(page: Page, local_app: ShinyAppProc) -> None: |
| 56 | + page.goto(local_app.url) |
| 57 | + |
| 58 | + slider = controller.InputSlider(page, "n") |
| 59 | + txt = controller.OutputTextVerbatim(page, "txt") |
| 60 | + |
| 61 | + slider.expect_value("20") |
| 62 | + txt.expect_value("n*2 is 40") |
| 63 | + |
| 64 | + slider.set("50") |
| 65 | + txt.expect_value("n*2 is 100") |
| 66 | +``` |
| 67 | + |
| 68 | +Run with `pytest test_app.py`. The `page` fixture comes from |
| 69 | +pytest-playwright; `local_app` starts `app.py` from the test file's directory |
| 70 | +(in test mode, `SHINY_TESTMODE=1`) and exposes the server's URL as `.url`. |
| 71 | + |
| 72 | +The pattern is always: construct a controller with the component's **id**, |
| 73 | +drive it with `.set()` / `.click()`, and assert with `.expect_*()` methods. |
| 74 | + |
| 75 | +## Test an app in another location |
| 76 | + |
| 77 | +```python |
| 78 | +from shiny.pytest import create_app_fixture |
| 79 | + |
| 80 | +# Path is relative to this test file. A list of paths parametrizes the tests. |
| 81 | +app = create_app_fixture("../apps/dashboard/app.py") |
| 82 | + |
| 83 | +def test_dashboard(page: Page, app: ShinyAppProc) -> None: |
| 84 | + page.goto(app.url) |
| 85 | + ... |
| 86 | +``` |
| 87 | + |
| 88 | +`create_app_fixture(app, scope="module", timeout_secs=30, env=None)` — `env` |
| 89 | +is merged over the parent environment and defaults to |
| 90 | +`{"SHINY_TESTMODE": "1"}`. |
| 91 | + |
| 92 | +## Find the right controller |
| 93 | + |
| 94 | +Controller names follow the UI function: `ui.input_slider` → |
| 95 | +`controller.InputSlider`, `ui.output_data_frame` → |
| 96 | +`controller.OutputDataFrame`, `ui.card` → `controller.Card`. |
| 97 | + |
| 98 | +| Category | Controllers | |
| 99 | +|---|---| |
| 100 | +| Inputs | `InputActionButton`, `InputActionLink`, `InputBookmarkButton`, `InputCheckbox`, `InputCheckboxGroup`, `InputCodeEditor`, `InputDarkMode`, `InputDate`, `InputDateRange`, `InputFile`, `InputNumeric`, `InputPassword`, `InputRadioButtons`, `InputSelect`, `InputSelectize`, `InputSlider`, `InputSliderRange`, `InputSubmitTextarea`, `InputSwitch`, `InputTaskButton`, `InputText`, `InputTextArea` | |
| 101 | +| Outputs | `OutputCode`, `OutputDataFrame`, `OutputImage`, `OutputPlot`, `OutputTable`, `OutputText`, `OutputTextVerbatim`, `OutputUi` | |
| 102 | +| Downloads | `DownloadButton`, `DownloadLink` | |
| 103 | +| Containers | `Accordion`, `AccordionPanel`, `Card`, `ValueBox`, `Sidebar`, `Offcanvas`, `Popover`, `Tooltip`, `Toast`, `Chat`, `ToolbarInputButton`, `ToolbarInputSelect` | |
| 104 | +| Navigation | `NavPanel`, `NavsetBar`, `NavsetCardPill`, `NavsetCardTab`, `NavsetCardUnderline`, `NavsetHidden`, `NavsetPill`, `NavsetPillList`, `NavsetTab`, `NavsetUnderline`, `PageNavbar` | |
| 105 | +| Server state | `AppTestValues` | |
| 106 | + |
| 107 | +## Client-side vs server-side values |
| 108 | + |
| 109 | +Component controllers and `AppTestValues` read from different places, and the |
| 110 | +"same" value can legitimately differ between them: |
| 111 | + |
| 112 | +- **Controllers read the browser DOM** — what the user sees. `expect_value()` / |
| 113 | + `get_value()` inspect the rendered widget, so values are strings formatted |
| 114 | + for display: a slider reads `"50"` (or `"1,000"` with a thousands separator), |
| 115 | + a date input reads the displayed date text, an output reads its rendered |
| 116 | + text/HTML. |
| 117 | +- **`controller.AppTestValues(page)` reads the server's snapshot** — the |
| 118 | + Python values the server function currently holds: `input.n()` as the int |
| 119 | + `50`, an output as the render function's return value, plus `export` values |
| 120 | + (internal reactives surfaced with `export_test_values()`) that have no DOM |
| 121 | + presence at all. |
| 122 | + |
| 123 | +They can also differ in *time*: the DOM updates instantly when the user |
| 124 | +interacts, but the server only sees the change after the websocket round-trip |
| 125 | +(and any debounce/throttle), so a client-side value can briefly be ahead of |
| 126 | +its server-side counterpart. |
| 127 | + |
| 128 | +Prefer controllers for user-visible behavior; reach for `AppTestValues` when |
| 129 | +asserting server state: |
| 130 | + |
| 131 | +```python |
| 132 | +app_values = controller.AppTestValues(page) |
| 133 | +app_values.expect_input("n", 50) # server-side: int, not "50" |
| 134 | +app_values.expect_export("doubled", 100) # internal reactive, no DOM |
| 135 | +``` |
| 136 | + |
| 137 | +See the `debugging` skill for the full test-mode snapshot API. |
| 138 | + |
| 139 | +## Debug a failing test |
| 140 | + |
| 141 | +```bash |
| 142 | +pytest test_app.py --headed # watch the browser |
| 143 | +pytest test_app.py --tracing retain-on-failure # then: playwright show-trace <trace.zip> |
| 144 | +``` |
| 145 | + |
| 146 | +## Generate a test with AI |
| 147 | + |
| 148 | +`shiny add test --app app.py` analyzes an app and writes a test file for it |
| 149 | +(requires `ANTHROPIC_API_KEY`, or `OPENAI_API_KEY` with `--provider openai`). |
| 150 | + |
| 151 | +## Common mistakes |
| 152 | + |
| 153 | +- Test times out with a blank page → you forgot `page.goto(local_app.url)`. |
| 154 | +- Flaky assertions → replace `time.sleep()` + plain `assert` with the |
| 155 | + controller's auto-waiting `expect_*()` methods. |
| 156 | +- Controller "not found" for a component inside a module → ids are |
| 157 | + namespaced: `controller.InputText(page, "mymodule-txt")`. |
| 158 | +- `local_app` fails to start → it only launches a file named `app.py` next to |
| 159 | + the test file; for any other name or location use `create_app_fixture()`. |
| 160 | +- `ImportError` from `shiny.playwright` → install `playwright` (and |
| 161 | + `pytest-playwright` when running under pytest). |
0 commit comments