feature: Test tags (TestTag) for UniTest — run testing layers separately#617
Conversation
Lets a `test(...)` carry tags (e.g. "ui", "electron", "slow") so one suite can span
multiple testing layers and still be run layer-by-layer, mirroring VSCode's separate
unit/integration/UI test commands — the mechanism that makes Uni's multi-layer testing
practical in CI.
- `test(name, flaky, tags = Seq("ui"))` records tags on the `TestDef`.
- sbt args: `-tag:a,b` include filter (run tests with any listed tag), `-xtag:a,b`
exclude filter (skip tests with any listed tag); exclusion wins over inclusion.
- `TestConfig` parses the args and exposes `includesTags`; `UniTestTask` applies the
filter alongside the existing `-t:` name filter.
- Backward compatible: `tags` defaults to empty, existing `test(...)` / `test(..., flaky=true)`
call sites are unchanged.
Documented in .github/instructions/unitest.instructions.md. TagFilterTest covers parsing +
filter semantics + end-to-end selection; full uni-test suite green on JVM (65), compiles on
JS and Native.
Part of plans/2026-06-27-multi-layer-testing.md (PR 3 of the unit → UI → Electron → plugin
roadmap).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces test tagging and filtering capabilities to the uni-test framework, allowing users to run or skip specific testing layers (such as unit, UI, or integration tests) using the -tag: and -xtag: command-line options. The changes include updates to the configuration parser, task runner, documentation, and a new suite of unit tests. The feedback suggests trimming and filtering empty strings from the tags during registration to prevent potential issues with leading or trailing whitespace.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
| protected def test(name: String, flaky: Boolean = false, tags: Seq[String] = Nil)( | ||
| body: => Any | ||
| ): Unit = _tests += TestDef(name, () => body, _context, isFlaky = flaky, tags = tags.toSet) |
There was a problem hiding this comment.
To prevent subtle bugs where tests are silently skipped due to accidental leading/trailing whitespace or empty strings in the tags parameter (e.g., Seq(" ui ")), it is safer to trim and filter the tags when registering them, just like we do when parsing command-line arguments.
protected def test(name: String, flaky: Boolean = false, tags: Seq[String] = Nil)(
body: => Any
): Unit = _tests += TestDef(name, () => body, _context, isFlaky = flaky, tags = tags.map(_.trim).filter(_.nonEmpty).toSet)… / --exclude-tags) Renames the tag-filter CLI args from -tag:/-xtag: to --tags:/--exclude-tags:, following the convention of double-hyphen prefixes for long option names. Short options (-l, -t:) keep their single hyphen. Updates TestConfig parsing, docs, and TagFilterTest. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Unifies the `flaky: Boolean` and `tags: Seq[String]` parameters of `test(...)` into a single
`tags: TestTag*` varargs, with `Flaky` now a built-in `TestTag` rather than a special boolean.
This makes flaky-ness "just another tag" and reads more naturally:
test("renders", UI) // was: test("renders", tags = Seq("ui"))
test("retried", Flaky, Slow) // was: test("retried", flaky = true, tags = Seq("slow"))
- `TestTag` trait (`name`); `TestTag.Flaky` (failure -> skipped) + common layer tags
`UI`/`Electron`/`Integration`/`Slow`, all exported into `UniTest` scope.
- Custom tags via `TestTag("name")` or a bare String (given Conversion).
- `TestDef` stays string-based internally, so the runner / `--tags`/`--exclude-tags` filtering
is unchanged; `isFlaky` derives from the presence of the `Flaky` tag.
- Migrated the one `flaky = true` call site; expanded TagFilterTest; updated docs.
Breaking: `test(name, flaky = true)` / `test(name, tags = Seq(...))` become
`test(name, Flaky)` / `test(name, UI, ...)`. uni-test JVM suite green (69); JS + Native and all
JVM test modules compile.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Scala 3 traits take parameters, so `TestTag(val name: String)` carries the name directly and
the built-ins become one-line case objects (Flaky/UI/Electron/Integration/Slow). This removes
the separate `Named` case class; ad-hoc tags are anonymous instances via `TestTag("name")`.
Behavior is unchanged (filtering is by name); case objects keep clean toStrings.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Smoke tests (quick cross-stack sanity checks) are a standard category, so add `TestTag.Smoke` alongside the other built-ins and export it into UniTest scope. Also adds doc comments distinguishing Smoke (fast pre-merge subset) from Slow. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The custom-tag example used TestTag("smoke"), which is now a built-in. Show Smoke as a
built-in category tag, switch the custom example to TestTag("legacy"), and add a
--tags:smoke pre-merge subset command.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Switches the tag filters from the colon/comma form (--tags:ui,electron) to plain space-separated values that follow the existing -l two-token style, with no special characters: --tags <tag>, repeatable to add more (--tags ui --tags electron). Same for --exclude-tags. Empty/whitespace values are ignored; the comma-splitting helper is removed. Updates the parser, docstrings, instructions, and TagFilterTest. uni-test JVM suite green (70); JS + Native compile. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Changes the include filter from OR (any listed tag) to AND (all listed tags): each repeated `--tags` further narrows the selection, matching how GitHub issue label filters compose. A test runs iff it carries every `--tags` tag and none of the `--exclude-tags` tags. OR/union is intentionally not supported (YAGNI) — narrowing is the dominant need, and a future `--tags <expr>` could add explicit AND/OR expression syntax without breaking plain single-tag usage (noted in the parser). Single-tag behavior is unchanged. Updates includesTags (now `includeTags.subsetOf(tags)`), docstrings, instructions, and TagFilterTest (adds an AND-narrowing case). Verified end-to-end via the sbt runner (`--tags meta --tags self` -> 1 test; `--tags meta --tags nonexistent` -> 0). uni-test JVM suite green (71); JS + Native compile. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
… (wvlet.uni.plugin) (#618) ## Why Final layer of the multi-layer testing goal: make Uni a foundation for **apps built from pluggable units, testable in isolation** — VSCode's extension/plugin testing layer. Uni had no plugin/extension model; this adds a minimal one plus the harness to test plugins. Since `wvlet.uni.plugin` is a top-level namespace alongside `design`, `rx`, `surface` — and uni is not Electron-specific — the core model must be generic: it hardcodes **no** contribution kinds and has **no** dependency on `wvlet.uni.http.rpc`. See [`adr/2026-07-06-plugin-extension-points.md`](https://github.com/wvlet/uni/blob/feature/uni-plugin-model-and-test-harness/adr/2026-07-06-plugin-extension-points.md). ## What `wvlet.uni.plugin` — a VSCode-style extension model built on **typed extension points**, shipped in the published `uni` artifact (cross-platform: JVM/JS/Native): - **`Plugin`** — `id` + `activate(context)`, mirroring VSCode's `activate(context)`. - **`PluginContext`** — deliberately minimal registration surface: `contribute(point)(value)` + `onDeactivate(hook)`. - **`ExtensionPoint[A]`** — a typed contribution slot, compared by identity (define points as shared `val`s). `ExtensionPoint.keyed(name)(keyOf)` makes contributions unique by key: a duplicate key — within a plugin or across plugins — is rejected at activation, so conflicts surface in tests rather than at runtime. - **`PluginHost`** — activates plugins and owns the per-point registries: `contributions(point)`, `contribution(point, key)`, `deactivate()` (FILO hooks + reset). Re-activating the same plugin id is rejected. Contribution kinds are defined **next to their types**, so dependency arrows point *into* the plugin layer: - **`wvlet.uni.plugin.Command`** — `Command.point` (keyed by command id) + extension methods `registerCommand`, `executeCommand`, `commandIds`, `hasCommand`. - **`wvlet.uni.http.rpc.RPCPlugin`** — `routerPoint` + extension methods `registerRpcRouter`, `rpcRouters`, `rpcDispatcher` (an `RPCDispatcher` over all contributed routers, ready to serve via HTTP or Electron IPC). New kinds (views, menus, `Design` bindings) are added by defining a point — not by editing the core. ```scala import wvlet.uni.plugin.Command.* import wvlet.uni.http.rpc.RPCPlugin.* class NotesPlugin extends Plugin: def id = "notes" def activate(ctx: PluginContext): Unit = ctx.registerRpcRouter(RPCRouter.of[NotesApi](NotesApiImpl())) ctx.registerCommand("notes.new")(_ => createNote()) ctx.onDeactivate(() => flushToDisk()) ``` ## Test harness - **`wvlet.uni.plugin.testing.PluginTestHost`** — `activate(plugin)` / `activateAll(plugins*)` activate plugin(s) against a fresh host through the real activation path and return the host for assertions. Because the host is cross-platform, **plugin tests run on the JVM with no Electron/browser runtime**. ```scala val host = PluginTestHost.activate(NotesPlugin()) host.commandIds shouldContain "notes.new" host.executeCommand("notes.new") shouldBe expectedNote host.rpcRouters.flatMap(_.routes.map(_.path)).exists(_.endsWith("/list")) shouldBe true ``` ## Tests `PluginHostTest` (10 tests, JVM green; compiles on JS + Native): command registration/execution, unknown-command error, RPC-router contribution, app-defined extension points, unkeyed multi-contribution, lifecycle teardown, and conflict rejection (duplicate within a plugin, across plugins, and double-activation). ## Roadmap status unit → UI (#616) → Electron (#616) → tags (#617) → **plugin (this)**. Richer contribution points (views, menus, a typed command API, `Design` bindings), and a JS bridge from `RPCPlugin.rpcDispatcher` into `ElectronTestbed` for full end-to-end plugin-RPC tests, are natural follow-ups. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Why
Part of making Uni a foundation for multi-layer testing of rich desktop apps (see
plans/2026-06-27-multi-layer-testing.md). VSCode keeps separate test commands per layer (unit / integration / UI / smoke); to do the same here,UniTestneeds a way to mark which layer a test belongs to and run/skip one layer at a time. This is the mechanism that makes the other layers (UI toolkit, Electron harness in #616) runnable as independent suites in CI.What
A
TestTagvarargs API ontest(...)that unifies the oldflakyboolean and ad-hoc tags:trait TestTag { def name: String }; built-insFlaky,UI,Electron,Integration,Slowexported intoUniTestscope; custom viaTestTag("name")or aString(givenConversion).test(name: String, tags: TestTag*)(body)—isFlakyderives from the presence ofFlaky;TestDefstays string-based internally so the runner is unchanged.--prefix per repo convention; short options-l/-t:keep one hyphen):--tags:a,b— include filter: run only tests carrying any of these tags--exclude-tags:a,b— exclude filter: skip tests carrying any of these tags.github/instructions/unitest.instructions.md.Breaking change
test(name, flaky = true)→test(name, Flaky);test(name, tags = Seq("ui"))→test(name, UI)(orTestTag("ui")). Only one in-repo call site usedflaky = true; it's migrated. No other open branch uses the old params.Tests
TagFilterTest(12) covers arg parsing,includesTagssemantics,TestTagnames, theFlaky→isFlakymapping, layer-tag registration, and the String conversion. Verified end-to-end via the sbt runner (--tags:meta→ 1 test,--exclude-tags:meta→ rest). Full uni-test JVM suite green (69); JS + Native and every JVM test module compile.Roadmap: unit → UI (#616) → Electron (#616) → tags (this) → plugin (#618).
🤖 Generated with Claude Code