feat(BA-1982): add traefik app-proxy frontend mode to the TUI installer by Yaminyam · Pull Request #12749 · lablup/backend.ai

Yaminyam · 2026-07-10T05:30:20Z

Resolves #5319 (BA-1982)

Summary

Automates the manual traefik-mode App Proxy setup (docs/app-proxy/traefik-integration.md) in the TUI installer:

./backend.ai install --frontend-mode=traefik

Traefik runs as a halfstack compose service gated behind the traefik compose profile (same pattern as the observability/storage profiles), so default installs are untouched. Supersedes #12744 (same feature originally targeted at install-dev.sh, which is planned for deprecation).

Changes

File	Change
`docker-compose.halfstack-main.yml`	New `backendai-half-traefik` service (`traefik:v3.7`, `profiles: ["traefik"]`): API entrypoint published at `8080` (the worker reconciles routes against `http://127.0.0.1:{api_port}`), `10205-10300` portproxy entrypoints, healthcheck via `traefik healthcheck --ping`
`configs/traefik/traefik.halfstack.yml`	Completed with `experimental.localPlugins` (appproxy Traefik plugin wasm + go) and `ping: {}`; marker-directory placeholder substituted by the installer
`src/ai/backend/install/configs/traefik.yml`	New resource symlink (same pattern as the other bundled configs)
`src/ai/backend/install/types.py`	`FrontendMode.TRAEFIK` (CLI choices derive from the enum) + `ServiceConfig.frontend_mode` for the install report
`src/ai/backend/install/context.py`	`prepare_traefik_assets()` before `compose up` (static config copy + marker path substitution + plugin download with `gh` CLI fallback while the plugin repo is private); `--profile traefik`; `configure_appproxy` traefik branches for the coordinator (`enable_traefik` + `[proxy_coordinator.traefik.etcd]`, namespace `traefik`) and the HTTP worker (`[proxy_worker.traefik]`, port sub-mode, `port_range = [10205, 10300]`)
`src/ai/backend/install/app.py`	Traefik-specific instructions in the install report (App-Proxy Workers tab)
`src/ai/backend/install/cli.py`	Help text

Design notes

Marker directory: bind-mounted into the container at the same absolute path as on the host, because the coordinator hands the worker's marker path verbatim to the Traefik plugin (circuit.py → lastusedmarkerpath).
TCP worker keeps the port-based frontend: Traefik's etcd rootKey is per worker authority, and the bundled static config serves the HTTP worker (worker-1). Extending to the TCP worker needs a second rootKey/entrypoint set — left out of scope.
Both DevContext and PackageContext run the same install_halfstack/configure_appproxy paths, so both install modes are covered.
The plugin repository is currently private: anonymous curl 404s, so the installer falls back to gh release download (developer credentials); when the repo goes public the curl path takes over.

Verification

The exact tomlkit mutations were validated against the actual appproxy pydantic config models — including that enable_traefik lands inside [proxy_coordinator] (not a subtable) and port_range uses the tuple form (see fix: render tuple-typed config examples as TOML arrays in generated samples #12747 for the sample.toml bug this uncovered).
docker compose config valid with and without the traefik profile.
Isolated compose run (etcd + traefik with these exact assets): both plugins load, /ping + dashboard 200, healthcheck healthy, and a router written under traefik/worker_worker-1/... in etcd appears via the etcd provider.
pants fmt lint check clean on all touched Python files.
Not yet exercised: a full backend.ai install --frontend-mode=traefik run with a live session app end-to-end; reviewer validation welcome.

🤖 Generated with Claude Code

Automate the manual traefik-mode App Proxy setup (docs/app-proxy/traefik-integration.md) behind the existing installer option, now accepting `--frontend-mode=traefik`: - Run Traefik as a halfstack compose service (`backendai-half-traefik`, gated behind the "traefik" compose profile) with the API entrypoint published at 127.0.0.1:8080 (the worker reconciles routes against it) and the portproxy_10205..10300 entrypoints exposed. - Complete configs/traefik/traefik.halfstack.yml with the `experimental.localPlugins` definition for the appproxy Traefik plugin and a `ping` endpoint used by the container healthcheck; bundle it into the installer resources (configs/traefik.yml symlink). - installer: add FrontendMode.TRAEFIK (the CLI choice list derives from the enum), prepare the Traefik assets before `compose up` (static config copy with the marker-directory substitution, plugin download with a `gh` fallback while the plugin repository is private), append `--profile traefik`, and generate the coordinator/worker traefik configuration in configure_appproxy (`enable_traefik` + `[proxy_coordinator.traefik.etcd]` with namespace "traefik"; worker `frontend_mode = "traefik"` + `[proxy_worker.traefik]` with the port sub-mode). The TCP worker keeps the port-based frontend (Traefik's etcd rootKey is per worker authority). - The last-used-time marker directory is bind-mounted into the container at the same absolute path as on the host because the coordinator hands the worker's marker path verbatim to the Traefik plugin. - Record the chosen frontend mode in ServiceConfig and show traefik-specific instructions in the install report. Verified: the exact tomlkit mutations validate against the actual appproxy pydantic config models (including `enable_traefik` staying inside [proxy_coordinator]); `docker compose config` is valid with and without the profile; an isolated compose run of etcd+traefik with these assets shows both plugins loading and an etcd-written router appearing via the etcd provider. Fixes #5319 Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Copilot

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

seedspirit

Overall, the comments are a bit verbose, so please just leave the key points.

seedspirit · 2026-07-13T05:01:25Z

+    # Enabled only via `install-dev.sh --app-proxy-frontend=traefik`
+    # (docker compose --profile traefik). The static configuration and the
+    # appproxy traefik plugin are prepared by install-dev.sh before `up`.
+    image: traefik:v3.7


Is there a reason why it was set to 3.7?

This is because 3.7 is the latest minor version among the current stable versions.

It was the latest stable minor line at implementation time (the plugin only requires Traefik v3.0+ with WASM support), and I verified it empirically with an isolated compose run (both appproxy plugins load, etcd-provider routing works). The v3.7 minor tag keeps patch updates while avoiding an accidental jump to v3.8/v4.

For reference, the pyinfra fleet installer pins v3.3.7 and one internal cluster runs v3.4.4 in production with the same plugin — so if we prefer aligning the dev halfstack with the production-proven line, I'm happy to change the pin to v3.4 (or v3.3). Let me know which you'd prefer.

The latest working version is best. Version 3.7 seems like the best choice.

Keep only the key points in the traefik-related comments (compose service, static config header, installer constants/docstrings). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

…n download The plugin repository went public, so the anonymous curl path is sufficient. Also fail the install loudly when the download/extract exits non-zero instead of continuing silently. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Yaminyam · 2026-07-21T00:48:37Z

The plugin repository (lablup/backend.ai-appproxy-worker-traefik) is now public, which unblocks this PR's install-time download path:

Verified the exact command the installer runs works anonymously: curl -fsSL <release-url> | tar xz → exit 0, extracts the expected plugins-local/src/backend.ai/... layout (wasm + go plugin, .traefik.yml manifests).
Pushed b83e5f1 dropping the now-unneeded gh CLI fallback (it existed only for the private-repo period) and making a failed download abort the install with a clear error instead of continuing silently.

Ready for review — the earlier v3.7 pin discussion is resolved above.

… flags `--frontend-mode traefik` conflated two independent axes: the traffic pattern (port vs wildcard) and the dataplane implementation (builtin vs Traefik). The appproxy worker config supports the full matrix (Traefik has its own port/wildcard sub-modes), and wildcard installs (--fqdn-prefix public setups) previously could not use the Traefik backend at all. - Add `--proxy-backend builtin|traefik`, composing with `--frontend-mode port|wildcard`. `--frontend-mode traefik` stays as a deprecated alias normalized to (port, traefik) at CLI parsing time. - Implement the Traefik wildcard sub-mode: the worker gets `[proxy_worker.traefik]` with `frontend_mode = "wildcard"` and a `wildcard_domain` table, and the installer rewrites the copied Traefik static config to swap the portproxy_* entrypoints for the fixed `domainproxy` entrypoint the coordinator emits wildcard routers on (portproxy_10250 would collide with it on :10250). - Split the compose port publish so :10250 serves either sub-mode. - The normalization also fixes a latent bug where the TCP worker config got `frontend_mode = "traefik"` without a traefik table (the TCP worker always keeps the builtin frontend). Verified: generated traefik tables (both sub-modes) validate against the real appproxy `TraefikConfig` model; the entrypoint rewrite preserves the plugin and etcd-provider config; `docker compose --profile traefik config` passes; pants fmt/lint/check green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Yaminyam · 2026-07-21T04:15:57Z

Design update in 52a5449: split the traffic pattern and the dataplane backend into orthogonal flags.

--frontend-mode traefik conflated two independent axes, which made the wildcard traffic pattern (used by --fqdn-prefix public setups) mutually exclusive with the Traefik backend — even though the appproxy worker config supports Traefik's own port/wildcard sub-modes.

New: --proxy-backend builtin|traefik × --frontend-mode port|wildcard (full matrix). --frontend-mode traefik remains as a deprecated alias for (port, traefik).
Traefik wildcard sub-mode implemented: worker gets [proxy_worker.traefik] frontend_mode="wildcard" + wildcard_domain table; the installer swaps the portproxy_* entrypoints for the fixed domainproxy entrypoint (:10250) that the coordinator emits wildcard routers on.
Bonus fix: the TCP worker config no longer receives frontend_mode="traefik" without a traefik table (latent validation failure in the previous shape); it always keeps the builtin frontend as intended.

Verification: both traefik sub-mode tables validate against the real TraefikConfig pydantic model; the static-config rewrite preserves the plugin + etcd provider blocks; docker compose --profile traefik config passes; pants fmt/lint/check green.

Yaminyam requested a review from a team as a code owner July 10, 2026 05:30

Copilot AI review requested due to automatic review settings July 10, 2026 05:30

Copilot AI reviewed Jul 10, 2026

github-actions Bot assigned Yaminyam Jul 10, 2026

github-actions Bot added the size:L 100~500 LoC label Jul 10, 2026

docs: add changelog for #12749

d20a2f0

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

seedspirit reviewed Jul 13, 2026

View reviewed changes

Yaminyam marked this pull request as draft July 13, 2026 06:50

chore: trim verbose comments per review

49653c4

Keep only the key points in the traefik-related comments (compose service, static config header, installer constants/docstrings). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

Yaminyam mentioned this pull request Jul 13, 2026

Make the AppProxy traefik plugin repo public, pin its version, and verify its contract with core #12816

Open

Yaminyam added this to the 26.5 milestone Jul 21, 2026

Merge remote-tracking branch 'origin/main' into feat/BA-1982-tui-traefik

db16b51

Yaminyam marked this pull request as ready for review July 21, 2026 00:55

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

feat(BA-1982): add traefik app-proxy frontend mode to the TUI installer#12749

feat(BA-1982): add traefik app-proxy frontend mode to the TUI installer#12749
Yaminyam wants to merge 6 commits into
mainfrom
feat/BA-1982-tui-traefik

Yaminyam commented Jul 10, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

seedspirit left a comment

Uh oh!

seedspirit Jul 13, 2026

Uh oh!

Yaminyam Jul 13, 2026

Uh oh!

Yaminyam Jul 13, 2026

Uh oh!

seedspirit Jul 13, 2026

Uh oh!

Yaminyam commented Jul 21, 2026

Uh oh!

Yaminyam commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Uh oh!

Conversation

Yaminyam commented Jul 10, 2026

Summary

Changes

Design notes

Verification

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Uh oh!

seedspirit left a comment

Choose a reason for hiding this comment

Uh oh!

seedspirit Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

Yaminyam Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

Yaminyam Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

seedspirit Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

Yaminyam commented Jul 21, 2026

Uh oh!

Yaminyam commented Jul 21, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants