Merge pull request #1994 from github/contrib/chromium-control-canvas

Add Chromium Control Canvas extension

Author: AndreaGriffiths11

extensions/chromium-control-canvas/README.md

@@ -0,0 +1,90 @@
+# Chromium Control Canvas
+
+A GitHub Copilot canvas that drives a real **headful Chromium** window via Playwright.
+The host app's built-in `browser` canvas is WebKit (WKWebView); this gives you actual
+Chromium, controllable both from the panel UI and by the agent.
+
+The canvas panel is a control strip (URL bar, back/forward/reload, screenshot). A separate
+Chromium window does the real rendering, because you can't embed Chromium inside a WebKit
+iframe.
+
+## Files
+
+- `extension.mjs` — the extension: canvas declaration, Playwright launch, a loopback HTTP
+  server for the panel, and the agent actions.
+- `index.html` — the control strip UI the panel renders.
+- `package.json` — declares the `playwright` dependency and `"type": "module"`.
+- `copilot-extension.json` — name/version metadata.
+
+## Prerequisites
+
+- **Node.js 20.19 or newer** (the Copilot SDK requires `node ^20.19.0 || >=22.12.0`).
+  The extension runs as a Node child process.
+- The app's **canvas / UI-extensions experiment enabled**. Without it, the extension
+  loads but the canvas never appears in the panel. Enable it in the app's
+  Settings → Experiments. (This may not be available to all accounts.)
+
+## Install
+
+Drop this folder at `~/.copilot/extensions/chromium-control-canvas/` (user scope) or in a repo's
+`.github/extensions/chromium-control-canvas/` (project scope), then install dependencies and the
+Chromium binary from inside the folder you copied:
+
+```sh
+# User scope
+cd ~/.copilot/extensions/chromium-control-canvas
+
+# Or project scope, from the repository root
+cd .github/extensions/chromium-control-canvas
+
+npm install                     # playwright is declared in package.json
+npx playwright install chromium # downloads the browser, a few hundred MB
+```
+
+Reload extensions in the app, then open the `chromium-control-canvas` canvas.
+
+Note: copying the extension files only places the source. It does **not** run the
+commands above or enable the experiment, so those steps are still required on first
+setup.
+
+## Attach to your own Chrome
+
+By default the canvas launches the bundled Chromium with a persistent profile. To drive
+a Chrome you already have running instead, start it with a debug port and pass `cdpUrl`
+when opening the canvas:
+
+```sh
+google-chrome --remote-debugging-port=9222   # then open the canvas with cdpUrl: http://localhost:9222
+```
+
+In this mode the extension connects over CDP and never launches or kills your browser;
+closing the canvas just disconnects.
+
+## Agent actions
+
+- `navigate { url }` — go to a URL or search query (blocklist-guarded).
+- `back` / `forward` / `reload` — history navigation.
+- `current_url` — current URL and page title.
+- `snapshot` — structured list of visible interactive elements, each with a stable ref.
+- `click { ref | selector }` — click an element by snapshot ref or CSS selector.
+- `type { ref | selector, text, submit? }` — fill an input; optionally press Enter.
+- `screenshot { fullPage? }` — save a PNG to `artifacts/` and return its path and size.
+
+## Notes
+
+- A persistent profile is stored under
+  `$COPILOT_HOME/extensions/chromium-control-canvas/profile` (default
+  `~/.copilot/extensions/chromium-control-canvas/profile`) so logins survive restarts.
+  **Do not commit or share this folder** — it contains real session cookies.
+- Raw `evaluate` (arbitrary in-page JS) is intentionally omitted.
+- `navigate` is checked against a blocklist, and a request interceptor also blocks
+  navigations to blocked hosts that happen via in-page redirects. The shipped
+  `BLOCKLIST` entries are illustrative examples, not real coverage — edit the list in
+  `extension.mjs` to fit your environment.
+- The loopback control server requires a per-launch token (templated into the panel),
+  so other pages in your browser can't drive it.
+- Typed text (e.g. passwords) is redacted in `audit.log`, and password field values are
+  excluded from snapshots.
+- Generated at runtime and not part of the source: `node_modules/` in the copied
+  extension folder, plus `profile/`, `artifacts/`, and `audit.log` under
+  `$COPILOT_HOME/extensions/chromium-control-canvas/`.

extensions/chromium-control-canvas/assets/preview.png

@@ -0,0 +1,4 @@
+{
+  "name": "chromium-control-canvas",
+  "version": 1
+}