Skip to content

Commit 48e4516

Browse files
authored
feat(self-driving): Simplify step 1 of self-driving Wizard, as we don't need to check FFs anymore (#739)
1 parent 69caa2a commit 48e4516

4 files changed

Lines changed: 51 additions & 17 deletions

File tree

src/lib/programs/__tests__/self-driving-detect.test.ts

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -67,6 +67,20 @@ describe('SELF_DRIVING_ABORT_CASES', () => {
6767
expect(matched[0].message).toBeTruthy();
6868
expect(matched[0].body).toBeTruthy();
6969
});
70+
71+
it('frames the unavailable-access abort as open beta, not a closed per-team beta', () => {
72+
// STEP 1 no longer gates on access — Self-driving is open beta — but the
73+
// abort is kept as a safety net. Its copy must say the product is still
74+
// in beta while dropping the old closed/per-team "join the beta" framing.
75+
const [accessCase] = SELF_DRIVING_ABORT_CASES.filter((c) =>
76+
c.match.test('self-driving is not available for this project'),
77+
);
78+
expect(accessCase).toBeDefined();
79+
const copy = `${accessCase.message} ${accessCase.body}`.toLowerCase();
80+
expect(copy).toContain('open beta');
81+
expect(copy).not.toContain('per team');
82+
expect(copy).not.toContain('join the beta');
83+
});
7084
});
7185

7286
describe('selfDrivingConfig', () => {

src/lib/programs/self-driving/ARCHITECTURE.md

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,7 @@ matching context-mill file carries the HOW.
5959

6060
**Step backbone (expected action, one line each):**
6161

62-
1. **Check access**probe the Signals API; if it's not available for the team, abort cleanly (`[ABORT] self-driving is not available for this project`).
62+
1. **Check access****instant, no probe.** Self-driving is in **open beta** (available to every team), so there is no access gate to check; the step just marks itself in_progress→completed (no MCP call) so the step-tracking funnel still fires and the user gets an immediate first checkmark. `[ABORT] self-driving is not available for this project` is kept only as a safety net for a genuine Signals-API outage during the run.
6363
2. **Read context** — build an evidence picture of which products are in use (setup report + `signals-scout-project-profile-get` + cheap usage probes + a light repo scan); read-only.
6464
3. **Connect GitHub** — required; if no `github` integration, send the user through the GitHub App install (one-click authorize deep-link) and re-verify; abort if declined.
6565
4. **Enable sources** — always enable the scout gate; enable native sources (error tracking, replay, support) only where step-2 evidence shows the product is in use.
@@ -72,7 +72,7 @@ The table below adds the skill reference and the tool/MCP surface for each.
7272

7373
| # | Step | Skill ref / file | Tools · surface |
7474
|---|---|---|---|
75-
| 1 | Check access | `1-check-access.md` | Probe `inbox-source-configs-list` (no readable beta flag — the API *is* the probe). Fail → `[ABORT] self-driving is not available for this project`. |
75+
| 1 | Check access | `1-check-access.md` | **No probe — instant** (open beta: available to every team). Marks the task in_progress→completed immediately, calls no MCP tool. The `[ABORT] self-driving is not available for this project` string remains a safety net for a genuine Signals-API outage during the run, not a beta gate. |
7676
| 2 | Read project & Signals state | `2-read-context.md` | `./posthog-setup-report.md` + `signals-scout-project-profile-get` + cheap usage probes. Prompt opt-ins are authoritative ("repo evidence rules a product IN, never OUT"). |
7777
| 3 | Connect GitHub (REQUIRED) | `3-github.md` | `integrations-list` for `kind:"github"`; else `wizard_ask` with the one-click `integrations/authorize?kind=github` deep-link (the single link covers fresh install / link-existing / re-auth — no separate settings "re-link" path), re-verify after a manual "done". Can't → `[ABORT] github connection declined`. |
7878
| 4 | Enable signal sources | `4-sources.md` | Create/enable `SignalSourceConfig` rows for products in use (`inbox-source-configs-*`). Always enables the scout gate `signals_scout`/`cross_source_issue`. Never enables an unconfirmed tool. |
@@ -232,6 +232,14 @@ scoped to `created_via=MCP`.
232232

233233
## 6. Gating & prerequisites — "will it actually work?"
234234

235+
> [!NOTE]
236+
> **Open beta — the wizard no longer probes access.** Self-driving is in open
237+
> beta (available to every team), so STEP 1 dropped its `inbox-source-configs-list`
238+
> access probe and runs instantly; the wizard surfaces no beta gate of its own.
239+
> The PostHog-side gates below still apply **server-side** (a flag not yet at 100%
240+
> just means findings won't surface), and the `[ABORT] self-driving is not available
241+
> for this project` path is now only a safety net for a genuine Signals-API outage.
242+
235243
1. **UI flag `product-autonomy`** (`posthog/frontend/src/lib/constants.tsx`,
236244
`FEATURE_FLAGS.PRODUCT_AUTONOMY`). Frontend-only — gates the Inbox scene, nav item, and source-config
237245
loading. Off → the user can't *see* the inbox; the pipeline is unaffected.
@@ -281,8 +289,10 @@ Plus the **Temporal coordinator schedule** (`signals-scout-coordinator-schedule`
281289
3. **posthog backend deploy** of the `feat/signals-scout-config-sync` work: the `sync` endpoint, companion
282290
seeding (`lazy_seed.py`), and the 10 canonical scout skills.
283291
4. **Temporal coordinator schedule** running in prod.
284-
5. **Flag rollout:** `signals-scout` 100%-on with target teams in `guaranteed_team_ids`; `product-autonomy`
285-
on for target users.
292+
5. **Flag rollout (open beta = everyone):** `signals-scout` 100%-on for all teams (still the real
293+
server gate for dispatch); `product-autonomy` on for all users. The wizard no longer probes access
294+
in STEP 1, so an un-flagged team isn't turned away at setup — it just won't see findings until the
295+
server-side flags are on.
286296
6. **Per-team runtime** (user's responsibility): org AI consent on, GitHub connected.
287297

288298
> [!NOTE]

src/lib/programs/self-driving/detect.ts

Lines changed: 16 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,18 @@
55
* `session.installDir` is a real, readable directory. We deliberately do
66
* NOT require the base posthog-integration report to be present — it is a
77
* report many users never commit, and `requires: ['posthog-integration']`
8-
* is metadata, not a hard runtime gate. Real readiness (integration state
9-
* + beta access) is established by the agent's STEP 1 Signals API probe at
10-
* the start of the run. The beta gates (the `product-autonomy` access flag
11-
* and `signals-scout` enrollment — PostHog-side flag names, unchanged by
12-
* the wizard-side "self-driving" rename) are PostHog-internal flags with no
13-
* customer-facing read API, which is why that probe lives in the run and
14-
* emits a structured `[ABORT]` when the product is not available.
8+
* is metadata, not a hard runtime gate.
9+
*
10+
* Self-driving is now in OPEN beta — available to every team — so STEP 1
11+
* no longer probes the Signals API as an access gate; it completes
12+
* instantly so the run opens with a fast first checkmark. The
13+
* `self-driving is not available for this project` abort below is kept
14+
* only as a safety net: if the Signals API genuinely can't be reached
15+
* during the run (a hard error that is unexpected in open beta), the skill
16+
* emits it and the wizard renders a friendly "try again" screen — now with
17+
* open-beta wording, not the old closed, per-team "join the beta" copy. The
18+
* PostHog-side flags (`product-autonomy`, `signals-scout`) are unchanged by
19+
* the wizard-side "self-driving" rename.
1520
*/
1621

1722
import { existsSync, statSync } from 'fs';
@@ -39,10 +44,10 @@ export const SELF_DRIVING_ABORT_CASES: AbortCase[] = [
3944
match: /^self-driving is not available for this project$/i,
4045
message: 'PostHog Self-driving is not available for this project',
4146
body:
42-
'Self-driving is in beta and is enabled per ' +
43-
'team by PostHog. This project does not appear to have access yet. ' +
44-
'Reach out to your PostHog contact (or wizard@posthog.com) to join ' +
45-
'the beta, then run the wizard again.',
47+
'Self-driving is in open beta and available to every team, so this ' +
48+
'is unexpected — the PostHog Signals API could not be reached for ' +
49+
'this project. Nothing was changed. Try again in a moment, and if it ' +
50+
'keeps happening reach out to wizard@posthog.com.',
4651
},
4752
{
4853
// Skill emits: [ABORT] github connection declined

src/lib/programs/self-driving/prompt.ts

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -72,8 +72,13 @@ Wizard mechanics:
7272
Follow these steps IN ORDER. Do not skip or reorder.
7373
7474
STEP 1 — Check Self-driving access. (skill: "Check access")
75-
Probe the Signals API as the skill describes. If the API is not
76-
available for this project (permission or not-found errors), emit
75+
Self-driving is in open beta and available to every team, so there is
76+
no access gate to probe. Do NOT call any MCP tool here — mark this task
77+
in_progress and then completed right away and emit the
78+
${AgentSignals.STATUS} line, so the user sees an immediate first step.
79+
Only if the Signals API later turns out to be genuinely unreachable for
80+
this project (a hard error on every Signals call, unexpected in open
81+
beta) should you emit
7782
${AgentSignals.ABORT} self-driving is not available for this project
7883
and halt.
7984

0 commit comments

Comments
 (0)