Skip to content

Commit c879af4

Browse files
fix(pi): enable cloud autosync for gentle-engram
1 parent 743f2d0 commit c879af4

20 files changed

Lines changed: 812 additions & 18 deletions

docs/AGENT-SETUP.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -35,12 +35,12 @@ Install Engram's Pi package, the MCP adapter, and Pi MCP config:
3535
engram setup pi
3636
```
3737

38-
`engram setup pi` runs `pi install npm:gentle-engram@0.1.5` and `pi install npm:pi-mcp-adapter`, then ensures Pi settings contain both packages and writes `mcpServers.engram` in the Pi agent MCP config when no Engram server is already configured. Existing `mcpServers.engram` entries are preserved.
38+
`engram setup pi` runs `pi install npm:gentle-engram@0.1.6` and `pi install npm:pi-mcp-adapter`, then ensures Pi settings contain both packages and writes `mcpServers.engram` in the Pi agent MCP config when no Engram server is already configured. Existing custom `mcpServers.engram` entries are preserved; the known generated older launcher is migrated safely.
3939

4040
Manual equivalent:
4141

4242
```bash
43-
pi install npm:gentle-engram@0.1.5
43+
pi install npm:gentle-engram@0.1.6
4444
pi install npm:pi-mcp-adapter
4545
pi-engram init
4646
```
@@ -681,6 +681,12 @@ engram mcp
681681
The process logs `[autosync] started (server=...)` on success. Missing token or server URL logs `[autosync] ERROR: ...` and the process starts normally without autosync.
682682
For `engram mcp`, autosync runs for the lifetime of the stdio MCP process and is stopped when that process exits.
683683

684+
### Pi `gentle-engram` parity
685+
686+
When Pi starts Engram through `gentle-engram`, the package keeps the sync loop in Engram core but can auto-enable it for Pi-managed child processes. If `ENGRAM_CLOUD_AUTOSYNC` is unset, `ENGRAM_CLOUD_TOKEN` is set, and a Cloud server is configured through `ENGRAM_CLOUD_SERVER` or `${ENGRAM_DATA_DIR:-~/.engram}/cloud.json`, Pi-launched `engram serve` and the `pi-engram init` MCP launcher pass `ENGRAM_CLOUD_AUTOSYNC=1` to the child process.
687+
688+
Set `ENGRAM_CLOUD_AUTOSYNC=0` or another explicit value to opt out for that Pi environment. Project enrollment and Cloud pause controls still decide what can actually sync.
689+
684690
---
685691

686692
## Cloud dashboard (templ contributors)

internal/setup/setup.go

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -77,7 +77,7 @@ const claudeCodeMarketplace = "Gentleman-Programming/engram"
7777

7878
const openCodeSubagentStatuslinePlugin = "opencode-subagent-statusline"
7979

80-
const piGentleEngramPackage = "npm:gentle-engram@0.1.5"
80+
const piGentleEngramPackage = "npm:gentle-engram@0.1.6"
8181
const piMCPAdapterPackage = "npm:pi-mcp-adapter"
8282

8383
// claudeCodeMCPTools are the MCP tool permission names for the agent profile

internal/setup/setup_test.go

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -315,7 +315,7 @@ func TestInstallPiInstallsPackagesAndWritesConfig(t *testing.T) {
315315
if result.Agent != "pi" || result.Destination != agentDir || result.Files != 2 {
316316
t.Fatalf("unexpected install result: %#v", result)
317317
}
318-
wantCommands := []string{"pi install npm:gentle-engram@0.1.5", "pi install npm:pi-mcp-adapter"}
318+
wantCommands := []string{"pi install npm:gentle-engram@0.1.6", "pi install npm:pi-mcp-adapter"}
319319
if !reflect.DeepEqual(commands, wantCommands) {
320320
t.Fatalf("unexpected pi install commands: got %#v want %#v", commands, wantCommands)
321321
}
@@ -330,7 +330,7 @@ func TestInstallPiInstallsPackagesAndWritesConfig(t *testing.T) {
330330
if err := json.Unmarshal(settingsRaw, &settings); err != nil {
331331
t.Fatalf("parse settings: %v", err)
332332
}
333-
for _, pkg := range []string{"npm:gentle-engram@0.1.5", "npm:pi-mcp-adapter"} {
333+
for _, pkg := range []string{"npm:gentle-engram@0.1.6", "npm:pi-mcp-adapter"} {
334334
if !slices.Contains(settings.Packages, pkg) {
335335
t.Fatalf("expected settings packages to include %q, got %#v", pkg, settings.Packages)
336336
}
@@ -398,7 +398,7 @@ func TestInstallPiPreservesExistingEngramMCPServer(t *testing.T) {
398398
if err != nil {
399399
t.Fatalf("read settings after install: %v", err)
400400
}
401-
if !strings.Contains(string(settingsRaw), "npm:existing") || !strings.Contains(string(settingsRaw), "npm:gentle-engram@0.1.5") || !strings.Contains(string(settingsRaw), "npm:pi-mcp-adapter") {
401+
if !strings.Contains(string(settingsRaw), "npm:existing") || !strings.Contains(string(settingsRaw), "npm:gentle-engram@0.1.6") || !strings.Contains(string(settingsRaw), "npm:pi-mcp-adapter") {
402402
t.Fatalf("expected settings packages to be preserved and extended, got %s", settingsRaw)
403403
}
404404
}
@@ -409,7 +409,7 @@ func TestInstallPiCommandFailure(t *testing.T) {
409409
return []byte("boom"), errors.New("exit 1")
410410
}
411411
_, err := Install("pi")
412-
if err == nil || !strings.Contains(err.Error(), "install npm:gentle-engram@0.1.5") {
412+
if err == nil || !strings.Contains(err.Error(), "install npm:gentle-engram@0.1.6") {
413413
t.Fatalf("expected pi install error, got %v", err)
414414
}
415415
}
Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,56 @@
1+
# Apply Progress: gentle-engram-cloud-autotick
2+
3+
## Implementation summary
4+
5+
Implemented Pi `gentle-engram` Cloud autosync parity by adding a child-process environment helper and wiring it into Pi-managed Engram processes.
6+
7+
## Changes
8+
9+
- Added `plugin/pi/cloud-autosync-env.js`.
10+
- Enables `ENGRAM_CLOUD_AUTOSYNC=1` only when token+server are configured and no explicit autosync env exists.
11+
- Supports `ENGRAM_CLOUD_SERVER` and persisted `${ENGRAM_DATA_DIR:-~/.engram}/cloud.json` `server_url`.
12+
- Updated `plugin/pi/index.ts`.
13+
- `spawnDetached()` now passes helper-generated env to Engram child processes.
14+
- Updated `plugin/pi/cli.js` and `plugin/pi/mcp-template.json`.
15+
- Generated MCP launcher now applies equivalent auto-enable behavior before running `engram mcp --tools=agent`.
16+
- `pi-engram init` now migrates only the known old generated Engram MCP launcher without `--force`; custom configs remain preserved.
17+
- Added package file entry for the helper.
18+
- Added Node tests for helper behavior, executable MCP launcher behavior, and safe init migration.
19+
- Updated Pi/setup docs.
20+
21+
## TDD evidence
22+
23+
RED:
24+
25+
```text
26+
cd plugin/pi && npm test
27+
# failed: missing cloud-autosync-env.js, index.ts did not use helper, MCP launcher/template lacked ENGRAM_CLOUD_AUTOSYNC logic
28+
# later failed: known old generated MCP launcher was kept instead of migrated
29+
```
30+
31+
GREEN:
32+
33+
```text
34+
cd plugin/pi && npm test
35+
# pass: 28 tests
36+
```
37+
38+
Additional syntax check:
39+
40+
```text
41+
cd plugin/pi && node --check cli.js && node --check cloud-autosync-env.js
42+
# pass
43+
```
44+
45+
## Decisions
46+
47+
- Kept the 30-second tick in Engram core (`internal/cloud/autosync.Manager`).
48+
- Treated token+server as the Pi-side Cloud opt-in signal.
49+
- Preserved explicit `ENGRAM_CLOUD_AUTOSYNC` values as authoritative.
50+
- Did not alter Go core sync behavior.
51+
- Migrated only the exact old generated launcher shape to avoid overwriting custom user MCP configs.
52+
53+
## Pending
54+
55+
- Round 2 Judgment Day re-review results.
56+
- Final verify report update.
Lines changed: 81 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,81 @@
1+
# Design: gentle-engram-cloud-autotick
2+
3+
## Summary
4+
5+
Add a tiny Pi-package environment helper and use it anywhere the Pi package starts/configures an Engram child process. The helper turns Cloud token+server configuration into `ENGRAM_CLOUD_AUTOSYNC=1` for the child only when the user has not set an explicit autosync value.
6+
7+
The 30-second tick stays in `internal/cloud/autosync.Manager`.
8+
9+
## Architecture
10+
11+
```text
12+
Pi session
13+
└─ gentle-engram extension
14+
├─ auto-starts `engram serve`
15+
│ └─ child env includes ENGRAM_CLOUD_AUTOSYNC=1 when token+server configured
16+
└─ pi-engram init writes MCP launcher
17+
└─ launcher starts `engram mcp --tools=agent` with same env gate
18+
19+
Engram core process
20+
└─ cmd/engram.tryStartAutosync
21+
└─ internal/cloud/autosync.Manager
22+
└─ 30s poll ticker + dirty debounce + push/pull + lease/backoff
23+
```
24+
25+
## Files
26+
27+
| File | Action | Purpose |
28+
|---|---|---|
29+
| `plugin/pi/cloud-autosync-env.js` | NEW | Pure helper for child-process env derivation. |
30+
| `plugin/pi/index.ts` | MODIFY | Use helper in `spawnDetached()` for `engram serve` / sync import child processes. |
31+
| `plugin/pi/cli.js` | MODIFY | Generate MCP launcher with equivalent env auto-enable logic. |
32+
| `plugin/pi/mcp-template.json` | MODIFY | Keep packaged template aligned with generated launcher. |
33+
| `plugin/pi/package.json` | MODIFY | Include helper in package files. |
34+
| `plugin/pi/test/cloud-autosync-env.test.mjs` | NEW | Helper unit tests. |
35+
| `plugin/pi/test/index-source.test.mjs` | NEW | Source guard for helper usage in extension. |
36+
| `plugin/pi/test/cli-source.test.mjs` | NEW | Source guard for launcher behavior/template alignment. |
37+
| `plugin/pi/README.md` | MODIFY | Document Pi Cloud autosync parity. |
38+
| `docs/AGENT-SETUP.md` | MODIFY | Clarify Pi package behavior vs raw CLI behavior. |
39+
40+
## Helper contract
41+
42+
```js
43+
cloudAutosyncProcessEnv(baseEnv = process.env) -> env
44+
```
45+
46+
Rules:
47+
48+
1. If `baseEnv.ENGRAM_CLOUD_AUTOSYNC` is non-blank, return `baseEnv` unchanged.
49+
2. If `baseEnv.ENGRAM_CLOUD_TOKEN` is blank, return unchanged.
50+
3. If `baseEnv.ENGRAM_CLOUD_SERVER` is non-blank, return a shallow copy with `ENGRAM_CLOUD_AUTOSYNC: "1"`.
51+
4. Else inspect `${ENGRAM_DATA_DIR || ~/.engram}/cloud.json` for non-empty `server_url`; if present, return shallow copy with autosync `1`.
52+
5. On absent/malformed config, return unchanged.
53+
54+
Returning unchanged by reference for no-op paths makes tests easy and avoids unnecessary env object copies. Returning a copy for enabled paths avoids mutating `process.env`.
55+
56+
## MCP launcher
57+
58+
The generated `node -e` launcher cannot import package files reliably, so it should embed equivalent minimal logic. This is acceptable because the launcher is already a generated adapter boundary. Keep it dependency-free and Node built-in only.
59+
60+
## Boundary rationale
61+
62+
- Plugin owns process setup, not Cloud sync behavior.
63+
- Core `tryStartAutosync()` remains the final authority for startup validation.
64+
- Core manager remains the only owner of tick interval, network push/pull, policy failure handling, and status.
65+
- Server-side enrollment/pause remains authoritative.
66+
67+
## Validation
68+
69+
Targeted:
70+
71+
```sh
72+
cd plugin/pi && npm test
73+
```
74+
75+
Recommended broader check if time allows:
76+
77+
```sh
78+
go test ./cmd/engram ./internal/cloud/autosync/...
79+
```
80+
81+
No Go behavior should be changed by this SDD.
Lines changed: 95 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,95 @@
1+
# Exploration: gentle-engram-cloud-autotick
2+
3+
## Status
4+
5+
- SDD mode: auto
6+
- Artifact store: OpenSpec
7+
- Worktree: `../engram-gentle-engram-cloud-autotick`
8+
9+
## Question
10+
11+
Pi `gentle-engram` currently connects Pi sessions to Engram memory, but Cloud autosync only starts when the underlying Engram process sees `ENGRAM_CLOUD_AUTOSYNC=1`. The user wants Pi package parity with Engram Cloud's 30-second autosync loop.
12+
13+
## Evidence
14+
15+
### Pi plugin behavior
16+
17+
- `plugin/pi/index.ts` registers Pi-native `mem_*` tools and lifecycle handlers.
18+
- `initOnce()` starts `engram serve` when `/health` is not reachable and `ENGRAM_URL` is not set.
19+
- The spawned `engram serve` process inherits the parent environment, but the plugin does not derive or inject `ENGRAM_CLOUD_AUTOSYNC=1` from Cloud configuration.
20+
- `initOnce()` runs one `engram sync --import` for git-synced chunks when `.engram/manifest.json` exists. This is not Cloud autosync.
21+
22+
### Engram core autosync behavior
23+
24+
- `internal/cloud/autosync/manager.go` owns durable background sync behavior.
25+
- `DefaultConfig()` sets `PollInterval: 30 * time.Second`.
26+
- `Run()` uses `time.NewTicker(m.cfg.PollInterval)` and runs `safeRun()` on each poll tick.
27+
- This manager also supports dirty notification debounce, push/pull, lease, backoff, and status.
28+
29+
### Core process startup gates
30+
31+
- `cmd/engram/main.go::tryStartAutosync()` starts autosync only when `ENGRAM_CLOUD_AUTOSYNC` is exactly `"1"`.
32+
- It also requires a Cloud server URL and `ENGRAM_CLOUD_TOKEN`.
33+
- The function is intentionally non-fatal: missing config logs and leaves Engram running without autosync.
34+
- `engram serve` and `engram mcp` call `tryStartAutosync()`.
35+
36+
### Pi MCP setup behavior
37+
38+
- `plugin/pi/cli.js` writes an MCP server launcher that runs `engram mcp --tools=agent`.
39+
- The launcher currently inherits the Pi process environment but does not infer `ENGRAM_CLOUD_AUTOSYNC=1` from Cloud token/server configuration.
40+
41+
### Docs behavior
42+
43+
- `docs/AGENT-SETUP.md` documents the current core toggle: users must export `ENGRAM_CLOUD_AUTOSYNC=1`, `ENGRAM_CLOUD_TOKEN`, and `ENGRAM_CLOUD_SERVER`.
44+
- `plugin/pi/README.md` says Cloud is opt-in and project-scoped, but does not document Pi package autosync parity.
45+
46+
## Findings
47+
48+
### What is missing in Pi parity today?
49+
50+
Pi does not automatically enable the existing Engram core autosync loop for Engram processes it launches/configures. Users who run Pi with Cloud token/server configured still need to know and export `ENGRAM_CLOUD_AUTOSYNC=1`; otherwise the 30-second core ticker never starts.
51+
52+
### Where should the 30-second tick live?
53+
54+
The tick must stay in `internal/cloud/autosync`. The Pi plugin should not implement its own polling or Cloud sync loop. `gentle-engram` should only provide thin process environment wiring so the existing Engram core process starts its own autosync manager.
55+
56+
### What gates preserve opt-in cloud sync?
57+
58+
Recommended plugin-side auto-enable gate:
59+
60+
1. Never override an explicit `ENGRAM_CLOUD_AUTOSYNC` value.
61+
2. Require `ENGRAM_CLOUD_TOKEN` to be non-empty.
62+
3. Require either `ENGRAM_CLOUD_SERVER` or persisted local `cloud.json.server_url`.
63+
4. Let Engram core continue enforcing project enrollment, server policy, pause controls, lease, retry, and status.
64+
65+
This keeps local-first behavior because no token means no Cloud replication; configured token+server is a strong Cloud opt-in signal.
66+
67+
### Tests that should fail first
68+
69+
1. `plugin/pi/test/cloud-autosync-env.test.mjs`
70+
- auto-enables when token + `ENGRAM_CLOUD_SERVER` are present;
71+
- auto-enables when token + persisted `cloud.json.server_url` are present;
72+
- does not enable without token;
73+
- does not override explicit `ENGRAM_CLOUD_AUTOSYNC=0` or `1`.
74+
2. `plugin/pi/test/index-source.test.mjs`
75+
- verifies `spawnDetached` uses the cloud-autosync env helper when starting Engram processes.
76+
3. `plugin/pi/test/cli-source.test.mjs` or CLI helper tests
77+
- verifies generated MCP launcher includes equivalent auto-enable env logic for `engram mcp`.
78+
79+
### Docs to update
80+
81+
- `plugin/pi/README.md`: add Pi Cloud autosync parity section.
82+
- `docs/AGENT-SETUP.md`: clarify that Pi `gentle-engram` can auto-enable autosync for its launched/configured Engram processes when token+server are present, while raw `engram serve`/`engram mcp` still support the explicit env toggle.
83+
84+
## Risks
85+
86+
| Risk | Mitigation |
87+
|---|---|
88+
| Plugin violates thin-adapter boundary | Only set child process environment; do not implement sync/polling in plugin. |
89+
| Surprise Cloud sync | Require token + server and preserve explicit `ENGRAM_CLOUD_AUTOSYNC` overrides. Project enrollment remains authoritative. |
90+
| Duplicate autosync workers | Existing SQLite lease in core manager prevents duplicate workers from syncing concurrently. |
91+
| Noisy logs when Cloud partially configured | Gate on token+server before injecting autosync. |
92+
93+
## Recommendation
94+
95+
Implement a small Pi helper that builds child-process env for Engram processes. Use it for `engram serve`, generated `engram mcp` launcher, and any plugin-spawned one-shot sync command where harmless. The helper should only set `ENGRAM_CLOUD_AUTOSYNC=1` when the user has Cloud token + server configured and did not set an explicit autosync value.
Lines changed: 60 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,60 @@
1+
# Proposal: gentle-engram-cloud-autotick
2+
3+
## Intent
4+
5+
Give Pi `gentle-engram` parity with Engram Cloud autosync by auto-enabling the existing Engram core 30-second autosync manager for Engram processes launched or configured by the Pi package when Cloud is clearly configured.
6+
7+
The Pi plugin must not implement Cloud sync itself. It should remain a thin adapter that starts/configures Engram processes with the right environment so `internal/cloud/autosync.Manager` owns the actual push/pull tick, lease, backoff, policy handling, and status.
8+
9+
## Scope
10+
11+
### In Scope
12+
13+
- Add a small Pi package helper that decides whether child Engram processes should receive `ENGRAM_CLOUD_AUTOSYNC=1`.
14+
- Use that helper when `plugin/pi/index.ts` auto-starts `engram serve`.
15+
- Update the `pi-engram init` MCP launcher so generated `engram mcp --tools=agent` processes get equivalent auto-enable behavior.
16+
- Update `plugin/pi/mcp-template.json` to match the generated launcher.
17+
- Add deterministic Node tests for the helper and source-level integration guardrails.
18+
- Update Pi and setup docs.
19+
20+
### Out of Scope
21+
22+
- Reimplementing the 30-second autosync loop in TypeScript.
23+
- Changing `internal/cloud/autosync` tick interval or sync algorithm.
24+
- Changing server-side project enrollment, pause, or auth semantics.
25+
- Enabling Cloud sync without token+server configuration.
26+
- Changing raw `engram serve` / `engram mcp` CLI semantics outside Pi package launchers.
27+
28+
## Behavioral Gate
29+
30+
The Pi package should inject `ENGRAM_CLOUD_AUTOSYNC=1` only when all are true:
31+
32+
1. `ENGRAM_CLOUD_AUTOSYNC` is unset/blank.
33+
2. `ENGRAM_CLOUD_TOKEN` is non-empty.
34+
3. A Cloud server is configured via either:
35+
- `ENGRAM_CLOUD_SERVER`, or
36+
- `${ENGRAM_DATA_DIR:-~/.engram}/cloud.json` with a non-empty `server_url`.
37+
38+
Explicit `ENGRAM_CLOUD_AUTOSYNC=0`, `false`, or `1` must be preserved.
39+
40+
## Success Criteria
41+
42+
- Pi-launched `engram serve` starts core autosync automatically when Cloud token+server are configured.
43+
- Pi-configured `engram mcp --tools=agent` starts core autosync automatically under the same gate.
44+
- No Cloud token means no auto-enable.
45+
- Explicit autosync env remains authoritative.
46+
- Tests cover env-server, persisted-server, missing-token, and explicit override cases.
47+
- Docs tell users that the 30-second tick remains owned by Engram core.
48+
49+
## Risks and Mitigations
50+
51+
| Risk | Mitigation |
52+
|---|---|
53+
| Surprise Cloud replication | Require token+server and preserve explicit overrides; core enrollment still gates projects. |
54+
| Plugin becomes thick | Only environment derivation lives in plugin; all sync behavior stays in Go core. |
55+
| MCP launcher drift | Keep `cli.js` and `mcp-template.json` aligned and add source tests. |
56+
| Partial cloud config causes noise | Do not inject autosync unless both token and server are present. |
57+
58+
## Review Workload Forecast
59+
60+
Estimated change: ~250-450 lines across helper, tests, launcher string, docs, and SDD artifacts. This is under the session budget of 800 changed lines and should fit a single PR.

0 commit comments

Comments
 (0)