You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/engram-cloud/troubleshooting.md
+27-3Lines changed: 27 additions & 3 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -101,13 +101,14 @@ You may also see the failure only during the final server push, even after `doct
101
101
102
102
```text
103
103
write chunk: cloud: push chunk ...: status 400: invalid push payload: sessions[N].directory is required
104
+
write chunk: cloud: push chunk ...: status 400: invalid push payload: observations[N].title is required
104
105
```
105
106
106
-
It means a historical `session` mutation in `sync_mutations` is missing `directory`, a local `sessions` row included in the project export still has an empty/null `directory`, or a historical `observation` mutation is missing one of the required upsert fields: `sync_id`, `session_id`, `type`, `title`, `content`, or `scope`. Newer Engram versions write these fields correctly, but old journal rowsor local session rows may still need repair before first cloud upload.
107
+
It means a historical `session` mutation in `sync_mutations` is missing `directory`, a local `sessions` row included in the project export still has an empty/null `directory`, a local `observations` row included in the project export is missing a cloud-push required field, or a historical `observation` mutation is missing one of the required upsert fields: `sync_id`, `session_id`, `type`, `title`, `content`, or `scope`. Newer Engram versions write these fields correctly, but old journal rows, local session rows, or exported local observation rows may still need repair before first cloud upload.
107
108
108
109
### Safe path: helper script
109
110
110
-
Engram includes a temporary rescue helper. Despite the historical file name, it repairs missing session directories in both `sync_mutations` payloads and local `sessions.directory` rows, plus missing observation payload fields:
111
+
Engram includes a temporary rescue helper. Despite the historical file name, it repairs missing session directories in both `sync_mutations` payloads and local `sessions.directory` rows, missing observation payload fields, and exported local `observations` rows whose required cloud-push fields are empty:
Loop mode repairs exactly one supported blocker (`entity=session|observation op=upsert`), reruns `engram cloud upgrade doctor --project <project>`, then repeats until doctor no longer reports a supported blocker. If doctor reports ready but local `sessions` rows included in the project export still have empty/null `directory`, loop mode applies that fallback repair and reruns doctor once more. It still stops on unsupported blockers, project mismatches, or observation payloads that cannot be fully inferred. In non-interactive loop mode, rerun with `--interactive` when the script asks for human-provided observation fields.
148
+
Loop mode repairs exactly one supported blocker (`entity=session|observation op=upsert`), reruns `engram cloud upgrade doctor --project <project>`, then repeats until doctor no longer reports a supported blocker. If doctor reports ready but local `sessions` rows included in the project export still have empty/null `directory`, loop mode applies that fallback repair and reruns doctor once more. If exported local `observations` rows are missing required fields such as `title`, loop mode applies that fallback repair next and reruns doctor again. It still stops on unsupported blockers, project mismatches, or observation data that cannot be fully inferred. In non-interactive loop mode, rerun with `--interactive` when the script asks for human-provided observation fields.
148
149
149
150
If one-shot mode finds no doctor blocker but reports local sessions with empty/null directory, preview and apply the fallback explicitly:
If final push fails with `observations[N].title is required` even though doctor is ready, preview and apply the exported observation fallback explicitly:
It does not invent values in non-interactive mode. If the local observation row is missing, or the required payload fields would still be empty after patching, the script exits non-zero without modifying the database. Use `--interactive` when you need to provide missing observation values manually after reviewing the payload and content excerpt.
188
203
204
+
For exported local observation repairs, the script changes only `observations` rows that belong to the requested project export scope:
205
+
206
+
- rows where `ifnull(project, '') = '<project>'`
207
+
- rows with empty `project` whose `session_id` belongs to a `sessions.project = '<project>'` session
208
+
209
+
It does not touch `sync_mutations`. Safe inferred fields are filled as follows: empty `sync_id` becomes the row `id`, empty `type` becomes `manual`, empty `title` is inferred from non-empty `content`, and empty `scope` becomes `project`. Empty `session_id` is always blocked. Empty `content` is blocked unless you run with `--interactive` and provide it yourself.
210
+
189
211
It never changes `last_acked_seq`, never deletes mutations, and creates a timestamped database backup before each applied blocker. Loop mode can be noisy because it intentionally keeps one backup per repair.
190
212
191
213
### How the script finds `seq`
@@ -253,6 +275,7 @@ sqlite3 ~/.engram/engram.db "select seq, entity, op, entity_key, payload from sy
253
275
sqlite3 ~/.engram/engram.db "select id, project, directory from sessions where id = 'manual-save-current';"
254
276
sqlite3 ~/.engram/engram.db "select id, project, started_at, directory from sessions where project = '<project>' and (directory is null or directory = '');"
255
277
sqlite3 ~/.engram/engram.db "select s.id, s.project, s.started_at, s.directory from sessions s where (s.directory is null or s.directory = '') and (s.project = '<project>' or s.id in (select session_id from observations where ifnull(project, '') = '<project>' union select session_id from user_prompts where ifnull(project, '') = '<project>'));"
278
+
sqlite3 ~/.engram/engram.db "select o.id, ifnull(o.sync_id, '') as sync_id, ifnull(o.session_id, '') as session_id, ifnull(o.project, '') as project, ifnull(o.type, '') as type, ifnull(o.title, '') as title, ifnull(o.scope, '') as scope from observations o where (ifnull(o.project, '') = '<project>' or (ifnull(o.project, '') = '' and o.session_id in (select id from sessions where ifnull(project, '') = '<project>'))) and (ifnull(o.sync_id, '') = '' or ifnull(o.session_id, '') = '' or ifnull(o.type, '') = '' or ifnull(o.title, '') = '' or ifnull(o.content, '') = '' or ifnull(o.scope, '') = '');"
256
279
```
257
280
258
281
Do not manually edit SQLite without a backup.
@@ -269,6 +292,7 @@ Do not manually edit SQLite without a backup.
269
292
|`session payload directory is required`| Run the missing session directory helper |
270
293
|`sessions[N].directory is required`| Run the missing session directory helper with `--fix-empty-sessions` or `--all`|
271
294
|`observation payload title is required for upsert`| Run the missing session directory helper; it also repairs missing observation payload fields from local `observations`|
295
+
|`observations[N].title is required`| Run the missing session directory helper with `--fix-empty-observations` or `--all`|
272
296
|`401` or `auth_required`| Check `ENGRAM_CLOUD_TOKEN` on the client and server |
273
297
|`403` or `policy_forbidden`| Check `ENGRAM_CLOUD_ALLOWED_PROJECTS` on the server |
274
298
|`server_unsupported`| Redeploy a cloud server with mutation endpoints |
0 commit comments