Skip to content

Commit ac488fe

Browse files
ersinkocclaude
andcommitted
chore(format): apply prettier across the workspace to clear CI drift
`pnpm format:check` in CI was failing on ~137 files that had accumulated prettier drift over many commits. The pre-commit hook (lint-staged + prettier) only formats *staged* files, so any commit that touched unrelated parts of a file left the rest unformatted. Over time this produced enough drift that the format-check job started red-flagging the build, but it was masked because the upstream Build step was dying first on the rollup duplicate (now fixed in ec278b3). This is a pure mechanical reformat — line wrapping at the configured printWidth, trailing-comma normalisation, no logic changes. ~137 files, all matching what `pnpm format` would produce on a clean clone. Drive-by: remove an unused `itemsTab` locator in customize.spec.ts that the reformat surfaced via eslint --max-warnings=0 in the pre-commit hook. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
1 parent 2bb2408 commit ac488fe

137 files changed

Lines changed: 2231 additions & 1323 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

AGENTS.md

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ PostgreSQL via pg adapter. Repositories in `packages/gateway/src/db/repositories
6161
- Unused variables prefixed with `_` (ESLint convention)
6262

6363
<!-- dfmt:v1 begin -->
64+
6465
# Context Discipline — REQUIRED
6566

6667
This project uses DFMT to keep large tool outputs from exhausting the
@@ -72,7 +73,7 @@ in this project.**
7273
Always use DFMT's MCP tools when an output might exceed 2 KB:
7374

7475
| Native | DFMT replacement |
75-
|------------|------------------|
76+
| ---------- | ---------------- |
7677
| `Bash` | `dfmt_exec` |
7778
| `Read` | `dfmt_read` |
7879
| `WebFetch` | `dfmt_fetch` |
@@ -90,7 +91,7 @@ a large output without flooding the context.
9091
DFMT is a strong preference, not a hard dependency. If a `dfmt_*` tool
9192
errors, times out, or is unavailable, report the failure to the user
9293
(one short line — which call, what error) and continue with the native
93-
equivalent so the session is not blocked. The ban is on *silent*
94+
equivalent so the session is not blocked. The ban is on _silent_
9495
fallback — every switch must be announced. After a fallback, drop a
9596
brief `dfmt_remember` note tagged `gap` when practical. If the native
9697
tool is also denied (permission rule, sandbox refusal), stop and ask
@@ -108,4 +109,5 @@ Some agents do not provide hooks to enforce these rules automatically.
108109
**Compliance is your responsibility as the agent.** A single raw shell
109110
output above 8 KB can push earlier context out of the window, erasing
110111
the conversation's history. Following the rules above preserves it.
112+
111113
<!-- dfmt:v1 end -->

architecture.md

Lines changed: 99 additions & 62 deletions
Large diffs are not rendered by default.

docs/ADR/ADR-001-persistent-job-queue.md

Lines changed: 13 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -101,23 +101,27 @@ CREATE TABLE job_history (
101101
## Integration Plan
102102

103103
### Phase 1: Infrastructure (gateway)
104+
104105
1. ✅ Add `jobs` + `job_history` tables to `schema/core.ts` + migration 031_job_queue.sql
105106
2. ✅ Add `JobQueueService` to `services/`
106107
3. ✅ Add `enqueueJob(name, payload, options)` function
107108
4. ✅ Add `enqueueJobAtPriority(queue, priority)` for system jobs
108109

109110
### Phase 2: Workflow System (gateway)
111+
110112
1. Refactor `WorkflowService.dispatchNode()` to enqueue each node as a job instead of executing synchronously
111113
2. Worker pool (configurable size, default 4) claims available jobs, executes, writes results, triggers dependents via gating
112114
3. Node outputs stored in DB — crash recovery reads last successful node output + resumes from next pending node
113115
4. Workflow-level job groups: all nodes in a workflow share a `workflow_run_id`; orphan cleanup marks incomplete runs on restart
114116

115117
### Phase 3: Trigger + Plan Systems (gateway)
118+
116119
1. `TriggerService.scheduleJob()` → enqueue for cron jobs
117120
2. `PlanExecutor` → each step becomes a job; step N+1 enqueued after step N succeeds
118121
3. Subagent sessions → wake up, re-claim in-progress job on reconnect
119122

120123
### Phase 4: Fleet + Subagent (gateway)
124+
121125
1. Fleet worker pool claims jobs from `fleet_tasks` queue
122126
2. Subagent worker pool claims from `subagent_jobs` queue
123127
3. `requeueOrphanedTasks()` replaced by jobs that self-recover on worker restart
@@ -127,30 +131,33 @@ CREATE TABLE job_history (
127131
## Consequences
128132

129133
### Positive
134+
130135
- **Crash recovery**: workers restart, see `active`/`available` jobs, continue
131136
- **Horizontal scaling**: multiple gateway instances share the same queue via `FOR UPDATE SKIP LOCKED`
132137
- **Retry with backoff**: no thundering herd on transient failures
133138
- **No extra infra**: uses existing Postgres, deployed via migration
134139

135140
### Negative
141+
136142
- **Added latency**: jobs are queued, not synchronous; UI feedback must reflect queued state
137143
- **Complexity**: requires managing job state machine + worker pool lifecycle
138144
- **Transaction boundaries**: job claim + result write must be atomic; careful use of `FOR UPDATE SKIP LOCKED`
139145

140146
### Risks
147+
141148
- **Queue depth explosion**: if workers are slower than enqueuers, queue grows unbounded. Mitigate: monitor queue depth in metrics, alert at threshold, circuit-breaker on enqueue
142149
- **Job lock contention**: many workers competing for same priority tier. Mitigate: partition by queue name + priority
143150

144151
---
145152

146153
## Alternatives Considered
147154

148-
| Option | Why Not |
149-
|--------|---------|
150-
| **Redis** (Bull/BullMQ) | Extra infrastructure to deploy/monitor; Redis failure = queue unavailable |
151-
| **Graphile Worker** | More opinionated (synchronous, migration-first); pg-boss async API fits our enqueue-anywhere pattern better |
152-
| **In-memory only** | Already the problem we are solving |
153-
| **SQS/IronMQ** | Cloud-only, not self-hosted; adds vendor lock-in |
155+
| Option | Why Not |
156+
| ----------------------- | ----------------------------------------------------------------------------------------------------------- |
157+
| **Redis** (Bull/BullMQ) | Extra infrastructure to deploy/monitor; Redis failure = queue unavailable |
158+
| **Graphile Worker** | More opinionated (synchronous, migration-first); pg-boss async API fits our enqueue-anywhere pattern better |
159+
| **In-memory only** | Already the problem we are solving |
160+
| **SQS/IronMQ** | Cloud-only, not self-hosted; adds vendor lock-in |
154161

155162
---
156163

docs/ADR/ADR-002-database-retention-policy.md

Lines changed: 37 additions & 28 deletions
Original file line numberDiff line numberDiff line change
@@ -21,21 +21,21 @@ Centralize retention enforcement at the database level using a single nightly cl
2121

2222
### Retention Intervals
2323

24-
| Table | Retention | Rationale |
25-
|-------|-----------|-----------|
26-
| `request_logs` | 30 days | API debugging, auditability |
27-
| `audit_log` | 90 days | Compliance requirement |
28-
| `claw_history` | 90 days | Audit trail |
29-
| `claw_audit_log` | 30 days | High-volume audit detail |
30-
| `workflow_logs` | 90 days | Workflow execution history |
31-
| `plan_history` | 90 days | Plan execution history |
32-
| `trigger_history` | 30 days | Trigger event log |
33-
| `heartbeat_log` | 30 days | High-frequency heartbeat events |
34-
| `subagent_history` | 90 days | Agent execution history |
35-
| `embedding_cache` | 7 days | LRU cache, auto-evicts |
36-
| `jobs` | 30 days | Completed/failed jobs |
37-
| `job_history` | 90 days | Dead-letter entries |
38-
| `provider_metrics` | 30 days | High-frequency telemetry |
24+
| Table | Retention | Rationale |
25+
| ------------------ | --------- | ------------------------------- |
26+
| `request_logs` | 30 days | API debugging, auditability |
27+
| `audit_log` | 90 days | Compliance requirement |
28+
| `claw_history` | 90 days | Audit trail |
29+
| `claw_audit_log` | 30 days | High-volume audit detail |
30+
| `workflow_logs` | 90 days | Workflow execution history |
31+
| `plan_history` | 90 days | Plan execution history |
32+
| `trigger_history` | 30 days | Trigger event log |
33+
| `heartbeat_log` | 30 days | High-frequency heartbeat events |
34+
| `subagent_history` | 90 days | Agent execution history |
35+
| `embedding_cache` | 7 days | LRU cache, auto-evicts |
36+
| `jobs` | 30 days | Completed/failed jobs |
37+
| `job_history` | 90 days | Dead-letter entries |
38+
| `provider_metrics` | 30 days | High-frequency telemetry |
3939

4040
---
4141

@@ -71,6 +71,7 @@ ON CONFLICT (table_name) DO NOTHING;
7171
## Cleanup Job
7272

7373
A single `nightly_retention_cleanup` job runs daily at 02:00 UTC (configurable). For each enabled policy:
74+
7475
1. Call the table's existing `cleanup{N}(retention_days)` method
7576
2. Update `retention_policies.last_cleanup = NOW()`
7677
3. Log count of deleted records
@@ -93,32 +94,40 @@ export class RetentionService {
9394

9495
private async cleanupTable(table: string, days: number): Promise<number> {
9596
const methods: Record<string, () => Promise<number>> = {
96-
request_logs: () => getRequestLogsRepository().cleanupOld(days),
97-
audit_log: () => getAuditRepository().cleanupOld(days),
98-
claw_history: () => getClawsRepository().cleanupOldHistory(days),
99-
claw_audit_log: () => getClawsRepository().cleanupOldAuditLog(days),
100-
workflow_logs: () => getWorkflowsRepository().cleanupOld(days),
101-
plan_history: () => getPlansRepository().cleanupOld(days),
102-
trigger_history: () => getTriggersRepository().cleanupHistory(days),
103-
heartbeat_log: () => getHeartbeatRepository().cleanupOld(days),
97+
request_logs: () => getRequestLogsRepository().cleanupOld(days),
98+
audit_log: () => getAuditRepository().cleanupOld(days),
99+
claw_history: () => getClawsRepository().cleanupOldHistory(days),
100+
claw_audit_log: () => getClawsRepository().cleanupOldAuditLog(days),
101+
workflow_logs: () => getWorkflowsRepository().cleanupOld(days),
102+
plan_history: () => getPlansRepository().cleanupOld(days),
103+
trigger_history: () => getTriggersRepository().cleanupHistory(days),
104+
heartbeat_log: () => getHeartbeatRepository().cleanupOld(days),
104105
subagent_history: () => getSubagentsRepository().cleanupOld(days),
105-
embedding_cache: () => getEmbeddingCacheRepository().cleanupOld(days),
106-
jobs: () => getJobsRepository().cleanupOld(days),
107-
job_history: () => getJobsRepository().cleanupHistory(days),
106+
embedding_cache: () => getEmbeddingCacheRepository().cleanupOld(days),
107+
jobs: () => getJobsRepository().cleanupOld(days),
108+
job_history: () => getJobsRepository().cleanupHistory(days),
108109
provider_metrics: () => getProviderMetricsRepository().cleanupOld(days),
109110
};
110111
const fn = methods[table];
111-
if (!fn) { log.warn('No cleanup method for table', { table }); return 0; }
112+
if (!fn) {
113+
log.warn('No cleanup method for table', { table });
114+
return 0;
115+
}
112116
return fn();
113117
}
114118
}
115119
```
116120

117121
Nightly job registration in `server.ts`:
122+
118123
```typescript
119124
const { getJobQueueService } = await import('./services/job-queue-service.js');
120125
const queue = getJobQueueService();
121-
await queue.enqueue('nightly_retention_cleanup', {}, { queue: 'system', priority: 100, runAfter: nextUTCMidnight() });
126+
await queue.enqueue(
127+
'nightly_retention_cleanup',
128+
{},
129+
{ queue: 'system', priority: 100, runAfter: nextUTCMidnight() }
130+
);
122131
```
123132

124133
---

docs/ADR/ADR-003-api-versioning-strategy.md

Lines changed: 10 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -20,13 +20,13 @@ OwnPilot's API is at `/api/v1/`. As the platform matures, breaking changes will
2020

2121
## Why Side-by-Side
2222

23-
| Factor | Side-by-Side | Header-based |
24-
|--------|-------------|--------------|
25-
| Complexity | Low — same handlers registered twice | High — version detection logic in every handler |
26-
| Backward compat | Full — callers migrate on their timeline | Partial — header must be present |
27-
| Testability | Easy — same tests for both versions | Hard — version branching in tests |
28-
| Extension impact | Extensions using v1 keep working | Extensions break if they don't send header |
29-
| Observability | Version visible in URL | Requires header log extraction |
23+
| Factor | Side-by-Side | Header-based |
24+
| ---------------- | ---------------------------------------- | ----------------------------------------------- |
25+
| Complexity | Low — same handlers registered twice | High — version detection logic in every handler |
26+
| Backward compat | Full — callers migrate on their timeline | Partial — header must be present |
27+
| Testability | Easy — same tests for both versions | Hard — version branching in tests |
28+
| Extension impact | Extensions using v1 keep working | Extensions break if they don't send header |
29+
| Observability | Version visible in URL | Requires header log extraction |
3030

3131
## Implementation
3232

@@ -51,6 +51,7 @@ packages/gateway/src/app.ts
5151
### Version Detection
5252

5353
No version detection middleware. The URL path determines the version:
54+
5455
- `/api/v1/chat` → v1 handlers
5556
- `/api/v2/chat` → v2 handlers (same implementation initially)
5657

@@ -94,11 +95,13 @@ When v1 is deprecated:
9495
## Consequences
9596

9697
**Positive:**
98+
9799
- Extensions and channel adapters continue working without changes
98100
- v2 can evolve independently
99101
- No complex version-detection logic in handlers
100102

101103
**Negative:**
104+
102105
- Route registration is duplicated (but consolidated in one file per domain)
103106
- Some code duplication if v2-specific handlers are needed
104107

0 commit comments

Comments
 (0)