Skip to content

Commit 526805e

Browse files
os-zhuangclaude
andauthored
feat(lifecycle): ADR-0057 follow-ups — retire per-plugin sweepers, dev telemetry datasource + db:clean, Studio lifecycle form (#2835)
Implements #2834 ①–③ (④ PG-partition rotation design-sketched on the issue, deferred until CI has a PostgreSQL service). ① JobRunRetention / NotificationRetention and their retentionDays/retentionSweepMs options removed — the LifecycleService is the one sweeper (windows tuned via the lifecycle settings namespace). Includes a fix: the blanket 30d lifecycle retention on sys_automation_run (from #2791) could strand suspended approval runs; removed — bounding stays with the automation store's terminal-only sweep. ② objectstack dev provisions a dedicated telemetry datasource (<primary>.telemetry.db; OS_TELEMETRY_DB to opt out/override) and new `os db clean` runs the one-time VACUUM legacy files need to adopt auto_vacuum=INCREMENTAL. ③ object.form.ts exposes the lifecycle block; metadata-forms i18n bundles regenerated with curated zh-CN. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
1 parent 51822cf commit 526805e

27 files changed

Lines changed: 1372 additions & 831 deletions
Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,15 @@
1+
---
2+
'@objectstack/spec': minor
3+
'@objectstack/cli': minor
4+
'@objectstack/service-job': minor
5+
'@objectstack/service-messaging': minor
6+
'@objectstack/service-automation': patch
7+
'@objectstack/platform-objects': patch
8+
---
9+
10+
ADR-0057 data-lifecycle follow-ups (#2834): the per-plugin retention sweepers are retired, telemetry separation goes live in dev, and the lifecycle contract reaches the Studio.
11+
12+
- **BREAKING (ships as minor per the launch-window convention)**: `JobRunRetention` / `NotificationRetention` and the `retentionDays` / `retentionSweepMs` options on `JobServicePlugin` / `MessagingServicePlugin` are removed. The platform LifecycleService enforces the same windows from the `lifecycle` declarations (`sys_job_run` 30d, notification pipeline 90d); tune them at runtime via the `lifecycle` settings namespace (`retention_overrides`, tenant-scoped).
13+
- **Fix**: `sys_automation_run` no longer declares a blanket 30d lifecycle retention — that table interleaves live SUSPENDED runs (an approval may stay paused for months) with terminal history, and a blanket age reap could strand in-flight approvals. Bounding stays with the automation store's terminal-only sweep.
14+
- **CLI**: `objectstack dev` now provisions a dedicated `telemetry` datasource (`<primary>.telemetry.db`) for file-backed SQLite primaries, so lifecycle-classed system data stops sharing the business dev DB (`OS_TELEMETRY_DB=0` opts out; `OS_TELEMETRY_DB=<path>` opts in anywhere). New `os db clean` runs the one-time `VACUUM` that lets legacy files adopt `auto_vacuum=INCREMENTAL` and reports reclaimed bytes.
15+
- **Studio**: the object metadata form exposes the `lifecycle` block (class + retention/TTL/rotation/archive/reclaim); metadata-forms i18n bundles regenerated with curated zh-CN translations.

content/docs/getting-started/cli.mdx

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,6 +74,7 @@ predicate/schema/binding mistakes that fail silently at runtime), and `os dev --
7474
| `os init [name]` | | Initialize a new ObjectStack project in the current directory |
7575
| `os dev [package]` | | Start development mode with hot reload |
7676
| `os serve [config]` | | Start the ObjectStack server with plugin auto-detection |
77+
| `os db clean` | | Reclaim SQLite free space with a one-time `VACUUM` (ADR-0057) |
7778

7879
#### `os init`
7980

@@ -134,6 +135,13 @@ os dev --database file:./data/test.db --auth-secret $(openssl rand -hex 32)
134135
| `--no-compile` || Skip the auto-compile step (errors if no artifact present) |
135136
| `-v, --verbose` || Verbose output |
136137

138+
With a file-backed SQLite database, dev also provisions a sibling
139+
`<db>.telemetry.<ext>` file registered as the `telemetry` datasource —
140+
lifecycle-classed system data (activity streams, job runs, notifications,
141+
audit) lands there instead of the business DB (ADR-0057). Opt out with
142+
`OS_TELEMETRY_DB=0`, or point it elsewhere (any mode, including `serve`)
143+
with `OS_TELEMETRY_DB=<path>`.
144+
137145
#### `os serve`
138146

139147
Starts the ObjectStack server with automatic plugin discovery:
@@ -195,6 +203,23 @@ export default defineStack({
195203
});
196204
```
197205

206+
#### `os db clean`
207+
208+
Reclaims SQLite free space with a one-time `VACUUM` (ADR-0057 §3.4). The
209+
platform reclaims space incrementally (`auto_vacuum=INCREMENTAL`), but that
210+
setting only takes effect on a fresh database — files created before it
211+
stay pinned at their high-water mark until one full `VACUUM` rebuilds them.
212+
Non-destructive: every row survives; free pages return to the OS. Cleans
213+
the telemetry sibling too when one exists.
214+
215+
```bash
216+
os db clean # default: the per-project dev DB
217+
os db clean --database file:./data/app.db # explicit target
218+
```
219+
220+
**Options:**
221+
- `-d, --database <url>` — SQLite database URL/path (defaults to `$OS_DATABASE_URL`, then the per-project dev DB)
222+
198223
#### Console UI
199224

200225
Launch the development server with the Console UI:

docs/launch-readiness.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -160,6 +160,15 @@ fix or acceptance.**
160160
`AutomationServicePluginOptions.maxLogSize` (default unchanged at 1000,
161161
`DEFAULT_MAX_EXECUTION_LOG_SIZE`); +2 tests. Durable `sys_automation_run`-style
162162
persistence is deferred to the HA fast-follow (roadmap), not a GA blocker.
163+
- **Superseded (ADR-0057, #2786/#2834):** the plugin-local sweepers above were
164+
the stop-gap. Retention is now a *declarative platform primitive*: objects
165+
carry a `lifecycle` block (`sys_job_run` 30d, notification pipeline 90d) and
166+
the ObjectQL-registered **LifecycleService** is the ONE sweeper —
167+
`JobRunRetention` / `NotificationRetention` and their `retentionDays`
168+
options were removed. Windows are tuned via the `lifecycle` settings
169+
namespace (`retention_overrides`, tenant-scoped). `sys_automation_run`
170+
deliberately keeps its OWN terminal-only sweep (suspended runs are live
171+
resumable state; the declarative contract has no status predicate).
163172
- **Owner:** _______ · Verify ✅ (confirmed real @ `main`; scope corrected) · Sign-off ☐ · Notes: `sys_job_run` retention is the one true fix; messaging default-flipped; automation already bounded (now tunable). Awaiting human sign-off.
164173

165174
### P1-3 — Graceful shutdown (mostly a false positive; one real drain bug fixed)
Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import { Command, Flags } from '@oclif/core';
4+
import { statSync, existsSync } from 'node:fs';
5+
import chalk from 'chalk';
6+
import { printError } from '../../utils/format.js';
7+
import { resolveDefaultDevDbUrl } from '../dev.js';
8+
import { resolveTelemetryDbPath } from '../../utils/telemetry-datasource.js';
9+
10+
/**
11+
* `os db clean` — one-time SQLite space reclamation (ADR-0057 §3.4).
12+
*
13+
* The platform LifecycleService reclaims space incrementally
14+
* (`auto_vacuum=INCREMENTAL` + `PRAGMA incremental_vacuum` after sweeps) —
15+
* but `auto_vacuum` only changes the page layout of a FRESH database, so a
16+
* file created before the ADR-0057 default stays pinned at its high-water
17+
* mark until one full `VACUUM` rebuilds it. This command runs exactly that:
18+
* set the pragma, `VACUUM`, report the reclaimed bytes. Non-destructive —
19+
* every row survives; only free pages are returned to the OS.
20+
*/
21+
export default class DbClean extends Command {
22+
static override description =
23+
'Reclaim SQLite free space (one-time VACUUM) so legacy files adopt auto_vacuum=INCREMENTAL — ADR-0057 §3.4';
24+
25+
static override examples = [
26+
'$ os db clean',
27+
'$ os db clean --database file:./.objectstack/data/dev.db',
28+
];
29+
30+
static override flags = {
31+
database: Flags.string({
32+
char: 'd',
33+
description: 'SQLite database URL/path (defaults to $OS_DATABASE_URL, then the per-project dev DB)',
34+
env: 'OS_DATABASE_URL',
35+
}),
36+
};
37+
38+
async run(): Promise<void> {
39+
const { flags } = await this.parse(DbClean);
40+
41+
const raw =
42+
flags.database?.trim() ||
43+
resolveDefaultDevDbUrl({ env: process.env, cwd: process.cwd() }) ||
44+
'';
45+
const primary = raw.replace(/^file:/i, '').replace(/^sqlite:/i, '');
46+
47+
if (!primary || primary === ':memory:' || primary.startsWith(':')) {
48+
printError('No file-backed SQLite database to clean (in-memory databases reclaim nothing).');
49+
this.exit(1);
50+
return;
51+
}
52+
if (!/\.(db|sqlite3|sqlite)$/i.test(primary) && !existsSync(primary)) {
53+
printError(`Not a SQLite database path: ${primary}`);
54+
this.exit(1);
55+
return;
56+
}
57+
58+
// Clean the telemetry sibling too when one exists (ADR-0057 §3.6).
59+
const telemetrySibling = resolveTelemetryDbPath({ primaryPath: primary, env: process.env, dev: true });
60+
const targets = [primary, telemetrySibling].filter(
61+
(p): p is string => !!p && existsSync(p),
62+
);
63+
if (targets.length === 0) {
64+
printError(`Database file not found: ${primary}`);
65+
this.exit(1);
66+
return;
67+
}
68+
69+
const { resolveSqliteDriver } = await import('@objectstack/service-datasource');
70+
71+
let failed = false;
72+
for (const file of targets) {
73+
const before = statSync(file).size;
74+
try {
75+
const resolved = await resolveSqliteDriver({
76+
filename: file,
77+
dev: true, // allow the wasm step-down; memory fallback is rejected below
78+
warn: (m) => console.warn(chalk.yellow(m)),
79+
});
80+
if (resolved.engine === 'memory') {
81+
throw new Error('no SQLite engine available (native and wasm both failed to load)');
82+
}
83+
// Order matters: the pragma must be set BEFORE the VACUUM so the
84+
// rebuilt file carries auto_vacuum=INCREMENTAL from here on.
85+
await resolved.driver.execute('PRAGMA auto_vacuum = INCREMENTAL');
86+
await resolved.driver.execute('VACUUM');
87+
await resolved.driver.disconnect();
88+
89+
const after = statSync(file).size;
90+
const saved = before - after;
91+
const fmt = (n: number) => `${(n / 1024 / 1024).toFixed(2)} MB`;
92+
console.log(
93+
`${chalk.green('✓')} ${file}: ${fmt(before)}${fmt(after)}` +
94+
(saved > 0 ? chalk.dim(` (reclaimed ${fmt(saved)})`) : chalk.dim(' (already compact)')),
95+
);
96+
} catch (error: any) {
97+
failed = true;
98+
printError(`VACUUM failed for ${file}: ${error?.message ?? error}`);
99+
}
100+
}
101+
if (failed) this.exit(1);
102+
}
103+
}

packages/cli/src/commands/serve.ts

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -664,6 +664,38 @@ export default class Serve extends Command {
664664
trackPlugin(resolved.engine === 'memory' ? 'MemoryDriver' : resolved.engine === 'sqlite-wasm' ? 'SqliteWasmDriver' : 'SqlDriver');
665665
resolvedDriverLabel = resolved.label;
666666
resolvedDatabaseUrl = resolved.engine === 'memory' ? '(in-memory)' : (databaseUrl ?? ':memory:');
667+
668+
// ADR-0057 §3.6 (#2834 ②): provision the dedicated `telemetry`
669+
// datasource — a sibling SQLite file the engine routes every
670+
// telemetry/event/audit-classed object to, so platform-generated
671+
// growth can never again bloat the business DB. Dev default-on
672+
// for file-backed primaries; `OS_TELEMETRY_DB=0` opts out,
673+
// `OS_TELEMETRY_DB=<path>` opts in anywhere (incl. serve).
674+
if (resolved.engine !== 'memory') {
675+
const { resolveTelemetryDbPath } = await import('../utils/telemetry-datasource.js');
676+
const telemetryPath = resolveTelemetryDbPath({ primaryPath: filePath, env: process.env, dev: isDev });
677+
if (telemetryPath) {
678+
try {
679+
const telemetry = await resolveSqliteDriver({
680+
filename: telemetryPath,
681+
dev: isDev,
682+
autoMigrate: isDev ? 'safe' : undefined,
683+
warn: (m) => console.warn(chalk.yellow(m)),
684+
});
685+
if (telemetry.engine !== 'memory') {
686+
// The engine keys datasources by driver name — the
687+
// lifecycle router looks this exact name up.
688+
Object.defineProperty(telemetry.driver, 'name', { value: 'telemetry' });
689+
await kernel.use(new DriverPlugin(telemetry.driver, { datasourceName: 'telemetry', registerAsDefault: false }));
690+
trackPlugin('TelemetryDatasource');
691+
console.log(chalk.dim(` telemetry datasource: ${telemetryPath} (lifecycle-classed system data; OS_TELEMETRY_DB=0 to disable)`));
692+
}
693+
} catch {
694+
// Best-effort: a failed telemetry provision must never block
695+
// boot — objects simply stay on the primary datasource.
696+
}
697+
}
698+
}
667699
} else if (driverType === 'sqlite-wasm' || driverType === 'wasm-sqlite' || driverType === 'wasm') {
668700
const { SqliteWasmDriver } = await import('@objectstack/driver-sqlite-wasm');
669701
const filePath = (databaseUrl ?? ':memory:').replace(/^file:/, '').replace(/^wasm-sqlite:\/\//, '').replace(/^sqlite:/, '');
Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import { describe, it, expect } from 'vitest';
4+
import { resolveTelemetryDbPath } from './telemetry-datasource.js';
5+
6+
describe('resolveTelemetryDbPath (ADR-0057 §3.6)', () => {
7+
it('derives a sibling file next to a file-backed dev primary', () => {
8+
expect(resolveTelemetryDbPath({ primaryPath: '/p/.objectstack/data/dev.db', env: {}, dev: true }))
9+
.toBe('/p/.objectstack/data/dev.telemetry.db');
10+
expect(resolveTelemetryDbPath({ primaryPath: './app.sqlite', env: {}, dev: true }))
11+
.toBe('./app.telemetry.sqlite');
12+
expect(resolveTelemetryDbPath({ primaryPath: 'data/store', env: {}, dev: true }))
13+
.toBe('data/store.telemetry.db');
14+
});
15+
16+
it('never derives one for in-memory primaries', () => {
17+
expect(resolveTelemetryDbPath({ primaryPath: ':memory:', env: {}, dev: true })).toBeUndefined();
18+
expect(resolveTelemetryDbPath({ primaryPath: '', env: {}, dev: true })).toBeUndefined();
19+
});
20+
21+
it('is opt-in outside dev', () => {
22+
expect(resolveTelemetryDbPath({ primaryPath: '/srv/prod.db', env: {}, dev: false })).toBeUndefined();
23+
expect(
24+
resolveTelemetryDbPath({ primaryPath: '/srv/prod.db', env: { OS_TELEMETRY_DB: '/srv/prod.telemetry.db' }, dev: false }),
25+
).toBe('/srv/prod.telemetry.db');
26+
});
27+
28+
it('OS_TELEMETRY_DB=0|false|off opts out entirely, even in dev', () => {
29+
for (const off of ['0', 'false', 'off', 'OFF']) {
30+
expect(resolveTelemetryDbPath({ primaryPath: '/p/dev.db', env: { OS_TELEMETRY_DB: off }, dev: true })).toBeUndefined();
31+
}
32+
});
33+
34+
it('an explicit OS_TELEMETRY_DB path wins over derivation and strips url prefixes', () => {
35+
expect(
36+
resolveTelemetryDbPath({ primaryPath: '/p/dev.db', env: { OS_TELEMETRY_DB: 'file:/tmp/t.db' }, dev: true }),
37+
).toBe('/tmp/t.db');
38+
});
39+
});
Lines changed: 44 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,44 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* ADR-0057 §3.6 (#2834 ②) — where the dedicated telemetry datasource lives.
5+
*
6+
* When a datasource named `telemetry` is registered, the engine routes every
7+
* `telemetry`/`event`/`audit`-classed object to it, so platform-generated
8+
* growth can never again bloat the business DB. This helper decides whether
9+
* (and where) the CLI should provision that second SQLite file:
10+
*
11+
* - `OS_TELEMETRY_DB=0|false|off` → never (explicit opt-out)
12+
* - `OS_TELEMETRY_DB=<path>` → always, at that path (dev AND serve)
13+
* - dev mode + file-backed primary → default ON, `<primary>.telemetry.<ext>`
14+
* (`dev.db` → `dev.telemetry.db`, same directory)
15+
* - everything else (prod serve, `:memory:`, non-sqlite) → off
16+
*
17+
* Production stays opt-in: a second file appearing next to a prod database
18+
* is a deployment-topology change an operator should choose, not inherit.
19+
*/
20+
export function resolveTelemetryDbPath(opts: {
21+
/** Primary sqlite file path (already stripped of `file:`/`sqlite:`). */
22+
primaryPath: string;
23+
env: Record<string, string | undefined>;
24+
dev: boolean;
25+
}): string | undefined {
26+
const raw = opts.env.OS_TELEMETRY_DB?.trim();
27+
if (raw) {
28+
const lowered = raw.toLowerCase();
29+
if (lowered === '0' || lowered === 'false' || lowered === 'off') return undefined;
30+
return raw.replace(/^file:/i, '').replace(/^sqlite:/i, '');
31+
}
32+
33+
if (!opts.dev) return undefined;
34+
35+
const primary = opts.primaryPath.trim();
36+
// Only a real on-disk primary gets a sibling: separating one `:memory:`
37+
// store into another has no reclamation value.
38+
if (!primary || primary === ':memory:' || primary.startsWith(':')) return undefined;
39+
40+
if (/\.(db|sqlite3|sqlite)$/i.test(primary)) {
41+
return primary.replace(/\.(db|sqlite3|sqlite)$/i, '.telemetry.$1');
42+
}
43+
return `${primary}.telemetry.db`;
44+
}

0 commit comments

Comments
 (0)