Skip to content

Commit 851530d

Browse files
committed
Add swappable storage, settings UI, and tests
Introduce a swappable storage proxy and settings-driven live configuration: adds SwappableStorageService (proxy that can swap inner adapter), wiring in StorageServicePlugin to bind to the `storage` Settings namespace and rebuild the adapter on changes (with a logged warning that files are not migrated). Adds a storage.manifest (with a fallback storage/test action handler exported from service-settings) and translations (en, ja-JP, zh-CN), plus manifest/unit tests. Storage plugin now registers a live storage/test probe action that uploads → downloads → deletes a probe blob to validate configs; exposes bindToSettings option to disable live binding for tests. Also updated exports, README docs (apps/objectos, cli, service-storage) to document UI-driven configuration and migration caveats. Includes unit/integration tests for the plugin and swappable proxy and various implementation fixes when building adapters from settings.
1 parent db4f939 commit 851530d

17 files changed

Lines changed: 977 additions & 40 deletions

apps/objectos/README.md

Lines changed: 13 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -441,16 +441,26 @@ plugins (in this order) via `mountDefaultProjectPlugins()`:
441441
tenants are fully isolated
442442
5. `EmailServicePlugin` — each tenant configures its own provider /
443443
api_key via Settings; api_key is stored encrypted in `sys_secret`
444-
6. `StorageServicePlugin` — see below
444+
6. `StorageServicePlugin` — see below. Adapter + credentials are
445+
live-configurable per tenant via the `storage` Settings namespace;
446+
the plugin swaps the inner adapter on every change.
445447

446448
### Storage adapter selection
447449

448450
- `OS_STORAGE_ADAPTER=s3` + `OS_S3_*` env → shared S3 bucket with
449451
`pathStylePrefix: projects/<projectId>` so tenant prefixes never
450-
collide
452+
collide. Per-project Settings can still override the bucket /
453+
credentials at runtime.
451454
- otherwise → local driver under
452455
`<dataRoot>/projects/<projectId>/uploads/`; a single boot warning
453-
fires in non-dev mode
456+
fires in non-dev mode.
457+
458+
Tenant admins can switch adapter from the Settings hub
459+
(`namespace=storage`) — the `SwappableStorageService` proxy rebuilds
460+
the inner adapter without restarting the kernel. ⚠ Existing files are
461+
**not** migrated; a warning is logged on every swap. The
462+
`storage/test` action uploads → reads → deletes a probe blob to
463+
validate the new configuration before users start uploading.
454464

455465
### Ops overrides
456466

packages/cli/README.md

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -265,7 +265,7 @@ service plugins for every preset **except** `--preset minimal`:
265265
| `queue` | `QueueServicePlugin` | Background retries (mail, audit batching) need a queue. |
266266
| `job` | `JobServicePlugin` | Scheduled jobs (cleanup, reports, cron). |
267267
| `cache` | `CacheServicePlugin` | In-memory adapter; performance baseline. |
268-
| `settings` | `SettingsServicePlugin` | Mail/branding/feature-flag UI + persistence. |
268+
| `settings` | `SettingsServicePlugin` | Mail / storage / branding / feature-flag UI + persistence. |
269269
| `email` | `EmailServicePlugin` | Auth callbacks (verify, reset, magic-link) hard-depend. |
270270
| `storage` | `StorageServicePlugin` | Avatars, attachments, exports. Local fallback in dev. |
271271

@@ -274,10 +274,17 @@ Apps no longer need to list these in `requires: [...]`.
274274
**Opt-out:** run `objectstack serve --preset minimal` to skip the slate
275275
and only load core + your explicit `requires`.
276276

277+
**Built-in Settings manifests:** `mail`, `storage`, `branding`,
278+
`feature_flags` are pre-registered. Both `mail` and `storage` expose a
279+
"Test connection" action that exercises the live transport / adapter
280+
end-to-end. Switching the storage adapter does **not** migrate
281+
previously uploaded files — the plugin logs a warning on every swap.
282+
277283
**Storage warning:** when no `storage:` block is configured the plugin
278284
falls back to the local driver under `.objectstack/data/uploads/`. In
279285
non-dev mode a single `console.warn` fires at boot — switch to S3/GCS/
280-
Azure for production by setting `config.storage` or `OS_STORAGE_*` env.
286+
Azure for production by setting `config.storage`, `OS_STORAGE_*` env,
287+
or via the Settings hub (`namespace=storage`).
281288

282289
## License
283290

packages/services/service-settings/README.md

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -52,5 +52,11 @@ per-project kernel on hosted objectos. Apps no longer need to declare
5252
no runtime cost (the registry is empty, no routes fire).
5353

5454
The Settings hub in `apps/console` therefore appears in every app, and
55-
the **Mail Settings** card is the first manifest registered (by
56-
`EmailServicePlugin`, also always-on).
55+
the following built-in manifests are pre-registered out of the box:
56+
57+
| Namespace | Owner plugin | Highlights |
58+
|----------------|-----------------------------|---------------------------------------------|
59+
| `mail` | `EmailServicePlugin` | SMTP / Resend / Postmark + `mail/test` |
60+
| `storage` | `StorageServicePlugin` | Local FS / S3 + encrypted secret + `storage/test` |
61+
| `branding` | (built-in fallback) | Workspace name, logo, accent colour |
62+
| `feature_flags`| (built-in fallback) | Opt-in experimental features |

packages/services/service-settings/src/index.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -46,6 +46,8 @@ export {
4646
featureFlagsSettingsManifest,
4747
mailSettingsManifest,
4848
mailTestActionHandler,
49+
storageSettingsManifest,
50+
storageTestActionHandler,
4951
} from './manifests/index.js';
5052

5153
// Re-export the spec types for convenience so plugin authors only need

packages/services/service-settings/src/manifests/index.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4,14 +4,17 @@
44
export { mailSettingsManifest, mailTestActionHandler } from './mail.manifest.js';
55
export { brandingSettingsManifest } from './branding.manifest.js';
66
export { featureFlagsSettingsManifest } from './feature-flags.manifest.js';
7+
export { storageSettingsManifest, storageTestActionHandler } from './storage.manifest.js';
78

89
import { mailSettingsManifest } from './mail.manifest.js';
910
import { brandingSettingsManifest } from './branding.manifest.js';
1011
import { featureFlagsSettingsManifest } from './feature-flags.manifest.js';
12+
import { storageSettingsManifest } from './storage.manifest.js';
1113

1214
/** Convenience aggregate — pass to `SettingsServicePlugin({ manifests })`. */
1315
export const builtinSettingsManifests = [
1416
brandingSettingsManifest,
1517
mailSettingsManifest,
18+
storageSettingsManifest,
1619
featureFlagsSettingsManifest,
1720
];
Lines changed: 90 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,90 @@
1+
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import { describe, it, expect } from 'vitest';
4+
import { SettingsManifestSchema } from '@objectstack/spec/system';
5+
import { storageSettingsManifest, storageTestActionHandler } from './storage.manifest';
6+
7+
describe('storageSettingsManifest', () => {
8+
it('parses against SettingsManifestSchema', () => {
9+
expect(() => SettingsManifestSchema.parse(storageSettingsManifest)).not.toThrow();
10+
});
11+
12+
it('declares namespace=storage, scope=global, version=1', () => {
13+
const parsed = SettingsManifestSchema.parse(storageSettingsManifest);
14+
expect(parsed.namespace).toBe('storage');
15+
expect(parsed.scope).toBe('global');
16+
expect(parsed.version).toBe(1);
17+
});
18+
19+
it('marks s3_secret_access_key as encrypted', () => {
20+
const sk = (storageSettingsManifest.specifiers as any[]).find(
21+
(s) => s.key === 's3_secret_access_key',
22+
);
23+
expect(sk).toBeDefined();
24+
expect(sk.encrypted).toBe(true);
25+
expect(sk.type).toBe('password');
26+
});
27+
28+
it('exposes a test action that POSTs to /api/settings/storage/test', () => {
29+
const test = (storageSettingsManifest.specifiers as any[]).find(
30+
(s) => s.type === 'action_button' && s.id === 'test',
31+
);
32+
expect(test).toBeDefined();
33+
expect(test.handler).toMatchObject({
34+
kind: 'http',
35+
method: 'POST',
36+
url: '/api/settings/storage/test',
37+
});
38+
});
39+
40+
it('hides S3 fields when adapter=local and vice-versa via visible expressions', () => {
41+
const specs = storageSettingsManifest.specifiers as any[];
42+
const s3Fields = ['s3_bucket', 's3_region', 's3_access_key_id', 's3_secret_access_key'];
43+
for (const key of s3Fields) {
44+
const spec = specs.find((s) => s.key === key);
45+
expect(spec).toBeDefined();
46+
expect(spec.visible).toBe("${data.adapter === 's3'}");
47+
}
48+
const localRoot = specs.find((s) => s.key === 'local_root');
49+
expect(localRoot.visible).toBe("${data.adapter === 'local'}");
50+
});
51+
});
52+
53+
describe('storageTestActionHandler (fallback)', () => {
54+
it('rejects local adapter without local_root', async () => {
55+
const r = await storageTestActionHandler({ values: { adapter: 'local' }, ctx: {} as any });
56+
expect(r.ok).toBe(false);
57+
expect(r.severity).toBe('error');
58+
});
59+
60+
it('accepts local adapter when local_root is set', async () => {
61+
const r = await storageTestActionHandler({
62+
values: { adapter: 'local', local_root: './uploads' },
63+
ctx: {} as any,
64+
});
65+
expect(r.ok).toBe(true);
66+
});
67+
68+
it('rejects s3 adapter when credentials are missing', async () => {
69+
const r = await storageTestActionHandler({
70+
values: { adapter: 's3', s3_bucket: 'x' },
71+
ctx: {} as any,
72+
});
73+
expect(r.ok).toBe(false);
74+
expect(r.message).toMatch(/s3_region|s3_access_key_id|s3_secret_access_key/);
75+
});
76+
77+
it('accepts a fully-specified s3 config', async () => {
78+
const r = await storageTestActionHandler({
79+
values: {
80+
adapter: 's3',
81+
s3_bucket: 'b',
82+
s3_region: 'us-east-1',
83+
s3_access_key_id: 'A',
84+
s3_secret_access_key: 'S',
85+
},
86+
ctx: {} as any,
87+
});
88+
expect(r.ok).toBe(true);
89+
});
90+
});
Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import type { SettingsManifest } from '@objectstack/spec/system';
4+
import type { SettingsActionHandler } from '../settings-service.types.js';
5+
6+
// Mirrors the shape of `mail.manifest.ts`. The actual adapter rebuild
7+
// + `storage/test` probe live in `@objectstack/service-storage`; this
8+
// manifest only declares the form + acts as a safe fallback when the
9+
// storage plugin is not present.
10+
const manifest = {
11+
namespace: 'storage',
12+
version: 1,
13+
label: 'File Storage',
14+
icon: 'HardDrive',
15+
description:
16+
'Backend used for attachments, exports, and user uploads. ' +
17+
'⚠ Switching adapter does not migrate existing files — files ' +
18+
'uploaded under the previous adapter become unreachable through ' +
19+
'the new one.',
20+
scope: 'global',
21+
readPermission: 'setup.access',
22+
writePermission: 'setup.write',
23+
category: 'Infrastructure',
24+
order: 20,
25+
specifiers: [
26+
{ type: 'group', id: 'adapter', label: 'Backend', required: false,
27+
description: 'Choose where uploaded files are stored.' },
28+
{ type: 'select', key: 'adapter', label: 'Adapter', required: true, default: 'local',
29+
options: [
30+
{ value: 'local', label: 'Local filesystem' },
31+
{ value: 's3', label: 'S3 / S3-compatible' },
32+
],
33+
},
34+
35+
{ type: 'group', id: 'local', label: 'Local', required: false,
36+
visible: "${data.adapter === 'local'}" },
37+
{ type: 'text', key: 'local_root', label: 'Root directory', required: false,
38+
default: './.objectstack/data/uploads',
39+
description: 'Filesystem path under which files are stored. Relative paths resolve from the server CWD.',
40+
visible: "${data.adapter === 'local'}" },
41+
42+
{ type: 'group', id: 's3', label: 'S3', required: false,
43+
visible: "${data.adapter === 's3'}" },
44+
{ type: 'text', key: 's3_bucket', label: 'Bucket', required: true,
45+
description: 'Shared host bucket. Per-project files are namespaced via the projects/<projectId>/ prefix.',
46+
visible: "${data.adapter === 's3'}" },
47+
{ type: 'text', key: 's3_region', label: 'Region', required: true,
48+
description: 'Example: us-east-1',
49+
visible: "${data.adapter === 's3'}" },
50+
{ type: 'text', key: 's3_endpoint', label: 'Endpoint', required: false,
51+
description: 'Custom endpoint for S3-compatible providers (R2, MinIO, Wasabi). Leave blank for AWS S3.',
52+
visible: "${data.adapter === 's3'}" },
53+
{ type: 'text', key: 's3_access_key_id', label: 'Access key ID', required: true,
54+
visible: "${data.adapter === 's3'}" },
55+
{ type: 'password', key: 's3_secret_access_key', label: 'Secret access key',
56+
required: true, encrypted: true,
57+
visible: "${data.adapter === 's3'}" },
58+
{ type: 'toggle', key: 's3_force_path_style', label: 'Force path-style URLs',
59+
required: false, default: false,
60+
description: 'Enable for MinIO and most S3-compatible providers; disable for AWS S3.',
61+
visible: "${data.adapter === 's3'}" },
62+
63+
{ type: 'group', id: 'limits', label: 'Limits', required: false },
64+
{ type: 'number', key: 'presigned_ttl', label: 'Presigned URL TTL (seconds)',
65+
required: false, default: 3600, min: 60, max: 604800 },
66+
{ type: 'number', key: 'session_ttl', label: 'Upload session TTL (seconds)',
67+
required: false, default: 86400, min: 300, max: 604800,
68+
description: 'How long a chunked-upload session stays resumable.' },
69+
{ type: 'number', key: 'max_upload_mb', label: 'Max upload size (MB)',
70+
required: false, default: 100, min: 1, max: 10240 },
71+
72+
{ type: 'action_button', id: 'test', label: 'Test connection',
73+
required: false, icon: 'Plug',
74+
handler: { kind: 'http', method: 'POST', url: '/api/settings/storage/test' } },
75+
],
76+
};
77+
78+
/** File Storage — local FS / S3-compatible backend configuration. */
79+
export const storageSettingsManifest = manifest as unknown as SettingsManifest;
80+
81+
/**
82+
* Built-in fallback action handler for `storage/test`. The real
83+
* implementation lives in `@objectstack/service-storage` and is
84+
* registered by `StorageServicePlugin` on `kernel:ready` (it overrides
85+
* this stub via `registerAction`). This fallback only validates the
86+
* form so the button is still useful when the storage plugin is
87+
* absent (e.g. in a unit-test kernel that mounts settings only).
88+
*/
89+
export const storageTestActionHandler: SettingsActionHandler = async ({ values }) => {
90+
const adapter = String(values.adapter ?? 'local');
91+
if (adapter === 'local') {
92+
const root = values.local_root as string | undefined;
93+
if (!root) {
94+
return { ok: false, severity: 'error', message: 'Configure a root directory before testing.' };
95+
}
96+
return {
97+
ok: true,
98+
severity: 'info',
99+
message: `Local adapter configured (root=${root}). Mount @objectstack/service-storage to exercise live I/O.`,
100+
};
101+
}
102+
if (adapter === 's3') {
103+
const missing: string[] = [];
104+
if (!values.s3_bucket) missing.push('s3_bucket');
105+
if (!values.s3_region) missing.push('s3_region');
106+
if (!values.s3_access_key_id) missing.push('s3_access_key_id');
107+
if (!values.s3_secret_access_key) missing.push('s3_secret_access_key');
108+
if (missing.length) {
109+
return { ok: false, severity: 'error', message: `Missing required field${missing.length > 1 ? 's' : ''}: ${missing.join(', ')}` };
110+
}
111+
return {
112+
ok: true,
113+
severity: 'info',
114+
message: `S3 adapter configured (bucket=${values.s3_bucket}, region=${values.s3_region}). Mount @objectstack/service-storage to exercise live I/O.`,
115+
};
116+
}
117+
return { ok: false, severity: 'error', message: `Unknown adapter: ${adapter}` };
118+
};

packages/services/service-settings/src/settings-service-plugin.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ import {
1818
import {
1919
builtinSettingsManifests,
2020
mailTestActionHandler,
21+
storageTestActionHandler,
2122
} from './manifests/index.js';
2223
import { settingsBuiltinTranslations } from './translations/index.js';
2324

@@ -78,6 +79,7 @@ export class SettingsServicePlugin implements Plugin {
7879
manifests: opts.manifests ?? builtinSettingsManifests,
7980
actionHandlers: opts.actionHandlers ?? {
8081
mail: { test: mailTestActionHandler },
82+
storage: { test: storageTestActionHandler },
8183
},
8284
};
8385
}

packages/services/service-settings/src/translations/en.ts

Lines changed: 40 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ import type { TranslationData } from '@objectstack/spec/system';
55
/**
66
* English (en) — built-in settings manifest translations.
77
*
8-
* Mirrors literals in `manifests/{mail,branding,feature-flags}.manifest.ts`.
8+
* Mirrors literals in `manifests/{mail,branding,feature-flags,storage}.manifest.ts`.
99
* Keeping them explicit here lets the resolver chain (locale → fallback → literal)
1010
* always have at least an English entry to fall back to.
1111
*/
@@ -89,5 +89,44 @@ export const en: TranslationData = {
8989
inline_comments: { label: 'Inline comments' },
9090
},
9191
},
92+
93+
storage: {
94+
title: 'File Storage',
95+
description:
96+
'Backend used for attachments, exports, and user uploads. ' +
97+
'⚠ Switching adapter does not migrate existing files — files ' +
98+
'uploaded under the previous adapter become unreachable through ' +
99+
'the new one.',
100+
groups: {
101+
adapter: { title: 'Backend', description: 'Choose where uploaded files are stored.' },
102+
local: { title: 'Local' },
103+
s3: { title: 'S3' },
104+
limits: { title: 'Limits' },
105+
},
106+
keys: {
107+
adapter: {
108+
label: 'Adapter',
109+
options: { local: 'Local filesystem', s3: 'S3 / S3-compatible' },
110+
},
111+
local_root: { label: 'Root directory',
112+
help: 'Filesystem path under which files are stored. Relative paths resolve from the server CWD.' },
113+
s3_bucket: { label: 'Bucket',
114+
help: 'Shared host bucket. Per-project files are namespaced via the projects/<projectId>/ prefix.' },
115+
s3_region: { label: 'Region', help: 'Example: us-east-1' },
116+
s3_endpoint: { label: 'Endpoint',
117+
help: 'Custom endpoint for S3-compatible providers (R2, MinIO, Wasabi). Leave blank for AWS S3.' },
118+
s3_access_key_id: { label: 'Access key ID' },
119+
s3_secret_access_key: { label: 'Secret access key' },
120+
s3_force_path_style: { label: 'Force path-style URLs',
121+
help: 'Enable for MinIO and most S3-compatible providers; disable for AWS S3.' },
122+
presigned_ttl: { label: 'Presigned URL TTL (seconds)' },
123+
session_ttl: { label: 'Upload session TTL (seconds)',
124+
help: 'How long a chunked-upload session stays resumable.' },
125+
max_upload_mb: { label: 'Max upload size (MB)' },
126+
},
127+
actions: {
128+
test: { label: 'Test connection' },
129+
},
130+
},
92131
},
93132
};

0 commit comments

Comments
 (0)