Skip to content

Commit a27cb6a

Browse files
hotlongCopilot
andcommitted
feat(plugin-audit): auto-write sys_audit_log + sys_activity on data mutations (M10.1)
Subscribes to ObjectQL wildcard beforeUpdate/beforeDelete/afterInsert/ afterUpdate/afterDelete hooks. Captures previous-state snapshots in before-hooks (via api.sudo().findOne) and writes immutable audit rows with field-level JSON diffs plus human-readable activity entries. - Filters noise fields (created_at/updated_at/created_by/updated_by) out of update diffs. - Skips writes when an update produces no meaningful field change. - Skips the audit/activity/comment/session/presence/auth tables themselves to prevent recursion. - Hook errors are swallowed (logged) so user CRUD is never broken. Verified end-to-end in the CRM example: create→update→delete on the same lead produces 3 audit rows with the expected before/after diffs and a parallel activity feed with human summaries. Closes ROADMAP M10.1. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
1 parent a29e452 commit a27cb6a

4 files changed

Lines changed: 293 additions & 2 deletions

File tree

ROADMAP.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -533,6 +533,53 @@ D9 / D10 / D11 (resolved through M9 sub-tasks above)
533533

534534
---
535535

536+
## M10 — CRM Production-Readiness (App-Layer + Platform Gaps)
537+
538+
> Derived from the 2026-05-18 real-customer audit of `examples/app-crm`.
539+
> Full report: `~/.copilot/session-state/.../files/production-readiness-assessment.md`.
540+
> Goal: take the CRM example from "single-user demo" to "5-20 person sales team pilot".
541+
542+
### M10 P0 — Blockers (must-have for any paying customer)
543+
544+
- [ ] **M10.1 — Audit / Activity auto-writers.** `plugin-audit` registers `sys_audit_log` but never writes; `sys_activity` and `sys_comment` likewise empty after CRUD. Subscribe to ObjectQL `data:*` events from `EventBus` and emit immutable audit rows + human-readable activity entries with field-level diffs. Anchor: [packages/plugins/plugin-audit/src/audit-plugin.ts](packages/plugins/plugin-audit/src/audit-plugin.ts).
545+
- [ ] **M10.2 — User invite + default roles.** No `/api/v1/admin/users/invite` endpoint exists. Wrap better-auth `organization.inviteMember` as a first-class REST route. Seed three default roles into `sys_role` (`admin`, `sales_manager`, `sales_rep`). Anchor: [packages/plugins/plugin-auth/](packages/plugins/plugin-auth/).
546+
- [ ] **M10.3 — Attachments.** No `sys_attachment` object, no upload component. Register the schema in `packages/spec/src/system/`, wire it to `service-storage`, add an `AttachmentList` view widget. P0 because every CRM contract/quote/PDF is currently un-storable.
547+
- [ ] **M10.4 — Validation envelope + Zod-at-rest.** Currently a malformed POST returns raw SQL text. Wrap REST handlers with a structured error envelope (`{code,message,fields[]}`) and run the canonical Zod schema inside ObjectQL `insert/update` before touching the driver. Anchors: [packages/rest/src/rest-server.ts](packages/rest/src/rest-server.ts), [packages/objectql/](packages/objectql/).
548+
- [ ] **M10.5 — Global search.** Header already shows `⌘K` but it is inert. Add `GET /api/v1/search?q=` (driver-side `LIKE` across registered searchable fields; FTS later) and wire the Studio command palette to it.
549+
550+
### M10 P1 — Critical (80%+ of customers need these)
551+
552+
- [ ] M10.6 — `POST /api/v1/data/lead/:id/convert` → atomic Lead → Account + Contact + Opportunity.
553+
- [ ] M10.7 — `service-email` + `sys_email` (IMAP/SMTP, OAuth Gmail/Outlook), thread linking to records.
554+
- [ ] M10.8 — `sys_notification` via existing `service-feed` + WebSocket via `service-realtime`; header bell badge.
555+
- [ ] M10.9 — `POST /api/v1/data/:obj/import` CSV/XLSX parser with dry-run + field mapping.
556+
- [ ] M10.10 — Comments / @mentions UI on detail pages (table `sys_comment` already exists).
557+
- [ ] M10.11 — Activity-timeline component (consumes M10.1 data).
558+
- [ ] M10.12 — `full_name` CEL formula bug (leading space when `salutation` is null).
559+
- [ ] M10.13 — Date / datetime column formatters in grid plugin (currently displays raw ISO).
560+
- [ ] M10.14 — Stop SQL error leakage in REST error responses.
561+
562+
### M10 P2 — Important (50%+ of customers need these)
563+
564+
- [ ] M10.15 — Workflow / approval engine (`sys_workflow`).
565+
- [ ] M10.16 — Saved reports + scheduled email (`sys_report`).
566+
- [ ] M10.17 — Record-level sharing rules (`sys_sharing_rule`) + team hierarchy (`sys_team`).
567+
- [ ] M10.18 — Tags (`sys_tag`).
568+
- [ ] M10.19 — Re-enable GraphQL (currently 501) and OpenAPI spec endpoint (currently 404).
569+
- [ ] M10.20 — Realtime channels (collaborative editing on the same opportunity).
570+
- [ ] M10.21 — Outbound webhooks + CSV export.
571+
- [ ] M10.22 — Mobile-optimized layouts (kanban + grid).
572+
- [ ] M10.23 — i18n review for CRM business terms (pipeline / forecast / SLA).
573+
574+
### M10 P3 — Polish
575+
576+
- [ ] M10.24 — Soft delete + recycle bin.
577+
- [ ] M10.25 — Dark-mode visual parity audit.
578+
- [ ] M10.26 — WCAG 2.1 AA accessibility audit.
579+
- [ ] M10.27 — 10K+ record performance (virtual scrolling, indices).
580+
581+
---
582+
536583
## Related Documents
537584

538585
| Document | Role |

packages/plugins/plugin-audit/src/audit-plugin.ts

Lines changed: 26 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,13 +1,18 @@
11
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
22

33
import type { Plugin, PluginContext } from '@objectstack/core';
4+
import type { IDataEngine } from '@objectstack/spec/contracts';
45
import { SysAuditLog, SysActivity, SysComment } from '@objectstack/platform-objects/audit';
6+
import { installAuditWriters } from './audit-writers.js';
57

68
/**
79
* AuditPlugin
810
*
9-
* Registers the sys_audit_log system object with ObjectQL so it is
10-
* discoverable by the studio and available for CRUD operations.
11+
* Registers the sys_audit_log / sys_activity / sys_comment system objects
12+
* and installs ObjectQL hook subscribers that automatically write audit
13+
* trail + activity stream rows on every data mutation.
14+
*
15+
* Implements ROADMAP M10.1 (CRM production-readiness).
1116
*/
1217
export class AuditPlugin implements Plugin {
1318
name = 'com.objectstack.audit';
@@ -30,4 +35,23 @@ export class AuditPlugin implements Plugin {
3035

3136
ctx.logger.info('Audit Plugin initialized');
3237
}
38+
39+
async start(ctx: PluginContext): Promise<void> {
40+
// ObjectQL engine is only resolvable after the kernel is ready.
41+
ctx.hook('kernel:ready', async () => {
42+
let engine: IDataEngine | null = null;
43+
try {
44+
engine = ctx.getService<IDataEngine>('objectql');
45+
} catch {
46+
// Fallback alias used in some kernels.
47+
try { engine = ctx.getService<IDataEngine>('data'); } catch { /* ignore */ }
48+
}
49+
if (!engine) {
50+
ctx.logger.warn('AuditPlugin: ObjectQL engine not available — audit writers NOT installed');
51+
return;
52+
}
53+
installAuditWriters(engine as any, this.name);
54+
ctx.logger.info('AuditPlugin: audit + activity writers installed');
55+
});
56+
}
3357
}
Lines changed: 219 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,219 @@
1+
// Copyright (c) 2025 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
import type { HookContext } from '@objectstack/spec/data';
4+
import type { IDataEngine } from '@objectstack/spec/contracts';
5+
6+
/**
7+
* Audit writer hook installer.
8+
*
9+
* Subscribes to the ObjectQL engine's wildcard `before*` / `after*` lifecycle
10+
* events and writes:
11+
*
12+
* - `sys_audit_log` rows — immutable, compliance-grade entries with
13+
* field-level `old_value` / `new_value` diffs.
14+
* - `sys_activity` rows — denormalized, human-readable summaries shown
15+
* in the dashboard recent-activity feed and per-record timelines.
16+
*
17+
* Skip rules avoid recursion and noise:
18+
* - Never audit the audit/activity tables themselves.
19+
* - Never audit session/presence/auth tables (high-frequency, low value).
20+
* - Read-only operations (`afterFind`) are never audited.
21+
*
22+
* All writes go through `ctx.api.sudo()` so they bypass record-level
23+
* permissions and always succeed regardless of the calling user's RBAC.
24+
*/
25+
26+
/** Tables that are intentionally excluded from audit/activity writes. */
27+
const SKIP_OBJECTS = new Set<string>([
28+
'sys_audit_log',
29+
'sys_activity',
30+
'sys_comment',
31+
'sys_session',
32+
'sys_presence',
33+
'sys_account',
34+
'sys_account_session',
35+
'sys_account_verification',
36+
'sys_account_account',
37+
]);
38+
39+
/** Fields that are noise in diffs (always change, never user-meaningful). */
40+
const NOISE_FIELDS = new Set<string>([
41+
'updated_at',
42+
'updated_by',
43+
'created_at',
44+
'created_by',
45+
]);
46+
47+
/** Action name produced from a HookContext.event string. */
48+
function actionFor(event: string): 'create' | 'update' | 'delete' | null {
49+
if (event === 'afterInsert') return 'create';
50+
if (event === 'afterUpdate') return 'update';
51+
if (event === 'afterDelete') return 'delete';
52+
return null;
53+
}
54+
55+
/** Activity type produced from an audit action. */
56+
function activityTypeFor(action: 'create' | 'update' | 'delete'): 'created' | 'updated' | 'deleted' {
57+
return action === 'create' ? 'created' : action === 'update' ? 'updated' : 'deleted';
58+
}
59+
60+
/**
61+
* Compute the human-readable record label from a record by trying common
62+
* label fields. Falls back to record id.
63+
*/
64+
function recordLabel(record: any, id: string): string {
65+
if (!record || typeof record !== 'object') return id;
66+
const candidates = ['name', 'subject', 'title', 'full_name', 'label', 'first_name', 'company', 'email'];
67+
for (const k of candidates) {
68+
const v = record[k];
69+
if (typeof v === 'string' && v.trim()) return v.trim();
70+
}
71+
return id;
72+
}
73+
74+
/**
75+
* Compute a shallow JSON diff between two records. Returns only keys whose
76+
* value changed (and ignores keys in `NOISE_FIELDS`). Both sides are
77+
* serialisable via `JSON.stringify` — values that fail to serialise are
78+
* coerced to `String(value)`.
79+
*/
80+
function diff(before: Record<string, any>, after: Record<string, any>): { old: Record<string, any>; next: Record<string, any> } {
81+
const oldOut: Record<string, any> = {};
82+
const newOut: Record<string, any> = {};
83+
const keys = new Set<string>([...Object.keys(before || {}), ...Object.keys(after || {})]);
84+
for (const k of keys) {
85+
if (NOISE_FIELDS.has(k)) continue;
86+
const b = before?.[k];
87+
const a = after?.[k];
88+
if (safeStringify(b) !== safeStringify(a)) {
89+
oldOut[k] = b ?? null;
90+
newOut[k] = a ?? null;
91+
}
92+
}
93+
return { old: oldOut, next: newOut };
94+
}
95+
96+
function safeStringify(v: any): string {
97+
try { return JSON.stringify(v); } catch { return String(v); }
98+
}
99+
100+
/**
101+
* Install audit + activity writers on the given engine. Idempotent per
102+
* `packageId` — calling twice with the same id replaces the previous
103+
* registration.
104+
*/
105+
export function installAuditWriters(engine: any, packageId = 'com.objectstack.audit'): void {
106+
if (!engine || typeof engine.registerHook !== 'function') return;
107+
108+
// Remove any prior installation so we can safely re-install on hot reload.
109+
if (typeof engine.unregisterHooksByPackage === 'function') {
110+
engine.unregisterHooksByPackage(packageId);
111+
}
112+
113+
/**
114+
* beforeUpdate / beforeDelete: capture "previous" snapshot via api.sudo()
115+
* so we can compute the diff in the afterXxx hook. We attach the snapshot
116+
* to the context (`(ctx as any).__previous`) since `HookContext.previous`
117+
* is officially typed but not always populated by the engine itself.
118+
*/
119+
const captureBefore = async (ctx: HookContext) => {
120+
if (SKIP_OBJECTS.has(ctx.object)) return;
121+
const id = (ctx.input as any)?.id;
122+
if (!id) return; // bulk update/delete — too costly to snapshot every row here
123+
const api: any = (ctx as any).api;
124+
if (!api?.sudo) return;
125+
try {
126+
const prev = await api.sudo().object(ctx.object).findOne({ where: { id } });
127+
if (prev) (ctx as any).__previous = prev;
128+
} catch {
129+
/* ignore — best-effort */
130+
}
131+
};
132+
133+
engine.registerHook('beforeUpdate', captureBefore, { packageId });
134+
engine.registerHook('beforeDelete', captureBefore, { packageId });
135+
136+
/**
137+
* afterInsert / afterUpdate / afterDelete: write audit_log + activity rows.
138+
* Errors are swallowed (logged) so user-facing CRUD is never broken by
139+
* audit failures.
140+
*/
141+
const writeAudit = async (ctx: HookContext) => {
142+
if (SKIP_OBJECTS.has(ctx.object)) return;
143+
const action = actionFor(ctx.event);
144+
if (!action) return;
145+
146+
const api: any = (ctx as any).api;
147+
if (!api?.sudo) return;
148+
149+
const after: any = ctx.result;
150+
const before: any = (ctx as any).__previous ?? (ctx as any).previous ?? null;
151+
152+
// Resolve record id from after (insert/update) or before (delete) or input.
153+
let recordId: string | undefined =
154+
(typeof after === 'object' && after?.id) ||
155+
(typeof before === 'object' && before?.id) ||
156+
((ctx.input as any)?.id);
157+
if (recordId !== undefined) recordId = String(recordId);
158+
159+
const sess: any = (ctx as any).session ?? {};
160+
const userId: string | undefined = sess.userId;
161+
const tenantId: string | undefined = sess.tenantId;
162+
163+
let oldValue: Record<string, any> | null = null;
164+
let newValue: Record<string, any> | null = null;
165+
if (action === 'create') {
166+
newValue = (after && typeof after === 'object') ? { ...after } : null;
167+
} else if (action === 'update') {
168+
const d = diff(before || {}, after || {});
169+
oldValue = d.old;
170+
newValue = d.next;
171+
// If nothing meaningfully changed, skip the audit row to avoid noise.
172+
if (Object.keys(newValue).length === 0) return;
173+
} else if (action === 'delete') {
174+
oldValue = before && typeof before === 'object' ? { ...before } : null;
175+
}
176+
177+
const auditRow: Record<string, any> = {
178+
action,
179+
user_id: userId ?? null,
180+
object_name: ctx.object,
181+
record_id: recordId ?? null,
182+
old_value: oldValue ? safeStringify(oldValue) : null,
183+
new_value: newValue ? safeStringify(newValue) : null,
184+
tenant_id: tenantId ?? null,
185+
};
186+
187+
const label = recordLabel(after ?? before, recordId ?? '');
188+
const summary =
189+
action === 'create' ? `Created ${ctx.object} "${label}"` :
190+
action === 'update' ? `Updated ${ctx.object} "${label}"` :
191+
`Deleted ${ctx.object} "${label}"`;
192+
193+
const activityRow: Record<string, any> = {
194+
type: activityTypeFor(action),
195+
summary,
196+
actor_id: userId ?? null,
197+
object_name: ctx.object,
198+
record_id: recordId ?? null,
199+
record_label: label,
200+
metadata: newValue || oldValue ? safeStringify({ old: oldValue, new: newValue }) : null,
201+
};
202+
203+
try {
204+
const sys = api.sudo();
205+
await sys.object('sys_audit_log').create(auditRow);
206+
await sys.object('sys_activity').create(activityRow);
207+
} catch (err) {
208+
// Log via engine logger if available, but never throw.
209+
try { (engine as any).logger?.warn?.('Audit write failed', { object: ctx.object, action, err: String((err as any)?.message ?? err) }); } catch {}
210+
}
211+
};
212+
213+
engine.registerHook('afterInsert', writeAudit, { packageId });
214+
engine.registerHook('afterUpdate', writeAudit, { packageId });
215+
engine.registerHook('afterDelete', writeAudit, { packageId });
216+
}
217+
218+
// Re-export for convenience.
219+
export type { IDataEngine };

packages/plugins/plugin-audit/src/index.ts

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,4 @@
88
*/
99

1010
export { AuditPlugin } from './audit-plugin.js';
11+
export { installAuditWriters } from './audit-writers.js';

0 commit comments

Comments
 (0)