Skip to content

Commit 772dcd4

Browse files
hotlonghotlong
authored
Merge pull request #1061 from objectstack-ai/claude/add-metadata-history-support
2 parents 2477741 + 1358200 commit 772dcd4

12 files changed

Lines changed: 1976 additions & 2 deletions

‎CHANGELOG.md‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,6 +8,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
88
## [Unreleased]
99

1010
### Added
11+
- **Metadata Versioning & History** — Comprehensive version history tracking and rollback capabilities
12+
for metadata items. Key features include:
13+
- `MetadataHistoryRecordSchema` defining structure for historical snapshots
14+
- `sys_metadata_history` system table for version storage
15+
- Automatic history tracking in `DatabaseLoader` with SHA-256 checksum deduplication
16+
- `getHistory()`, `rollback()`, and `diff()` methods in `IMetadataService`
17+
- REST API endpoints: `GET /history`, `POST /rollback`, `GET /diff`
18+
- `HistoryCleanupManager` with configurable retention policies (age-based and count-based)
19+
- Comprehensive test suite covering all history operations
20+
21+
This aligns ObjectStack with enterprise platforms like Salesforce Setup Audit Trail and
22+
ServiceNow Update Sets. See `docs/METADATA_HISTORY.md` for detailed usage.
1123

1224
- **CLI: Remote API Commands** - Added 12 new CLI commands for interacting with remote ObjectStack servers:
1325
- **Authentication**: `os auth login`, `os auth logout`, `os auth whoami`

‎docs/METADATA_HISTORY.md‎

Lines changed: 226 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,226 @@
1+
# Metadata Versioning & History
2+
3+
## Overview
4+
5+
The ObjectStack metadata system now supports comprehensive version history tracking and rollback capabilities. This feature allows administrators to:
6+
7+
- View complete change history for any metadata item
8+
- Compare versions to see what changed
9+
- Rollback to previous versions when needed
10+
- Track who made changes and when
11+
- Automatically clean up old history records
12+
13+
## Architecture
14+
15+
### Core Components
16+
17+
1. **MetadataHistoryRecordSchema** (`packages/spec/src/system/metadata-persistence.zod.ts`)
18+
- Defines the structure for history records
19+
- Includes version, checksum, operation type, change notes, and audit fields
20+
21+
2. **sys_metadata_history Object** (`packages/metadata/src/objects/sys-metadata-history.object.ts`)
22+
- System table for storing historical snapshots
23+
- Automatically created when history tracking is enabled
24+
25+
3. **DatabaseLoader History Tracking** (`packages/metadata/src/loaders/database-loader.ts`)
26+
- Automatically writes history records on create/update
27+
- Calculates SHA-256 checksums for change detection
28+
- Skips duplicate history records when content is unchanged
29+
30+
4. **MetadataManager History Methods** (`packages/metadata/src/metadata-manager.ts`)
31+
- `getHistory()` - Query version timeline
32+
- `rollback()` - Restore previous version
33+
- `diff()` - Compare two versions
34+
35+
5. **REST API Endpoints** (`packages/metadata/src/routes/history-routes.ts`)
36+
- `GET /api/v1/metadata/:type/:name/history` - View history
37+
- `POST /api/v1/metadata/:type/:name/rollback` - Rollback to version
38+
- `GET /api/v1/metadata/:type/:name/diff` - Compare versions
39+
40+
6. **Cleanup Manager** (`packages/metadata/src/utils/history-cleanup.ts`)
41+
- Age-based retention (maxAgeDays)
42+
- Count-based retention (maxVersions)
43+
- Automatic scheduled cleanup
44+
45+
## Usage
46+
47+
### Enable History Tracking
48+
49+
History tracking is enabled by default when using a DatabaseLoader:
50+
51+
```typescript
52+
import { DatabaseLoader } from '@objectstack/metadata';
53+
54+
const dbLoader = new DatabaseLoader({
55+
driver: myDriver,
56+
trackHistory: true, // Default: true
57+
});
58+
```
59+
60+
### Query Version History
61+
62+
```typescript
63+
const history = await metadataService.getHistory('object', 'account', {
64+
limit: 50,
65+
offset: 0,
66+
operationType: 'update',
67+
since: '2025-01-01T00:00:00Z',
68+
});
69+
70+
console.log(`Total versions: ${history.total}`);
71+
history.records.forEach(record => {
72+
console.log(`Version ${record.version} - ${record.operationType} by ${record.recordedBy} at ${record.recordedAt}`);
73+
});
74+
```
75+
76+
### Rollback to Previous Version
77+
78+
```typescript
79+
const restored = await metadataService.rollback('object', 'account', 5, {
80+
changeNote: 'Reverting problematic changes',
81+
recordedBy: 'admin@example.com',
82+
});
83+
```
84+
85+
### Compare Versions
86+
87+
```typescript
88+
const diff = await metadataService.diff('object', 'account', 5, 6);
89+
90+
console.log(`Identical: ${diff.identical}`);
91+
console.log(`Summary: ${diff.summary}`);
92+
console.log(`Changes: ${diff.patch.length} operations`);
93+
94+
diff.patch.forEach(op => {
95+
console.log(`${op.op} ${op.path}: ${JSON.stringify(op.value)}`);
96+
});
97+
```
98+
99+
### Configure Retention Policy
100+
101+
```typescript
102+
import { HistoryCleanupManager } from '@objectstack/metadata';
103+
104+
const cleanupManager = new HistoryCleanupManager(
105+
{
106+
maxVersions: 100, // Keep last 100 versions per item
107+
maxAgeDays: 180, // Keep history for 6 months
108+
autoCleanup: true, // Enable automatic cleanup
109+
cleanupIntervalHours: 24, // Run daily
110+
},
111+
dbLoader
112+
);
113+
114+
// Start automatic cleanup
115+
cleanupManager.start();
116+
117+
// Manual cleanup
118+
const result = await cleanupManager.runCleanup();
119+
console.log(`Deleted ${result.deleted} records, ${result.errors} errors`);
120+
121+
// Preview cleanup
122+
const stats = await cleanupManager.getCleanupStats();
123+
console.log(`Would delete ${stats.total} records`);
124+
125+
// Stop cleanup
126+
cleanupManager.stop();
127+
```
128+
129+
### REST API Examples
130+
131+
```bash
132+
# Get history
133+
curl "http://localhost:3000/api/v1/metadata/object/account/history?limit=10"
134+
135+
# Rollback to version 5
136+
curl -X POST "http://localhost:3000/api/v1/metadata/object/account/rollback" \
137+
-H "Content-Type: application/json" \
138+
-d '{"version": 5, "changeNote": "Reverting changes"}'
139+
140+
# Compare versions
141+
curl "http://localhost:3000/api/v1/metadata/object/account/diff?version1=5&version2=6"
142+
```
143+
144+
### Register History Routes
145+
146+
In your Hono app:
147+
148+
```typescript
149+
import { registerMetadataHistoryRoutes } from '@objectstack/metadata';
150+
151+
registerMetadataHistoryRoutes(app, metadataService);
152+
```
153+
154+
## Implementation Details
155+
156+
### Checksum Calculation
157+
158+
- Uses SHA-256 hashing of normalized JSON
159+
- Keys are sorted recursively for deterministic output
160+
- Fallback to simple hash for environments without Web Crypto API
161+
162+
### History Record Structure
163+
164+
```typescript
165+
{
166+
id: string; // UUID
167+
metadataId: string; // FK to sys_metadata.id
168+
name: string; // Denormalized for queries
169+
type: string; // Denormalized for queries
170+
version: number; // Version number
171+
operationType: 'create' | 'update' | 'publish' | 'revert' | 'delete';
172+
metadata: Record<string, unknown>; // Full snapshot
173+
checksum: string; // SHA-256 hash
174+
previousChecksum?: string; // For diff optimization
175+
changeNote?: string; // Human-readable description
176+
tenantId?: string; // Multi-tenant isolation
177+
recordedBy?: string; // User identifier
178+
recordedAt: string; // ISO datetime
179+
}
180+
```
181+
182+
### Database Schema
183+
184+
The `sys_metadata_history` table includes indexes for:
185+
- `(metadata_id, version)` - Unique constraint
186+
- `(metadata_id, recorded_at)` - Timeline queries
187+
- `(type, name)` - Cross-type queries
188+
- `(recorded_at)` - Age-based cleanup
189+
- `(operation_type)` - Operation filtering
190+
- `(tenant_id)` - Multi-tenant isolation
191+
192+
### Diff Algorithm
193+
194+
Uses a simple recursive diff algorithm that generates operations:
195+
- `add` - New field added
196+
- `remove` - Field removed
197+
- `replace` - Field value changed
198+
199+
The diff result includes:
200+
- Array of change operations
201+
- Human-readable summary (e.g., "2 fields added, 1 field modified")
202+
203+
## Performance Considerations
204+
205+
- History records are written synchronously as part of each save operation, ensuring consistency between metadata state and the history timeline
206+
- Checksum deduplication prevents storing identical versions
207+
- Indexes optimize common query patterns
208+
- Automatic cleanup prevents unbounded growth
209+
- History queries default to 50 records with pagination
210+
211+
## Comparison with Other Platforms
212+
213+
| Platform | Version Control | History Duration |
214+
|----------|----------------|------------------|
215+
| Salesforce | Setup Audit Trail | 6 months |
216+
| ServiceNow | Update Sets + Versions | Unlimited |
217+
| ObjectStack | sys_metadata_history | Configurable |
218+
219+
## Future Enhancements
220+
221+
- [ ] Export/import history bundles
222+
- [ ] Visual diff UI in Studio
223+
- [ ] Change request workflow integration
224+
- [ ] History annotations and tagging
225+
- [ ] Merge conflict resolution
226+
- [ ] Branch/fork metadata workflow

‎packages/metadata/src/index.ts‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -21,6 +21,14 @@ export { DatabaseLoader, type DatabaseLoaderOptions } from './loaders/database-l
2121

2222
// Objects
2323
export { SysMetadataObject } from './objects/sys-metadata.object.js';
24+
export { SysMetadataHistoryObject } from './objects/sys-metadata-history.object.js';
25+
26+
// Routes
27+
export { registerMetadataHistoryRoutes } from './routes/history-routes.js';
28+
29+
// Utils
30+
export { calculateChecksum, generateSimpleDiff, generateDiffSummary } from './utils/metadata-history-utils.js';
31+
export { HistoryCleanupManager } from './utils/history-cleanup.js';
2432

2533
// Serializers
2634
export { type MetadataSerializer, type SerializeOptions } from './serializers/serializer-interface.js';
@@ -43,6 +51,11 @@ export type {
4351
MetadataCollectionInfo,
4452
MetadataLoaderContract,
4553
MetadataManagerConfig,
54+
MetadataHistoryRecord,
55+
MetadataHistoryQueryOptions,
56+
MetadataHistoryQueryResult,
57+
MetadataDiffResult,
58+
MetadataHistoryRetentionPolicy,
4659
} from '@objectstack/spec/system';
4760

4861
// Re-export IMetadataService contract

0 commit comments

Comments
 (0)