Skip to content

Commit 908d95c

Browse files
Copilothotlong
andcommitted
docs: Add ADR-0001 for metadata service architecture and flow documentation
Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
1 parent 9c44bb5 commit 908d95c

4 files changed

Lines changed: 557 additions & 0 deletions

File tree

ARCHITECTURE.md

Lines changed: 15 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,6 +13,7 @@
1313
5. [Plugin System](#plugin-system)
1414
6. [Dependency Graph](#dependency-graph)
1515
7. [Design Decisions](#design-decisions)
16+
8. [Additional Resources](#additional-resources)
1617

1718
---
1819

@@ -639,6 +640,20 @@ export type Field = z.infer<typeof FieldSchema>;
639640

640641
---
641642

643+
## Additional Resources
644+
645+
### Architecture Decision Records (ADRs)
646+
647+
Important architectural decisions are documented as ADRs in `docs/adr/`:
648+
649+
- [ADR-0001: Metadata Service Architecture](docs/adr/0001-metadata-service-architecture.md) - Explains why both ObjectQL and MetadataPlugin can provide metadata service and how they work together
650+
651+
### Component-Specific Documentation
652+
653+
- [Metadata Flow Documentation](docs/METADATA_FLOW.md) - Detailed explanation of how metadata flows from definition to runtime, including configuration examples and troubleshooting
654+
655+
---
656+
642657
## Future Considerations
643658

644659
### Planned Enhancements

docs/METADATA_FLOW.md

Lines changed: 295 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,295 @@
1+
# Metadata Service: Architecture and Flow
2+
3+
This document explains how metadata flows through ObjectStack from definition to runtime.
4+
5+
## Overview
6+
7+
ObjectStack supports **two modes** for metadata provision:
8+
9+
1. **Simple Mode (ObjectQL-only)**: Metadata defined in code, stored in memory
10+
2. **Advanced Mode (with MetadataPlugin)**: Metadata loaded from files, with watch/export/persistence features
11+
12+
Both modes are compatible and use the same service interface.
13+
14+
## Architecture Decision
15+
16+
See [ADR-0001: Metadata Service Architecture](./adr/0001-metadata-service-architecture.md) for the full decision rationale.
17+
18+
## Service Providers
19+
20+
### ObjectQL as Metadata Provider
21+
22+
**Package**: `@objectstack/objectql`
23+
**Service Registered**: `metadata`, `objectql`, `data`
24+
**When Used**: Default fallback when MetadataPlugin is not loaded
25+
26+
**Characteristics**:
27+
- ✅ Simple setup - no additional configuration
28+
- ✅ Fast in-memory access
29+
- ✅ Good for testing and simple applications
30+
- ⚠️ No file persistence
31+
- ⚠️ No file watching
32+
- ⚠️ Limited to programmatically registered metadata
33+
34+
**Flow**:
35+
```
36+
objectstack.config.ts
37+
38+
defineStack({ plugins: [new AppPlugin(manifest)] })
39+
40+
ObjectQLPlugin.start() discovers apps via kernel.getServices()
41+
42+
ObjectQL.registerApp(manifest) → Internal Registry
43+
44+
metadata service queries → Registry lookup → Response
45+
```
46+
47+
### MetadataPlugin as Provider
48+
49+
**Package**: `@objectstack/metadata`
50+
**Service Registered**: `metadata`
51+
**When Used**: Explicitly loaded in plugin configuration
52+
53+
**Characteristics**:
54+
- ✅ File system persistence
55+
- ✅ File watching for hot reload
56+
- ✅ Multi-format support (YAML, JSON, TypeScript)
57+
- ✅ Multi-source (filesystem, HTTP, database)
58+
- ✅ Export/import capabilities
59+
- ⚠️ Requires explicit setup
60+
- ⚠️ Slightly more complex
61+
62+
**Flow**:
63+
```
64+
File System (objects/, views/, apps/, etc.)
65+
66+
MetadataPlugin.start() loads all metadata types
67+
68+
NodeMetadataManager (uses FilesystemLoader)
69+
70+
Serializers (YAML/JSON/TS) parse files
71+
72+
metadata service queries → MetadataManager.load() → File lookup → Response
73+
```
74+
75+
## Integration Flow
76+
77+
When **both** ObjectQL and MetadataPlugin are loaded:
78+
79+
```
80+
1. MetadataPlugin.init()
81+
→ Registers 'metadata' service FIRST
82+
83+
2. ObjectQLPlugin.init()
84+
→ Checks for existing 'metadata' service
85+
→ Finds MetadataPlugin
86+
→ Does NOT register 'metadata' (already exists)
87+
→ Registers 'objectql' and 'data' only
88+
89+
3. MetadataPlugin.start()
90+
→ Loads metadata from file system
91+
→ Service now provides file-based metadata
92+
93+
4. ObjectQLPlugin.start()
94+
→ Detects external metadata service
95+
→ Loads definitions from it
96+
→ Populates internal registry for fast queries
97+
→ Discovers apps and drivers from kernel
98+
99+
5. Runtime Queries
100+
→ API calls ctx.getService('metadata')
101+
→ Gets MetadataPlugin instance
102+
→ Returns file-based metadata
103+
104+
→ ObjectQL queries use registry (pre-loaded from MetadataPlugin)
105+
→ Fast in-memory access for data operations
106+
```
107+
108+
## Example Configurations
109+
110+
### Simple Mode (ObjectQL Only)
111+
112+
```typescript
113+
// objectstack.config.ts
114+
import { defineStack } from '@objectstack/spec';
115+
import { ObjectQLPlugin } from '@objectstack/objectql';
116+
import { AppPlugin } from '@objectstack/runtime';
117+
import myApp from './myapp.config';
118+
119+
export default defineStack({
120+
manifest: {
121+
id: 'my-app',
122+
name: 'my_app',
123+
version: '1.0.0',
124+
type: 'app'
125+
},
126+
plugins: [
127+
new ObjectQLPlugin(),
128+
new AppPlugin(myApp), // Metadata defined in code
129+
]
130+
});
131+
```
132+
133+
**Result**: ObjectQL provides metadata service, serves in-memory definitions.
134+
135+
### Advanced Mode (With MetadataPlugin)
136+
137+
```typescript
138+
// objectstack.config.ts
139+
import { defineStack } from '@objectstack/spec';
140+
import { ObjectQLPlugin } from '@objectstack/objectql';
141+
import { MetadataPlugin } from '@objectstack/metadata';
142+
143+
export default defineStack({
144+
manifest: {
145+
id: 'my-app',
146+
name: 'my_app',
147+
version: '1.0.0',
148+
type: 'app'
149+
},
150+
plugins: [
151+
new MetadataPlugin({
152+
rootDir: process.cwd(),
153+
watch: true
154+
}),
155+
new ObjectQLPlugin(),
156+
]
157+
});
158+
```
159+
160+
**Result**: MetadataPlugin provides metadata service, ObjectQL reads from it.
161+
162+
**File Structure**:
163+
```
164+
project/
165+
├── objectstack.config.ts
166+
├── objects/
167+
│ ├── account.object.ts
168+
│ └── contact.object.ts
169+
├── views/
170+
│ ├── account-list.view.yaml
171+
│ └── contact-form.view.json
172+
└── apps/
173+
└── crm.app.ts
174+
```
175+
176+
## API Metadata Endpoints
177+
178+
All API metadata endpoints use the kernel's `metadata` service:
179+
180+
```typescript
181+
// In API handler
182+
const metadataService = ctx.getService('metadata');
183+
184+
// Single object
185+
const object = await metadataService.load('object', 'account');
186+
return { data: object };
187+
188+
// List all objects
189+
const objects = await metadataService.loadMany('object');
190+
return { data: objects };
191+
```
192+
193+
The API **doesn't know or care** whether metadata comes from ObjectQL or MetadataPlugin.
194+
195+
## When to Use Each Mode
196+
197+
### Use ObjectQL-only when:
198+
- Building prototypes or POCs
199+
- Writing tests
200+
- Creating simple single-file applications
201+
- All metadata is programmatically generated
202+
203+
### Use MetadataPlugin when:
204+
- Building production applications
205+
- Need file-based metadata (version control friendly)
206+
- Want hot reload during development
207+
- Need export/import capabilities
208+
- Multiple developers editing metadata
209+
- Metadata stored in external systems
210+
211+
## Metadata Service Interface
212+
213+
Both providers implement this interface:
214+
215+
```typescript
216+
interface IMetadataService {
217+
/**
218+
* Load a single metadata item
219+
*/
220+
load<T>(type: string, name: string, options?: MetadataLoadOptions): Promise<T | null>;
221+
222+
/**
223+
* Load multiple metadata items of a type
224+
*/
225+
loadMany<T>(type: string, options?: MetadataLoadOptions): Promise<T[]>;
226+
227+
/**
228+
* Save a metadata item
229+
*/
230+
save<T>(type: string, name: string, data: T, options?: MetadataSaveOptions): Promise<MetadataSaveResult>;
231+
232+
/**
233+
* Check if metadata item exists
234+
*/
235+
exists(type: string, name: string): Promise<boolean>;
236+
237+
/**
238+
* List all items of a type
239+
*/
240+
list(type: string): Promise<string[]>;
241+
}
242+
```
243+
244+
## Troubleshooting
245+
246+
### "Service 'metadata' already registered" Error
247+
248+
**Cause**: Both ObjectQL and MetadataPlugin trying to register the service.
249+
250+
**Solution**: Ensure MetadataPlugin is loaded BEFORE ObjectQLPlugin in the plugins array. ObjectQL will detect it and not register.
251+
252+
```typescript
253+
// ❌ Wrong order
254+
plugins: [
255+
new ObjectQLPlugin(),
256+
new MetadataPlugin(), // Too late, ObjectQL already registered metadata
257+
]
258+
259+
// ✅ Correct order
260+
plugins: [
261+
new MetadataPlugin(), // Registers first
262+
new ObjectQLPlugin(), // Detects metadata service, doesn't register
263+
]
264+
```
265+
266+
### Metadata Changes Not Reflected
267+
268+
**Cause**: Using ObjectQL-only mode, which doesn't watch files.
269+
270+
**Solution**: Add MetadataPlugin with `watch: true`:
271+
272+
```typescript
273+
plugins: [
274+
new MetadataPlugin({ watch: true }),
275+
new ObjectQLPlugin(),
276+
]
277+
```
278+
279+
### "Cannot find metadata" in API
280+
281+
**Cause**: Metadata not loaded into the service.
282+
283+
**Debug**:
284+
1. Check logs for "Loaded X objects" messages
285+
2. Verify file paths are correct
286+
3. Check that files have correct naming (e.g., `*.object.ts`, `*.view.yaml`)
287+
4. Ensure MetadataPlugin `rootDir` points to correct location
288+
289+
## References
290+
291+
- [ADR-0001: Metadata Service Architecture](./adr/0001-metadata-service-architecture.md)
292+
- [ObjectQL Package](../packages/objectql/README.md)
293+
- [Metadata Package](../packages/metadata/README.md)
294+
- [Metadata Spec](../packages/spec/src/api/metadata.zod.ts)
295+
- [Metadata Loader Protocol](../packages/spec/src/kernel/metadata-loader.zod.ts)

0 commit comments

Comments
 (0)