Skip to content

Commit dcea041

Browse files
hotlonghotlong
authored
Merge pull request #8 from objectql/copilot/organize-raw-data-structure
2 parents 4edd814 + ad37654 commit dcea041

45 files changed

Lines changed: 3939 additions & 0 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎docs/.vitepress/config.mts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -61,6 +61,7 @@ export default defineConfig({
6161
text: 'Core Fundamentals',
6262
items: [
6363
{ text: 'Data Modeling', link: '/guide/data-modeling' },
64+
{ text: 'Metadata Organization', link: '/guide/metadata-organization' },
6465
{ text: 'Querying Data', link: '/guide/querying' },
6566
{ text: 'Business Logic', link: '/guide/logic-hooks' },
6667
]

‎docs/guide/metadata-organization.md‎

Lines changed: 339 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,339 @@
1+
# Organizing Metadata for Large Projects
2+
3+
When building enterprise-scale applications with ObjectQL, proper metadata organization becomes critical. This guide demonstrates best practices for structuring your object definitions, actions, hooks, and translations in a maintainable way.
4+
5+
## The Challenge
6+
7+
As your application grows beyond 30-50 objects, a flat file structure becomes problematic:
8+
9+
**Problems with Flat Structure:**
10+
```
11+
src/objects/
12+
├── user.object.yml
13+
├── organization.object.yml
14+
├── account.object.yml
15+
├── contact.object.yml
16+
├── opportunity.object.yml
17+
├── employee.object.yml
18+
├── department.object.yml
19+
├── invoice.object.yml
20+
├── payment.object.yml
21+
├── project.object.yml
22+
├── task.object.yml
23+
... (100+ files)
24+
```
25+
26+
- ❌ Hard to find related objects
27+
- ❌ Merge conflicts when multiple teams work simultaneously
28+
- ❌ Unclear ownership boundaries
29+
- ❌ Can't deploy modules independently
30+
- ❌ Difficult to understand relationships
31+
32+
## Recommended Structure: Domain-Driven Modules
33+
34+
For applications with **30+ objects** and **multiple teams**, organize by business domain:
35+
36+
```
37+
src/
38+
├── core/ # Foundation objects (user, org, etc.)
39+
│ ├── objects/
40+
│ ├── i18n/
41+
│ └── index.ts
42+
│
43+
├── modules/ # Business domain modules
44+
│ ├── crm/ # Customer management
45+
│ │ ├── objects/
46+
│ │ ├── actions/
47+
│ │ ├── hooks/
48+
│ │ ├── i18n/
49+
│ │ ├── README.md
50+
│ │ └── index.ts
51+
│ │
52+
│ ├── hr/ # Human resources
53+
│ ├── finance/ # Finance & accounting
54+
│ └── project/ # Project management
55+
│
56+
├── extensions/ # Custom overrides
57+
├── shared/ # Shared utilities
58+
└── index.ts # Application entry
59+
```
60+
61+
## Real-World Example
62+
63+
See the complete working example at:
64+
```
65+
examples/scenarios/enterprise-structure/
66+
```
67+
68+
This demonstrates:
69+
- ✅ 20+ objects organized across 5 modules
70+
- ✅ Domain-driven structure (CRM, HR, Finance, Project)
71+
- ✅ Cross-module relationships
72+
- ✅ Extension pattern for customization
73+
- ✅ Comprehensive indexing strategy
74+
- ✅ Multi-language support (en, zh-CN)
75+
- ✅ Module documentation
76+
77+
## Module Anatomy
78+
79+
Each module follows a consistent pattern:
80+
81+
```
82+
modules/[domain]/
83+
├── objects/ # Object definitions
84+
│ ├── [object1].object.yml
85+
│ └── [object2].object.yml
86+
├── actions/ # Custom actions
87+
│ └── [object1].action.ts
88+
├── hooks/ # Lifecycle hooks
89+
│ └── [object1].hooks.ts
90+
├── i18n/ # Translations
91+
│ ├── en/
92+
│ └── zh-CN/
93+
├── README.md # Module documentation
94+
└── index.ts # Module exports
95+
```
96+
97+
### Module Documentation Template
98+
99+
Each module should have a README explaining:
100+
101+
1. **Overview** - What business domain it covers
102+
2. **Objects** - List of objects with descriptions
103+
3. **Relationships** - How objects relate to each other
104+
4. **Team Ownership** - Who maintains this module
105+
5. **Dependencies** - What other modules/objects it depends on
106+
6. **Usage Examples** - Common query patterns
107+
108+
## Object Naming Conventions
109+
110+
### Prefixing Strategy
111+
112+
For large projects with multiple modules, use prefixes to avoid name collisions:
113+
114+
```yaml
115+
# ✅ Good: Clear module ownership
116+
name: crm_account
117+
name: finance_invoice
118+
name: project_task
119+
120+
# ❌ Avoid: Risk of collision
121+
name: account # Which account? CRM or Finance?
122+
name: task # Project task or general task?
123+
```
124+
125+
**When to prefix:**
126+
- ✅ Multi-module applications (30+ objects)
127+
- ✅ Plugin architectures
128+
- ✅ When similar concepts exist across domains
129+
130+
**When NOT to prefix:**
131+
- ❌ Core shared objects (`user`, `organization`)
132+
- ❌ Small applications (< 30 objects)
133+
- ❌ When it reduces clarity
134+
135+
## Dependency Management
136+
137+
### Dependency Layers
138+
139+
```
140+
┌─────────────────────────────────┐
141+
│ Application Layer │
142+
│ (modules/*) │
143+
│ - Can depend on Core │
144+
│ - Can depend on other modules │
145+
└─────────────────────────────────┘
146+
↓
147+
┌─────────────────────────────────┐
148+
│ Foundation Layer │
149+
│ (core/*) │
150+
│ - No dependencies │
151+
│ - Used by everyone │
152+
└─────────────────────────────────┘
153+
```
154+
155+
### Cross-Module References
156+
157+
When modules need to reference each other's objects:
158+
159+
```yaml
160+
# In modules/finance/objects/invoice.object.yml
161+
fields:
162+
account:
163+
type: lookup
164+
reference_to: crm_account # Reference to CRM module
165+
```
166+
167+
**Best Practices:**
168+
1. Document cross-module dependencies in module README
169+
2. Avoid circular dependencies
170+
3. Use core objects to break dependency cycles
171+
4. Consider extracting shared concepts to core layer
172+
173+
## Index Strategy by Module
174+
175+
Different modules have different performance requirements:
176+
177+
### High-Traffic Modules (CRM, Sales)
178+
```yaml
179+
# Aggressive indexing
180+
fields:
181+
status: { type: select, index: true }
182+
owner: { type: lookup, index: true }
183+
created_at: { type: datetime, index: true }
184+
185+
indexes:
186+
owner_status_idx: { fields: [owner, status] }
187+
status_created_idx: { fields: [status, created_at] }
188+
```
189+
190+
### Low-Traffic Modules (Admin, Config)
191+
```yaml
192+
# Minimal indexing
193+
fields:
194+
name: { type: text, index: true }
195+
status: { type: select, index: true }
196+
# Add more indexes only when needed
197+
```
198+
199+
## Extension Pattern
200+
201+
Use extensions to customize objects without modifying source:
202+
203+
**Original** (`core/objects/user.object.yml`):
204+
```yaml
205+
name: user
206+
fields:
207+
name: { type: text }
208+
email: { type: text }
209+
```
210+
211+
**Extension** (`extensions/user.extension.object.yml`):
212+
```yaml
213+
name: user # Same name triggers merge
214+
fields:
215+
employee_id: { type: text }
216+
email: { required: true, unique: true }
217+
```
218+
219+
**Result:** ObjectQL merges both definitions, adding `employee_id` and making `email` required.
220+
221+
## Internationalization at Scale
222+
223+
### Three-Layer i18n Strategy
224+
225+
```
226+
1. Core Layer (core/i18n/)
227+
→ Shared objects (user, organization)
228+
229+
2. Module Layer (modules/[domain]/i18n/)
230+
→ Domain-specific objects
231+
232+
3. Extension Layer (extensions/i18n/)
233+
→ Customer/regional customizations
234+
```
235+
236+
### Directory Structure
237+
```
238+
core/i18n/
239+
en/core.json
240+
zh-CN/core.json
241+
242+
modules/crm/i18n/
243+
en/crm.json
244+
zh-CN/crm.json
245+
```
246+
247+
## Migration Path
248+
249+
### From Flat to Modular
250+
251+
**Step 1: Analyze**
252+
Group existing objects by business domain.
253+
254+
**Step 2: Plan**
255+
Create module structure, decide on prefixes.
256+
257+
**Step 3: Migrate Gradually**
258+
```
259+
src/
260+
├── objects/ # Legacy (keep temporarily)
261+
├── modules/ # New structure
262+
│ └── crm/ # Start with one module
263+
└── index.ts # Loads from both
264+
```
265+
266+
**Step 4: Update References**
267+
Update imports and references to use new module structure.
268+
269+
**Step 5: Clean Up**
270+
Remove legacy flat structure once migration is complete.
271+
272+
## Project Size Guidelines
273+
274+
| Size | Objects | Teams | Recommended Structure |
275+
|------|---------|-------|----------------------|
276+
| **Small** | 1-30 | 1 | Flat `objects/` directory |
277+
| **Medium** | 30-100 | 2-3 | Domain modules |
278+
| **Large** | 100-500 | 5-10 | Modules + plugins |
279+
| **Enterprise** | 500+ | 10+ | Monorepo with packages |
280+
281+
## Testing Strategy
282+
283+
### Module Tests
284+
```typescript
285+
// modules/crm/__tests__/integration.test.ts
286+
describe('CRM Module', () => {
287+
it('should handle lead conversion', async () => {
288+
const lead = await createLead();
289+
await convertLead(lead.id);
290+
// Verify account, contact, opportunity created
291+
});
292+
});
293+
```
294+
295+
### Object Schema Tests
296+
```typescript
297+
// modules/crm/objects/__tests__/account.test.ts
298+
describe('Account Object', () => {
299+
it('should have required fields', () => {
300+
const schema = loadObjectSchema('crm_account');
301+
expect(schema.fields.name.required).toBe(true);
302+
});
303+
});
304+
```
305+
306+
## Complete Working Example
307+
308+
Explore the full example with 20+ objects:
309+
```bash
310+
cd examples/scenarios/enterprise-structure
311+
pnpm install
312+
pnpm build
313+
```
314+
315+
The example includes:
316+
- Core module (user, organization, attachment)
317+
- CRM module (account, contact, opportunity, lead)
318+
- HR module (employee, department, position, timesheet)
319+
- Finance module (invoice, payment, expense, budget)
320+
- Project module (project, task, milestone, timesheet entry)
321+
- Extensions pattern
322+
- Multi-language support
323+
- Comprehensive documentation
324+
325+
## Key Takeaways
326+
327+
1. **Start simple** - Don't over-engineer for small projects
328+
2. **Think in domains** - Organize by business capability, not technical layers
329+
3. **Document boundaries** - Make module ownership and dependencies explicit
330+
4. **Plan for scale** - Use prefixes and modules when you hit 30-50 objects
331+
5. **Test modules** - Each module should be testable independently
332+
6. **Version control** - Use module-level versioning for independent deployments
333+
334+
## See Also
335+
336+
- [Data Modeling Guide](./data-modeling.md)
337+
- [Plugin Development](./plugins.md)
338+
- [Logic Hooks](./logic-hooks.md)
339+
- [Complete Example](../../examples/scenarios/enterprise-structure/)

‎examples/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ Welcome to the ObjectQL examples collection. This directory is organized to help
2222

2323
| Example | Description | Proficiency |
2424
| :--- | :--- | :--- |
25+
| **[Enterprise Structure](./scenarios/enterprise-structure)** | **[NEW]** Best practices for organizing metadata in large-scale applications. Shows domain-driven module structure with 20+ objects across CRM, HR, Finance, and Project modules. | 🏢 Advanced |
2526
| **[Preset Usage](./scenarios/preset-usage)** | Shows how to consume pre-packaged business logic (presets) in an application. | 💡 Intermediate |
2627

2728
## 🚧 Coming Soon

‎examples/scenarios/enterprise-structure/CHANGELOG.md‎

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
# @example/enterprise-structure
2+
3+
## 1.0.0
4+
5+
### Added
6+
7+
- Initial release with enterprise-scale metadata organization example
8+
- Core module with user, organization, and attachment objects
9+
- CRM module with account, contact, opportunity, and lead objects
10+
- HR module with employee, department, position, and timesheet objects
11+
- Finance module with invoice, payment, expense, and budget objects
12+
- Project module with project, task, milestone, and timesheet entry objects
13+
- Extension pattern demonstration with user extensions
14+
- Comprehensive documentation for each module
15+
- Shared utilities, constants, and validators
16+
- Module index files for clean exports
17+
- README with best practices and architecture guidance

0 commit comments

Comments
 (0)