Skip to content

Add enterprise-scale metadata organization guide and reference implementation#8

Merged
hotlong merged 6 commits into
mainfrom
copilot/organize-raw-data-structure
Jan 11, 2026
Merged

Add enterprise-scale metadata organization guide and reference implementation#8
hotlong merged 6 commits into
mainfrom
copilot/organize-raw-data-structure

Conversation

Copy link
Copy Markdown
Contributor

Copilot AI commented Jan 11, 2026

User asked for guidance on structuring metadata in large ObjectQL projects. No such documentation existed.

Changes

Reference Implementation (examples/scenarios/enterprise-structure/)

Complete working example with 20 objects across 4 domain modules:

  • Core layer: Foundation objects (user, organization, attachment)
  • Module layer: Domain-driven modules (CRM, HR, Finance, Project)
  • Extensions layer: Schema merge pattern for customization
  • Shared utilities: Constants, validators, type-safe helpers

Demonstrates:

  • Domain prefixing strategy (crm_account, hr_employee, etc.)
  • Cross-module relationships
  • Composite indexing strategies
  • Three-layer i18n architecture
  • Module-level documentation with ownership

Documentation (docs/guide/metadata-organization.md)

Covers:

  • Migration path from flat to modular structure
  • Project size thresholds (30/100/500+ objects)
  • Dependency layer architecture
  • Extension pattern vs direct modification
  • Index optimization by traffic patterns

Example Structure

# modules/crm/objects/account.object.yml
name: crm_account
fields:
  name: { type: text, required: true, index: true }
  industry: { type: select, index: true }
  owner: { type: lookup, reference_to: user, index: true }

indexes:
  owner_industry_idx: { fields: [owner, industry] }

# extensions/user.extension.object.yml (merges with core/objects/user.object.yml)
name: user
fields:
  employee_id: { type: text, unique: true }
  email: { required: true }  # Override core definition

Bilingual documentation (en, zh-CN) with 35KB of guides and module READMEs.

Original prompt

考虑一下大型项目该如何安排原数据的代码结构,给我一个例子


💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.

Copilot AI and others added 3 commits January 11, 2026 08:22
Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com>
Copilot AI changed the title [WIP] Organize raw data code structure for large project Add enterprise-scale metadata organization guide and reference implementation Jan 11, 2026
Copilot AI requested a review from hotlong January 11, 2026 08:29
Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This pull request adds a comprehensive enterprise-scale metadata organization example and guide to the ObjectQL repository. It demonstrates best practices for structuring large projects with 20+ objects across 4 domain modules (CRM, HR, Finance, Project).

Changes:

  • Complete working reference implementation with modular structure across core, modules, and extensions layers
  • Comprehensive documentation guide explaining organization patterns, migration paths, and thresholds
  • Bilingual documentation (English and Chinese) with module-level READMEs

Reviewed changes

Copilot reviewed 46 out of 46 changed files in this pull request and generated 8 comments.

Show a summary per file
File Description
examples/scenarios/enterprise-structure/src/index.ts Main entry point demonstrating module loading pattern
examples/scenarios/enterprise-structure/src/shared/*.ts Shared utilities for validation, formatting, and constants
examples/scenarios/enterprise-structure/src/core/objects/*.yml Foundation objects (user, organization, attachment)
examples/scenarios/enterprise-structure/src/modules//objects/.yml Domain objects across CRM, HR, Finance, and Project modules
examples/scenarios/enterprise-structure/src/modules/*/index.ts Module metadata exports
examples/scenarios/enterprise-structure/src/modules/*/README.md Module documentation with ownership and usage examples
examples/scenarios/enterprise-structure/src/extensions/*.yml Extension pattern demonstration
examples/scenarios/enterprise-structure/src/core/i18n//.json Core object translations (en, zh-CN)
examples/scenarios/enterprise-structure/src/modules/crm/i18n//.json CRM module translations (incomplete)
examples/scenarios/enterprise-structure/README.md & README.zh-CN.md Comprehensive example documentation in both languages
examples/scenarios/enterprise-structure/package.json Package configuration with dependencies
examples/scenarios/enterprise-structure/tsconfig.json TypeScript configuration
docs/guide/metadata-organization.md New documentation guide for large-scale organization
docs/.vitepress/config.mts Added navigation link to new guide
examples/README.md Added enterprise structure example to examples index
Comments suppressed due to low confidence (1)

examples/scenarios/enterprise-structure/src/modules/hr/index.ts:27

  • The i18n directory structure is incomplete. According to the documentation and the three-layer i18n strategy described, each module should have its own i18n directory with translations. However, only the CRM and Core modules have i18n directories, while HR, Finance, and Project modules are missing them entirely. This is inconsistent with the documented architecture.

Comment on lines +31 to +92

// Register objects from each module
// In a real application, you might use dynamic imports or a plugin system
await registerCoreObjects(app);
await registerCRMModule(app);
await registerHRModule(app);
await registerFinanceModule(app);
await registerProjectModule(app);
await registerExtensions(app);

return app;
}

/**
* Register core objects (foundation layer)
*/
async function registerCoreObjects(app: ObjectQL) {
// Load objects from core/objects/
// user, organization, attachment
}

/**
* Register CRM module
*/
async function registerCRMModule(app: ObjectQL) {
// Load objects from modules/crm/objects/
// crm_account, crm_contact, crm_opportunity, crm_lead
}

/**
* Register HR module
*/
async function registerHRModule(app: ObjectQL) {
// Load objects from modules/hr/objects/
// hr_employee, hr_department, hr_position, hr_timesheet
}

/**
* Register Finance module
*/
async function registerFinanceModule(app: ObjectQL) {
// Load objects from modules/finance/objects/
// finance_invoice, finance_payment, finance_expense, finance_budget
}

/**
* Register Project module
*/
async function registerProjectModule(app: ObjectQL) {
// Load objects from modules/project/objects/
// project_project, project_task, project_milestone, project_timesheet_entry
}

/**
* Register extensions (override layer)
*/
async function registerExtensions(app: ObjectQL) {
// Load extensions from extensions/
// These will merge with existing objects
}

/**
Copy link

Copilot AI Jan 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The register functions (registerCoreObjects, registerCRMModule, etc.) are defined but have no implementation - they only contain comments. Since this is an example, these functions should either have actual implementations that load the YAML files, or be removed if the loading happens automatically via connect(). The comment on line 29 suggests automatic discovery, which would make these empty functions misleading.

Suggested change
// Register objects from each module
// In a real application, you might use dynamic imports or a plugin system
await registerCoreObjects(app);
await registerCRMModule(app);
await registerHRModule(app);
await registerFinanceModule(app);
await registerProjectModule(app);
await registerExtensions(app);
return app;
}
/**
* Register core objects (foundation layer)
*/
async function registerCoreObjects(app: ObjectQL) {
// Load objects from core/objects/
// user, organization, attachment
}
/**
* Register CRM module
*/
async function registerCRMModule(app: ObjectQL) {
// Load objects from modules/crm/objects/
// crm_account, crm_contact, crm_opportunity, crm_lead
}
/**
* Register HR module
*/
async function registerHRModule(app: ObjectQL) {
// Load objects from modules/hr/objects/
// hr_employee, hr_department, hr_position, hr_timesheet
}
/**
* Register Finance module
*/
async function registerFinanceModule(app: ObjectQL) {
// Load objects from modules/finance/objects/
// finance_invoice, finance_payment, finance_expense, finance_budget
}
/**
* Register Project module
*/
async function registerProjectModule(app: ObjectQL) {
// Load objects from modules/project/objects/
// project_project, project_task, project_milestone, project_timesheet_entry
}
/**
* Register extensions (override layer)
*/
async function registerExtensions(app: ObjectQL) {
// Load extensions from extensions/
// These will merge with existing objects
}
/**
return app;
}
// Domain modules (core, CRM, HR, Finance, Project, extensions) are
// automatically discovered from their .object.yml files during app.connect().
// No manual registration functions are required in this example.
/**
* Example usage
*/

Copilot uses AI. Check for mistakes.
Comment on lines +42 to +47
/**
* Deep clone an object
*/
export function deepClone<T>(obj: T): T {
return JSON.parse(JSON.stringify(obj));
}
Copy link

Copilot AI Jan 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The deepClone function uses JSON.parse(JSON.stringify(obj)) which has known limitations: it doesn't handle functions, undefined values, Symbols, circular references, Date objects (they become strings), or other non-JSON-serializable values. For an enterprise example, either add a comment documenting these limitations or use a more robust cloning approach.

Copilot uses AI. Check for mistakes.
Comment on lines +272 to +280
## Project Size Guidelines

| Size | Objects | Teams | Recommended Structure |
|------|---------|-------|----------------------|
| **Small** | 1-30 | 1 | Flat `objects/` directory |
| **Medium** | 30-100 | 2-3 | Domain modules |
| **Large** | 100-500 | 5-10 | Modules + plugins |
| **Enterprise** | 500+ | 10+ | Monorepo with packages |

Copy link

Copilot AI Jan 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The guidance on when to use modular structure is inconsistent. The table shows "Small" projects handle 10-30 objects with flat structure and "Medium" starts at 30-100 with modules. However, the text says "Use modules when you hit 30-50 objects". This creates ambiguity about whether 30 objects should use flat or modular structure. Consider clarifying that 30 objects is the transition point.

Copilot uses AI. Check for mistakes.
Comment on lines +23 to +28
"peerDependencies": {
"@objectql/core": "workspace:*",
"@objectql/types": "workspace:*",
"@objectql/driver-knex": "workspace:*",
"sqlite3": "^5.1.7"
},
Copy link

Copilot AI Jan 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The @objectql/driver-knex package is listed in peerDependencies but not in devDependencies. This means it won't be available during development/build. Either add it to devDependencies or remove it from peerDependencies if it's only used at runtime by consumers.

Copilot uses AI. Check for mistakes.
Comment on lines +22 to +26
* Validate currency amount (positive number with max 2 decimals)
*/
export function isValidCurrency(amount: number): boolean {
return amount >= 0 && Number.isFinite(amount);
}
Copy link

Copilot AI Jan 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation comment says "positive number with max 2 decimals" but the implementation only checks if the amount is non-negative and finite. It doesn't validate the decimal precision (max 2 decimals). The function should either validate decimal places or the comment should be corrected to remove the "max 2 decimals" claim.

Copilot uses AI. Check for mistakes.
label: Employee Record
unique: true

# Override email to make it required and unique
Copy link

Copilot AI Jan 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The email field in the core user object is already defined as required and unique (lines 15, 17). The extension file comment on line 13 says "Override email to make it required and unique" but this is redundant since the base definition already has these properties. The extension should demonstrate an actual override (e.g., adding validation) or the comment should be updated to clarify that this is just showing the merge pattern.

Suggested change
# Override email to make it required and unique
# Demonstrate overriding email; base already marks it required and unique,
# this shows the field merge/override pattern

Copilot uses AI. Check for mistakes.
"actions": {
"mark_won": "Mark as Won",
"mark_lost": "Mark as Lost"
}
Copy link

Copilot AI Jan 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CRM i18n file is incomplete. It includes translations for crm_account, crm_contact, and crm_opportunity, but is missing translations for crm_lead which is defined in the CRM module. The same translations should be added for consistency.

Suggested change
}
}
},
"crm_lead": {
"label": "Lead",
"fields": {
"name": "Lead Name",
"status": "Status",
"source": "Lead Source",
"email": "Email",
"phone": "Phone",
"owner": "Lead Owner"
}

Copilot uses AI. Check for mistakes.
"actions": {
"mark_won": "标记为已赢",
"mark_lost": "标记为已失"
}
Copy link

Copilot AI Jan 11, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The CRM i18n file is incomplete. It includes translations for crm_account, crm_contact, and crm_opportunity, but is missing translations for crm_lead which is defined in the CRM module. The same translations should be added for consistency.

Suggested change
}
}
},
"crm_lead": {
"label": "线索",
"fields": {
"name": "线索名称",
"status": "状态",
"source": "线索来源",
"email": "电子邮件",
"phone": "电话",
"owner": "线索负责人",
"account": "客户",
"description": "描述"
}

Copilot uses AI. Check for mistakes.
Deleted the Chinese README file and removed the language switcher link from the English README. This simplifies documentation to a single language version.
@hotlong hotlong marked this pull request as ready for review January 11, 2026 14:02
@hotlong hotlong merged commit dcea041 into main Jan 11, 2026
0 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants