Skip to content

Latest commit

 

History

History
615 lines (469 loc) · 24.5 KB

File metadata and controls

615 lines (469 loc) · 24.5 KB

@objectstack/spec Development Plan

Date: 2026-02-07
Based on: ZOD_SCHEMA_AUDIT_REPORT.md
Current Version: 1.0.11
Target Version: 1.1.0 (Phase 1-2), 1.2.0 (Phase 3-4)


Overview

Based on the full audit of 139 .zod.ts files (43,746 LOC, 1,089 schemas), the spec package achieves a B+ quality grade. This plan targets the systematic resolution of all identified issues across 4 phases, prioritized by impact and risk.

Key Metrics Baseline

Metric Original Current Phase 4 Target Status
z.any() usages 397 8 8 (filter operators only)
z.unknown() usages 8 351 > 350
z.infer coverage 93% (1,011/1,089) ~100% (1,055) 100%
.describe() annotations 5,026 5,671 5,600
z.input<> exports 0 122 Comprehensive
Schema duplications 13+ pairs 0 0
Runtime logic violations 2 files 2 files (deprecated) 0 (moved to @objectstack/core)
Naming violations 3 0 0

Phase 1: Critical Bug Fixes (1-2 days)

Goal: Fix all P0 bugs that cause runtime errors, silent data loss, or complete type-safety bypass.

1.1 Fix z.any() in union consuming all branches

File: src/data/hook.zod.ts L65
Problem: handler: z.union([z.string(), z.any()])z.any() swallows the entire union.
Fix:

// Before
handler: z.union([z.string(), z.any()])
// After
handler: z.union([z.string(), z.function()])
  .describe('Handler function name (string) or inline function reference')

1.2 Fix ValidationRuleSchema inferring as any

File: src/data/validation.zod.ts L302
Problem: z.ZodType<any> causes ValidationRule to infer as any.
Fix: Define an explicit discriminated union of all validation types:

export const ValidationRuleSchema = z.discriminatedUnion('type', [
  ScriptValidationSchema,
  UniquenessValidationSchema,
  StateMachineValidationSchema,
  FormatValidationSchema,
  CrossFieldValidationSchema,
  JSONValidationSchema,
  AsyncValidationSchema,
  CustomValidatorSchema,
  ConditionalValidationSchema,
]);
export type ValidationRule = z.infer<typeof ValidationRuleSchema>;

1.3 Fix auth-config.zod.ts invalid computed key syntax

File: src/system/auth-config.zod.ts
Problem: [z.string().regex(...).toString()]: z.any() is invalid in z.object().
Fix: Replace with proper passthrough or .catchall():

// Use z.record() for dynamic keys, or .passthrough() for loose validation

1.4 Fix MongoDriverSpec capability key mismatches

File: src/data/driver/mongo.zod.ts L56-70
Problem: Capabilities use wrong property names, silently stripped by parse().
Fix: Align capability keys with DatasourceCapabilities schema:

// Before: aggregation: true
// After:  queryAggregations: true
// Before: mutableSchema: true
// After:  dynamicSchema: true

1.5 Fix DatasourceConfig type alias pointing to wrong schema

File: src/data/datasource.zod.ts L131
Fix:

// Before
export type DatasourceConfig = z.infer<typeof DatasourceCapabilities>;
// After
export type DatasourceCapabilities = z.infer<typeof DatasourceCapabilitiesSchema>;

1.6 Replace z.instanceof(Error) with JSON-serializable shape

Files: src/kernel/plugin-lifecycle-events.zod.ts, src/kernel/startup-orchestrator.zod.ts
Problem: z.instanceof(Error) cannot generate JSON Schema.
Fix:

// Before
error: z.instanceof(Error)
// After
error: z.object({
  name: z.string().describe('Error class name'),
  message: z.string().describe('Error message'),
  stack: z.string().optional().describe('Stack trace'),
  code: z.string().optional().describe('Error code'),
}).describe('Serializable error representation')

1.7 Fix filter operator typo $exist$exists

File: src/data/filter.zod.ts L107
Fix: Rename $exist to $exists to match MongoDB standard.

Phase 1 Checklist

# Task File(s) Status
1.1 Fix z.any() in handler union data/hook.zod.ts
1.2 Fix ValidationRuleSchema type-safety data/validation.zod.ts
1.3 Fix invalid computed key syntax system/auth-config.zod.ts
1.4 Fix Mongo capabilities key names data/driver/mongo.zod.ts
1.5 Fix DatasourceConfig alias data/datasource.zod.ts
1.6 Replace z.instanceof(Error) kernel/plugin-lifecycle-events.zod.ts, kernel/startup-orchestrator.zod.ts
1.7 Fix $exist$exists typo data/filter.zod.ts
1.8 Run full test suite, verify build

Phase 2: Deduplication & Naming Consistency (3-5 days)

Goal: Eliminate schema duplications, unify naming patterns, extract shared types.

2.1 Create shared/enums.zod.ts — Shared Enumeration Registry

Extract all hardcoded string literals repeated across 3+ files:

// src/shared/enums.zod.ts

/** Aggregation functions used across query, data-engine, analytics, field */
export const AggregationFunctionEnum = z.enum([
  'count', 'sum', 'avg', 'min', 'max',
  'count_distinct', 'percentile', 'median', 'stddev', 'variance',
]).describe('Standard aggregation functions');

/** Sort direction used across query, data-engine, analytics */
export const SortDirectionEnum = z.enum(['asc', 'desc'])
  .describe('Sort order direction');

/** CRUD mutation events used across hook, validation, object CDC */
export const MutationEventEnum = z.enum([
  'insert', 'update', 'delete', 'upsert',
]).describe('Data mutation event types');

/** Database isolation levels — unified format */
export const IsolationLevelEnum = z.enum([
  'read_uncommitted', 'read_committed', 'repeatable_read', 'serializable', 'snapshot',
]).describe('Transaction isolation levels (snake_case standard)');

/** Cache eviction strategies */
export const CacheStrategyEnum = z.enum(['lru', 'lfu', 'ttl', 'fifo'])
  .describe('Cache eviction strategy');

Files to update: data/query.zod.ts, data/data-engine.zod.ts, data/analytics.zod.ts, data/field.zod.ts, data/hook.zod.ts, data/validation.zod.ts, data/driver.zod.ts, data/external-lookup.zod.ts

2.2 Create shared/metadata-types.zod.ts — Resolve 13-schema duplication

Problem: kernel/metadata-loader.zod.ts and system/metadata-persistence.zod.ts define 13 overlapping schemas with different structures.

Solution:

  1. Create shared/metadata-types.zod.ts with base schemas
  2. kernel/metadata-loader.zod.ts imports and extends for kernel use-case
  3. system/metadata-persistence.zod.ts imports and extends for persistence use-case
// src/shared/metadata-types.zod.ts
export const MetadataFormatSchema = z.enum(['yaml', 'json', 'typescript', 'javascript'])
  .describe('Metadata file format');

export const BaseMetadataRecordSchema = z.object({
  id: z.string(),
  type: z.string(),
  name: SnakeCaseIdentifierSchema,
  format: MetadataFormatSchema,
}).describe('Base metadata record fields shared across kernel and system');

2.3 Resolve Security Schema duplications

Canonical Location Schema Remove From
hub/plugin-security.zod.ts SecurityVulnerabilitySchema kernel/plugin-security-advanced.zod.ts (import instead)
hub/plugin-security.zod.ts SecurityScanResultSchema kernel/plugin-security-advanced.zod.ts
hub/plugin-security.zod.ts SecurityPolicySchema kernel/plugin-security-advanced.zod.ts
kernel/plugin-versioning.zod.ts DependencyConflictSchema hub/plugin-security.zod.ts (import instead)
kernel/plugin-lifecycle-advanced.zod.ts HotReloadConfigSchema kernel/plugin-loading.zod.ts (import as PluginHotReloadSchema)
kernel/plugin-security-advanced.zod.ts SandboxConfigSchema kernel/plugin-loading.zod.ts (import as PluginSandboxingSchema)

2.4 Resolve MetricType export name collision

Problem: system/metrics.zod.ts and hub/license.zod.ts both export MetricType with completely different enum values.

Fix:

// hub/license.zod.ts — rename to avoid collision
export const LicenseMetricType = z.enum(['boolean', 'counter', 'gauge']);
export type LicenseMetricType = z.infer<typeof LicenseMetricType>;

// system/metrics.zod.ts — keep as canonical
export const MetricType = z.enum(['counter', 'gauge', 'histogram', 'summary']);

2.5 Unify SnakeCaseIdentifierSchema usage across UI

Replace inline regex in these files with SnakeCaseIdentifierSchema import from shared/identifiers.zod.ts:

File Line Current Action
ui/dashboard.zod.ts L75 z.string().regex(/^[a-z_][a-z0-9_]*$/) Import SnakeCaseIdentifierSchema
ui/report.zod.ts L54 Same inline regex Import SnakeCaseIdentifierSchema
ui/widget.zod.ts L255 Same inline regex Import SnakeCaseIdentifierSchema
ui/theme.zod.ts L195 Same inline regex Import SnakeCaseIdentifierSchema

2.6 Fix metadata-persistence.zod.ts property key naming

// Before (snake_case — violates camelCase property key rule)
id: z.string(),
created_by: z.string(),
created_at: z.string().datetime(),
updated_by: z.string(),
updated_at: z.string().datetime(),

// After (camelCase)
id: z.string(),
createdBy: z.string(),
createdAt: z.string().datetime(),
updatedBy: z.string(),
updatedAt: z.string().datetime(),

2.7 Standardize z.date()z.string().datetime()

Replace z.date() with z.string().datetime() in serializable schemas:

File Fields
identity/identity.zod.ts createdAt, updatedAt, emailVerified
identity/organization.zod.ts createdAt, updatedAt
api/auth.zod.ts createdAt, updatedAt, expiresAt
kernel/metadata-loader.zod.ts modifiedAt, timestamp, lastModified
system/object-storage.zod.ts lastModified
system/metadata-persistence.zod.ts created_at (also fix to createdAt)

2.8 Unify isolation level enum in data/driver.zod.ts

Problem: L101 uses kebab-case ('read-committed'), L570 uses SQL uppercase ('READ COMMITTED').
Fix: Both reference the new IsolationLevelEnum from shared/enums.zod.ts.

2.9 Resolve service-registry.zod.ts filename collision

Problem: kernel/service-registry.zod.ts and system/service-registry.zod.ts share the same filename.
Fix: Rename system/service-registry.zod.tssystem/core-services.zod.ts.

Phase 2 Checklist

# Task File(s) Status
2.1 Create shared/enums.zod.ts + update consumers 8+ files
2.2 Create shared/metadata-types.zod.ts 3 files
2.3 Deduplicate security schemas 4 files ✅ (Kernel uses Kernel-prefixed variants)
2.4 Rename MetricType in license.zod.ts hub/license.zod.ts
2.5 Unify SnakeCaseIdentifierSchema usage 4 UI files
2.6 Fix snake_case property keys system/metadata-persistence.zod.ts
2.7 Replace z.date() with z.string().datetime() 6 files
2.8 Unify isolation level enum data/driver.zod.ts
2.9 Rename system/service-registry.zod.ts 1 file + index
2.10 Deduplicate Presence schemas (realtime/websocket) 2 files
2.11 Run full test suite, update index.ts re-exports

Phase 3: Type Safety Hardening (1-2 weeks)

Goal: Reduce z.any() from 397 to < 100 through systematic replacement.

3.1 Bulk z.any()z.unknown() migration (~140 occurrences)

Pattern: Replace z.record(z.string(), z.any()) with z.record(z.string(), z.unknown()) for all metadata, config, options, params, properties fields.

Script approach:

# Identify all candidates (manual review required before applying)
grep -rn "z\.record(z\.string(), z\.any())" --include="*.zod.ts" src/ \
  | grep -E "(metadata|config|options|params|properties|settings|customizations)"

Expected reduction: ~140 z.any()z.unknown()

3.2 Tighten id fields

Replace id: z.any() with z.union([z.string(), z.number()]):

File Fields
data/data-engine.zod.ts L200 DataEngineUpdateRequestSchema.id
data/data-engine.zod.ts L207 DataEngineDeleteRequestSchema.id

3.3 Fix critical z.any() hotspots (per-file)

kernel/plugin.zod.ts (20+ z.any() — worst file)

Strategy: Define minimal interface schemas for plugin context services:

// Define minimal service interfaces instead of z.any()
const MinimalLoggerSchema = z.object({
  debug: z.function(),
  info: z.function(),
  warn: z.function(),
  error: z.function(),
}).passthrough();

const MinimalEventBusSchema = z.object({
  emit: z.function(),
  on: z.function(),
  off: z.function(),
}).passthrough();

// Apply to PluginContextSchema
PluginContextSchema = z.object({
  logger: MinimalLoggerSchema,
  events: MinimalEventBusSchema,
  // ... other services with minimal interfaces
});

data/driver.zod.ts (10+ z.any())

Strategy: Define return type shapes for CRUD operations:

const RecordShape = z.record(z.string(), z.unknown());
const RecordArrayShape = z.array(RecordShape);

// Replace z.promise(z.any()) with z.promise(RecordShape)

data/data-engine.zod.ts (8+ z.any())

Strategy: Same as driver.zod.ts — use RecordShape for return types.

kernel/events.zod.ts (8 z.any())

Strategy: Replace payload: z.any() with payload: z.unknown(), handlers with function schemas.

kernel/manifest.zod.ts (7 z.any())

Strategy: Define ConfigPropertySchema for configuration properties:

const ConfigPropertySchema = z.object({
  type: z.enum(['string', 'number', 'boolean', 'object', 'array']),
  default: z.unknown(),
  description: z.string().optional(),
});

3.4 Replace z.array(z.any()) in UI schemas with proper references

File Field Replace With
ui/view.zod.ts L185 filter: z.array(z.any()) z.array(FilterConditionSchema) (import from data/filter.zod)
ui/app.zod.ts L169 objects: z.array(z.any()) z.array(ObjectSchema) or z.array(z.string())
ui/app.zod.ts L170 apis: z.array(z.any()) z.array(z.string()) (API name references)
ui/component.zod.ts L22 children: z.array(z.any()) z.lazy(() => z.array(PageComponentSchema))

3.5 Remove deprecated location: z.any() from action.zod.ts

File: src/ui/action.zod.ts L63
Already replaced by locations array. Remove the deprecated field.

Phase 3 Checklist

# Task Scope z.any() Reduction Status
3.1 Bulk metadata/config z.any() → z.unknown() ~88 files -140
3.2 Tighten id fields 2 files -2
3.3a Harden kernel/plugin.zod.ts 1 file -23
3.3b Harden data/driver.zod.ts 1 file -10
3.3c Harden data/data-engine.zod.ts 1 file -8
3.3d Harden kernel/events.zod.ts 1 file -8
3.3e Harden kernel/manifest.zod.ts 1 file -7
3.4 Fix UI z.any() with proper imports 4 files -6
3.5 Remove deprecated action.location 1 file -1
3.6 Replace z.any() in api/protocol.zod.ts 1 file -28
3.7 Replace z.any() in hook, core-services, widget 3 files -3
3.8 Run full test suite

Actual total reduction: 397 → 8 (z.any()) — remaining 8 are legitimate filter operators ($eq/$ne/$in/$nin)


Phase 4: Completeness & Polish (ongoing)

Goal: Achieve 100% type export coverage, maximise .describe() documentation, align with industry best practices.

4.1 Add missing z.infer type exports

File Missing Types Priority
system/notification.zod.ts All types (~6) High
system/auth-config.zod.ts All types (~3) High
system/change-management.zod.ts All types (~6) High
kernel/plugin-structure.zod.ts All types (~3) High
api/contract.zod.ts RecordData, BaseResponse, etc. High
api/analytics.zod.ts Partial types Medium
ai/rag-pipeline.zod.ts All 16 schemas missing types High
ai/nlq.zod.ts 9 of 13 types missing Medium
ui/chart.zod.ts ChartAnnotation, ChartInteraction Low
ui/page.zod.ts PageVariable Low
ui/widget.zod.ts WidgetSource Low
data/analytics.zod.ts CubeJoin, enum types Low
system/translation.zod.ts TranslationData Low
system/service-registry.zod.ts ServiceStatus, ServiceConfig Low

4.2 Add z.input<> type exports for transform/default schemas

Following ui/report.zod.ts as the pattern, add z.input<> exports to all schemas that use .default() or .transform():

// Pattern: Export both output and input types
export type Report = z.infer<typeof ReportSchema>;       // output (after parse)
export type ReportInput = z.input<typeof ReportSchema>;   // input (before parse)

Target files: All schemas with .default() or .transform() — especially in ui/, data/object.zod.ts, kernel/manifest.zod.ts.

4.3 Improve .describe() coverage in under-documented files

Files with < 50% annotation coverage:

File Current Action
system/migration.zod.ts Minimal Add field-level descriptions
system/translation.zod.ts Minimal Add field-level descriptions
system/cache.zod.ts Minimal Add field-level descriptions
system/encryption.zod.ts Minimal Add field-level descriptions
system/message-queue.zod.ts Minimal Add field-level descriptions
system/compliance.zod.ts None Add complete descriptions
system/masking.zod.ts Minimal Add field-level descriptions
system/search-engine.zod.ts Minimal Add field-level descriptions
kernel/plugin-structure.zod.ts Minimal Add field-level descriptions

4.4 Remove runtime logic from spec repository

File Runtime Logic Move To
api/errors.zod.ts createErrorResponse(), getHttpStatusForCategory() @objectstack/core or @objectstack/runtime
kernel/manifest.zod.ts definePlugin() @objectstack/core
kernel/plugin.zod.ts definePlugin() @objectstack/core

4.5 Unify Factory Helper pattern

Decision: Adopt const X = { create } as const pattern (used by action, dashboard, report). Remove Object.assign(Schema, { create }) from app.zod.ts.

All factory create() methods should consistently call Schema.parse(config) for validated output.

4.6 Add missing industry-standard fields

Data Protocol

Schema Missing Fields Reference
FieldSchema sortable, inlineHelpText, trackFeedHistory, caseSensitive, autonumberFormat Salesforce, ServiceNow
ObjectSchema recordTypes, sharingModel, keyPrefix Salesforce
DatasourceSchema healthCheck, retryPolicy Production best practice
HookSchema description, retryPolicy, timeout Resilience patterns

UI Protocol

Schema Missing Fields Reference
ListViewSchema emptyState, rowActions, bulkActions, virtualScroll, conditionalFormatting, inlineEdit, exportOptions Modern list UX
FormViewSchema validationRules, submitAction Form best practice
DashboardSchema refreshInterval, globalFilters, layout Dashboard UX
ReportSchema parameters, scheduling, exportFormats Enterprise reporting
ActionSchema disabled, shortcut, bulkEnabled, timeout Action UX
AppSchema locale, mobileEnabled, offlineCapable, theme App features
PageSchema responsive, permissions, lifecycle Page lifecycle

4.7 Deprecation management

Add explicit deprecation markers and migration paths for:

Field File Migration
formula data/field.zod.ts L367 expression
encryption: z.boolean() data/field.zod.ts L449 encryptionConfig
geoSpatial data/driver.zod.ts L175 geospatialQuery
TenantSchema hub/tenant.zod.ts → New isolation config
stateMachine (singular) data/object.zod.ts L225 stateMachines (plural)

Pattern:

/** @deprecated Use `expression` instead. Will be removed in v2.0.0 */
formula: z.string().optional()
  .describe('DEPRECATED: Use `expression` field instead. Scheduled for removal in v2.0.0'),

Phase 4 Checklist

# Task Scope Status
4.1 Add missing z.infer exports 14 files
4.2 Add z.input<> exports for transform schemas 17 files (62 exports)
4.3 Improve .describe() coverage 9 files ✅ (5,341 total annotations)
4.4 Mark runtime logic as deprecated 3 functions ✅ (deprecated with @objectstack/core migration path)
4.5 Unify factory helper pattern 1 file ✅ (App.create now uses Schema.parse)
4.6 Add industry-standard fields 6 files ✅ (field, object, datasource, hook, view, dashboard, action)
4.7 Add deprecation markers + migration paths 5 fields ✅ (formula, encryption, geoSpatial, stateMachine, TenantSchema)
4.8 Update JSON Schema generation scripts 1 file ✅ (verified: 1,207 schemas generated)
4.9 Fix index.ts barrel exports 1 file ✅ (removed duplicate auth/storage exports in api/index.ts)
4.10 Full regression test + build verification ✅ (97 test files, 3,074 tests pass)

Release Plan

v1.1.0 — Type Safety Release

Includes: Phase 1 + Phase 2
Timeline: 1 week
Breaking changes:

  • MetricType renamed to LicenseMetricType in hub/license.zod.ts
  • MetricType renamed to AggregationMetricType in data/analytics.zod.ts
  • system/service-registry.zod.ts renamed to system/core-services.zod.ts
  • metadata-persistence.zod.ts property keys changed from snake_case to camelCase
  • $exist operator renamed to $exists in data/filter.zod.ts
  • z.date()z.string().datetime() in 6 files

Migration guide needed: Yes (for MetricType rename, property key changes)

v1.2.0 — Type Hardening Release

Includes: Phase 3 + Phase 4
Timeline: 2-3 weeks
Breaking changes:

  • z.any()z.unknown() in ~140 metadata/config records (consumers need type narrowing)
  • ValidationRuleSchema now returns proper union type instead of any
  • Deprecated fields marked for v2.0.0 removal
  • Runtime helpers moved to @objectstack/core

Migration guide needed: Yes (for z.unknown() migration, consumers must add type narrowing)


Validation Strategy

Automated Checks

# 1. Count z.any() (target: < 100 after Phase 3)
grep -rc "z\.any()" --include="*.zod.ts" src/ | awk -F: '{s+=$2} END {print s}'

# 2. Count z.unknown() (target: > 200 after Phase 3)
grep -rc "z\.unknown()" --include="*.zod.ts" src/ | awk -F: '{s+=$2} END {print s}'

# 3. Verify no z.date() in serializable schemas
grep -rn "z\.date()" --include="*.zod.ts" src/ | grep -v "filter.zod.ts"

# 4. Verify no snake_case property keys (exclude name/object/table machine identifiers)
# Manual review required

# 5. Build & type check
pnpm build
pnpm typecheck

# 6. JSON Schema generation
pnpm generate:json-schema

# 7. Run tests
pnpm test

Per-Phase CI Gate

Phase Gate Criteria
Phase 1 Build passes, all existing tests pass, 0 new TypeScript errors
Phase 2 Build passes, z.any() < 300, no filename collisions in index.ts
Phase 3 Build passes, z.any() < 100, z.unknown() > 200
Phase 4 Build passes, z.infer coverage = 100%, .describe() > 5,300, JSON Schema generation succeeds

Appendix: Best Practice Reference Files

When implementing changes, use these files as patterns:

Pattern Reference File What to Learn
100% .describe() coverage ui/theme.zod.ts Every field annotated
z.unknown() usage qa/testing.zod.ts Zero z.any(), all z.unknown()
Input+Output types ui/report.zod.ts z.infer + z.input exports
Shared identifier usage ui/app.zod.ts Imports SnakeCaseIdentifierSchema
Factory pattern ui/dashboard.zod.ts const X = { create } as const
Discriminated union data/validation.zod.ts z.discriminatedUnion('type', [...])
Recursive types automation/state-machine.zod.ts z.lazy() with manual type definition
Zero z.any() data schema data/object.zod.ts All fields strictly typed
Integration module quality integration/connector.zod.ts Best documentation, cleanest types