|
1 | 1 | # ObjectQL Kernel Optimizations |
2 | 2 |
|
3 | | -This document describes the 10 kernel optimizations implemented for ObjectQL to improve performance, scalability, and resource efficiency. |
| 3 | +This document describes the kernel optimizations that have been **directly integrated** into ObjectQL to improve performance, scalability, and resource efficiency. |
4 | 4 |
|
5 | 5 | ## Overview |
6 | 6 |
|
7 | | -The ObjectQL kernel has been enhanced with the following optimizations: |
| 7 | +The ObjectQL kernel includes the following built-in optimizations: |
8 | 8 |
|
9 | | -1. **Metadata Registry Optimization** - O(k) package uninstall with secondary indexes |
10 | | -2. **Query AST Compilation with LRU Cache** - Cached query plan compilation |
11 | | -3. **Hook Pipeline Compilation** - Pre-compiled hook pipelines |
12 | | -4. **Connection Pool Management** - Kernel-level global connection pooling |
13 | | -5. **Validation Engine Optimization** - Compiled validation rules |
14 | | -6. **Lazy Metadata Loading** - On-demand metadata loading with predictive preload |
15 | | -7. **TypeScript Type Generation** - (Deferred for compatibility) |
16 | | -8. **Smart Dependency Graph** - DAG-based dependency resolution |
17 | | -9. **Query Optimizer (SQL-specific)** - SQL-aware optimization with index hints |
18 | | -10. **Memory-Mapped Storage** - (Deferred for compatibility) |
| 9 | +1. **Metadata Registry Optimization** ✅ - O(k) package uninstall with secondary indexes (INTEGRATED) |
| 10 | +2. **Query AST Compilation with LRU Cache** ✅ - Cached query plan compilation (INTEGRATED) |
| 11 | +3. **Hook Pipeline Compilation** ✅ - Pre-compiled hook pipelines (INTEGRATED) |
| 12 | +4. **Connection Pool Management** - Kernel-level global connection pooling (AVAILABLE) |
| 13 | +5. **Validation Engine Optimization** - Compiled validation rules (AVAILABLE) |
| 14 | +6. **Lazy Metadata Loading** - On-demand metadata loading with predictive preload (AVAILABLE) |
| 15 | +7. **Smart Dependency Graph** - DAG-based dependency resolution (AVAILABLE) |
| 16 | +8. **SQL Query Optimizer** - SQL-aware optimization with index hints (AVAILABLE) |
19 | 17 |
|
20 | | -## 1. Metadata Registry Optimization |
| 18 | +## Built-in Optimizations (Always Active) |
21 | 19 |
|
22 | | -### Problem |
23 | | -The original `unregisterPackage` operation had O(n*m) complexity, iterating over all metadata types and all items within each type. |
| 20 | +These optimizations are automatically enabled and require no configuration: |
24 | 21 |
|
25 | | -### Solution |
26 | | -Implemented a secondary index that maps package names to their metadata references. This reduces complexity to O(k), where k is the number of items in the package being unregistered. |
| 22 | +### 1. Metadata Registry Optimization ✅ INTEGRATED |
27 | 23 |
|
28 | | -### Usage |
| 24 | +**Location:** `packages/foundation/types/src/registry.ts` |
29 | 25 |
|
30 | | -```typescript |
31 | | -import { OptimizedMetadataRegistry } from '@objectql/core'; |
| 26 | +The MetadataRegistry now uses a secondary index for O(k) package uninstallation. |
32 | 27 |
|
33 | | -const registry = new OptimizedMetadataRegistry(); |
| 28 | +**What Changed:** |
| 29 | +- Added `packageIndex` Map for tracking package-to-metadata references |
| 30 | +- `unregisterPackage()` now does direct lookup instead of iterating all items |
| 31 | +- Complexity reduced from O(n*m) to O(k) |
34 | 32 |
|
35 | | -// Register items with package information |
36 | | -registry.register('object', { |
37 | | - name: 'user', |
38 | | - package: 'crm', |
39 | | - fields: {...} |
40 | | -}); |
| 33 | +**Performance Impact:** |
| 34 | +- **10x faster** package operations |
| 35 | +- No performance degradation as metadata grows |
| 36 | + |
| 37 | +**Usage:** |
| 38 | +No changes needed - this is now the default behavior: |
| 39 | + |
| 40 | +```typescript |
| 41 | +import { MetadataRegistry } from '@objectql/types'; |
41 | 42 |
|
42 | | -// Fast O(k) package uninstallation |
| 43 | +const registry = new MetadataRegistry(); |
| 44 | +registry.register('object', { name: 'user', package: 'crm' }); |
| 45 | + |
| 46 | +// Fast O(k) operation - no configuration needed |
43 | 47 | registry.unregisterPackage('crm'); |
44 | 48 | ``` |
45 | 49 |
|
46 | | -### Performance Impact |
47 | | -- **10x faster** package operations |
48 | | -- Eliminates performance degradation as metadata grows |
| 50 | +### 2. Query AST Compilation with LRU Cache ✅ INTEGRATED |
49 | 51 |
|
50 | | -## 2. Query AST Compilation with LRU Cache |
| 52 | +**Location:** `packages/foundation/core/src/repository.ts` |
51 | 53 |
|
52 | | -### Problem |
53 | | -Query AST was reinterpreted on every execution, causing redundant computation for repeated queries. |
| 54 | +Query AST compilation now includes automatic caching via QueryCompiler. |
54 | 55 |
|
55 | | -### Solution |
56 | | -Implemented a query compiler with LRU cache that compiles AST to optimized execution plans and caches results. |
| 56 | +**What Changed:** |
| 57 | +- Added static `QueryCompiler` instance shared across all repositories |
| 58 | +- `buildQueryAST()` now caches compiled queries automatically |
| 59 | +- LRU cache (1000 entries) prevents memory growth |
57 | 60 |
|
58 | | -### Usage |
59 | | - |
60 | | -```typescript |
61 | | -import { QueryCompiler } from '@objectql/core'; |
| 61 | +**Performance Impact:** |
| 62 | +- **10x faster** query planning for repeated queries |
| 63 | +- **50% lower** CPU usage |
| 64 | +- Automatic cache eviction using LRU policy |
62 | 65 |
|
63 | | -const compiler = new QueryCompiler(1000); // Cache size |
| 66 | +**Usage:** |
| 67 | +No changes needed - caching happens automatically: |
64 | 68 |
|
65 | | -// Compile and cache query |
66 | | -const compiled = compiler.compile('user', { |
67 | | - filters: { status: 'active' }, |
68 | | - sort: [{ field: 'created', order: 'desc' }] |
69 | | -}); |
| 69 | +```typescript |
| 70 | +// First call compiles and caches |
| 71 | +const repo = context.object('user'); |
| 72 | +await repo.find({ filters: { status: 'active' } }); |
70 | 73 |
|
71 | | -// Subsequent calls return cached plan |
72 | | -const cached = compiler.compile('user', { /* same query */ }); |
| 74 | +// Second call uses cached plan |
| 75 | +await repo.find({ filters: { status: 'active' } }); |
73 | 76 | ``` |
74 | 77 |
|
75 | | -### Performance Impact |
76 | | -- **10x faster** query planning |
77 | | -- **50% lower** CPU usage for repeated queries |
78 | | -- Automatic cache eviction using LRU policy |
| 78 | +### 3. Hook Pipeline Compilation ✅ INTEGRATED |
79 | 79 |
|
80 | | -## 3. Hook Pipeline Compilation |
| 80 | +**Location:** `packages/foundation/core/src/app.ts` |
81 | 81 |
|
82 | | -### Problem |
83 | | -Hook patterns were matched on every operation, causing O(n) pattern matching overhead for every hook execution. |
84 | | - |
85 | | -### Solution |
86 | | -Pre-compiles hook pipelines by event pattern at registration time. Uses direct lookup at runtime with no pattern matching. |
| 82 | +Hook management now uses CompiledHookManager for pre-compiled pipelines. |
87 | 83 |
|
88 | | -### Usage |
| 84 | +**What Changed:** |
| 85 | +- Replaced local hook management with `CompiledHookManager` |
| 86 | +- Wildcard patterns (`before*`, `*`) expanded at registration time |
| 87 | +- Runtime uses O(1) direct lookup instead of pattern matching |
89 | 88 |
|
90 | | -```typescript |
91 | | -import { CompiledHookManager } from '@objectql/core'; |
| 89 | +**Performance Impact:** |
| 90 | +- **5x faster** hook execution |
| 91 | +- Parallel async hook support |
| 92 | +- No runtime pattern matching overhead |
92 | 93 |
|
93 | | -const hookManager = new CompiledHookManager(); |
| 94 | +**Usage:** |
| 95 | +No changes needed - pattern compilation happens automatically: |
94 | 96 |
|
95 | | -// Register hooks - patterns are expanded at registration |
96 | | -hookManager.registerHook('before*', 'user', async (ctx) => { |
| 97 | +```typescript |
| 98 | +// Patterns are expanded at registration |
| 99 | +app.on('before*', 'user', async (ctx) => { |
97 | 100 | // Handler code |
98 | 101 | }); |
99 | 102 |
|
100 | | -// Fast O(1) lookup and execution |
101 | | -await hookManager.runHooks('beforeCreate', 'user', context); |
| 103 | +// Fast O(1) lookup at runtime |
| 104 | +await app.triggerHook('beforeCreate', 'user', context); |
102 | 105 | ``` |
103 | 106 |
|
104 | | -### Features |
105 | | -- Wildcard pattern expansion (`before*`, `*`) |
106 | | -- Parallel async execution |
107 | | -- Priority-based ordering |
| 107 | +## Available Optimizations (Opt-in) |
108 | 108 |
|
109 | | -### Performance Impact |
110 | | -- **5x faster** hook execution |
111 | | -- Parallel async hook support |
112 | | -- No runtime pattern matching |
| 109 | +## Available Optimizations (Opt-in) |
| 110 | + |
| 111 | +These optimizations are available as separate modules for advanced use cases: |
113 | 112 |
|
114 | | -## 4. Global Connection Pool Management |
| 113 | +### 4. Connection Pool Management |
115 | 114 |
|
116 | 115 | ### Problem |
117 | 116 | Each driver managed connections independently with no global resource limits, leading to resource exhaustion. |
|
0 commit comments