Merge pull request #552 from objectstack-ai/copilot/assess-plugin-system-architecture

Author: hotlong

packages/spec/PLUGIN_STANDARDS.md

@@ -116,3 +116,66 @@ To help AI understand the "intent" of a file, use a standard JSDoc header.
  */
 export const LeadObject = ...
 ```
+
+---
+
+## 5. Plugin Runtime Capabilities
+
+The microkernel architecture provides the following runtime capabilities for plugins. The Zod schemas governing each capability live in `src/kernel/`.
+
+### 5.1 Hot Reload (`plugin-loading.zod.ts` → `PluginHotReloadSchema`)
+
+Hot reload supports **development, staging, and production** environments. The `environment` field controls the safety level:
+
+| Environment | Behavior |
+| :--- | :--- |
+| `development` | Fast reload with file watchers, no health validation required |
+| `staging` | Production-like reload with validation but relaxed rollback |
+| `production` | Full safety: health validation, auto-rollback, connection draining |
+
+Production safety features (`productionSafety`):
+- **Health validation** — run health checks after reload before accepting traffic
+- **Rollback on failure** — auto-rollback if reloaded plugin fails health check
+- **Connection draining** — gracefully drain active requests before reloading
+- **Concurrency control** — limit concurrent reloads (`maxConcurrentReloads`)
+- **Reload cooldown** — minimum interval between reloads of the same plugin (≥1s)
+
+### 5.2 Plugin Isolation (`plugin-loading.zod.ts` → `PluginSandboxingSchema`)
+
+Sandboxing supports configurable **scope** and **isolation level**:
+
+| Scope | Description |
+| :--- | :--- |
+| `automation-only` | Sandbox automation/scripting plugins only (default) |
+| `untrusted-only` | Sandbox plugins below a trust threshold |
+| `all-plugins` | Sandbox all plugins for maximum isolation |
+
+Isolation levels: `none`, `process`, `vm`, `iframe`, `web-worker`.
+
+**Inter-Plugin Communication (IPC):** Isolated plugins communicate with the kernel and other plugins via configurable IPC:
+- Transports: `message-port`, `unix-socket`, `tcp`, `memory`
+- Configurable message size limit, timeout, and service ACL (`allowedServices`)
+
+### 5.3 Dynamic Loading (`plugin-runtime.zod.ts`)
+
+Plugins can be loaded and unloaded at runtime **without restarting the kernel**:
+
+- **`DynamicLoadRequestSchema`** — Load a plugin from `npm`, `local`, `url`, `registry`, or `git` sources with optional integrity verification
+- **`DynamicUnloadRequestSchema`** — Graceful/forceful/drain unload with dependency awareness (`cascade`, `warn`, or `block` dependents)
+- **`ActivationEventSchema`** — Lazy activation triggers: `onCommand`, `onRoute`, `onObject`, `onEvent`, `onService`, `onSchedule`, `onStartup`
+- **`PluginDiscoveryConfigSchema`** — Runtime discovery from registries and local directories with polling and trust filtering
+- **`DynamicLoadingConfigSchema`** — Subsystem configuration: max dynamic plugins, default sandbox policy, allowed sources, integrity requirements
+
+### 5.4 Plugin System Assessment Summary
+
+| Capability | Status | Schema / Details |
+| :--- | :--- | :--- |
+| Plugin Registration | ✅ | `manifest.zod.ts` — `objectstack.config.ts` plugin array, ordered initialization |
+| Lifecycle Hooks | ✅ | `plugin.zod.ts` — `init()` → `start()` → `healthCheck()` → `destroy()` |
+| Service Registry | ✅ | `service-registry.zod.ts` — 17 services across 13 plugins via `ctx.registerService()` |
+| Event Bus | ✅ | `events.zod.ts` — Pub/sub with pattern matching |
+| Dependency Resolution | ✅ | `plugin-loading.zod.ts` — Declared dependencies with conflict resolution |
+| Health Checks | ✅ | `plugin-lifecycle-advanced.zod.ts` — Per-plugin health + system aggregation |
+| Hot Reload | ✅ | `plugin-loading.zod.ts` — Dev + production-safe with rollback and draining |
+| Plugin Isolation | ✅ | `plugin-loading.zod.ts` — Configurable scope + IPC for process boundaries |
+| Dynamic Loading | ✅ | `plugin-runtime.zod.ts` — Runtime load/unload with activation events and discovery |

packages/spec/PROTOCOL_MAP.md

@@ -208,12 +208,13 @@ This document serves as the **Grand Map** of the ObjectStack specification. It l
 | [`feature.zod.ts`](src/kernel/feature.zod.ts) | | **Feature Flags**. Toggleable system features. |
 | [`service-registry.zod.ts`](src/kernel/service-registry.zod.ts) | | **Service Registry**. Internal dependency injection. |
 | [`metadata-loader.zod.ts`](src/kernel/metadata-loader.zod.ts) | | **Loader**. Logic for loading definitions from disk/DB. |
-| [`plugin-loading.zod.ts`](src/kernel/plugin-loading.zod.ts) | | **Plugin Loading**. Phases of plugin initialization. |
+| [`plugin-loading.zod.ts`](src/kernel/plugin-loading.zod.ts) | ⭐ | **Plugin Loading**. Loading strategies, production-safe hot reload (`environment`, `productionSafety`), full plugin sandboxing (`scope`, `ipc`), code splitting, caching, and performance monitoring. |
+| [`plugin-runtime.zod.ts`](src/kernel/plugin-runtime.zod.ts) | ⭐ | **Dynamic Loading**. Runtime load/unload of plugins (`DynamicLoadRequest`, `DynamicUnloadRequest`), activation events, plugin discovery from registries/directories, and source resolution (npm/local/url/registry/git). |
 | [`plugin-versioning.zod.ts`](src/kernel/plugin-versioning.zod.ts) | | **Versioning**. Semantic versioning rules for plugins. |
 | [`plugin-validator.zod.ts`](src/kernel/plugin-validator.zod.ts) | | **Validation**. Integrity checks for plugins. |
 | [`plugin-structure.zod.ts`](src/kernel/plugin-structure.zod.ts) | | **Structure**. Zod rules for folder layout and file naming. |
 | [`plugin-capability.zod.ts`](src/kernel/plugin-capability.zod.ts) | | **Capabilities**. What a plugin can do. |
 | [`plugin-lifecycle-events.zod.ts`](src/kernel/plugin-lifecycle-events.zod.ts) | | **Lifecycle Events**. Hooks for plugin state changes. |
-| [`plugin-lifecycle-advanced.zod.ts`](src/kernel/plugin-lifecycle-advanced.zod.ts) | | **Advanced Lifecycle**. Deep lifecycle hooks. |
-| [`plugin-security-advanced.zod.ts`](src/kernel/plugin-security-advanced.zod.ts) | | **Advanced Security**. Sandboxing and isolation. |
+| [`plugin-lifecycle-advanced.zod.ts`](src/kernel/plugin-lifecycle-advanced.zod.ts) | | **Advanced Lifecycle**. Health monitoring, hot reload state management, graceful degradation, and update strategies. |
+| [`plugin-security-advanced.zod.ts`](src/kernel/plugin-security-advanced.zod.ts) | | **Advanced Security**. Permission system, sandbox configuration (V8/WASM/container/process), security scanning, and trust levels. |
 | [`startup-orchestrator.zod.ts`](src/kernel/startup-orchestrator.zod.ts) | | **Startup**. Boot sequence orchestration. |

packages/spec/src/kernel/index.ts

@@ -8,6 +8,7 @@ export * from './plugin-capability.zod';
 export * from './plugin-lifecycle-advanced.zod';
 export * from './plugin-lifecycle-events.zod';
 export * from './plugin-loading.zod';
+export * from './plugin-runtime.zod';
 export * from './plugin-security-advanced.zod';
 export * from './plugin-structure.zod';
 export * from './plugin-validator.zod';

packages/spec/src/kernel/plugin-loading.test.ts

@@ -209,6 +209,51 @@ describe('Plugin Loading Protocol', () => {
       expect(result.enabled).toBe(false);
       expect(result.strategy).toBe('full');
       expect(result.debounceMs).toBe(300);
+      expect(result.environment).toBe('development');
+    });
+
+    it('should accept production environment with safety config', () => {
+      const config = {
+        enabled: true,
+        environment: 'production' as const,
+        strategy: 'state-preserve' as const,
+        productionSafety: {
+          healthValidation: true,
+          rollbackOnFailure: true,
+          healthTimeout: 60000,
+          drainConnections: true,
+          drainTimeout: 30000,
+          maxConcurrentReloads: 2,
+          minReloadInterval: 10000,
+        },
+      };
+      const result = PluginHotReloadSchema.parse(config);
+      expect(result.environment).toBe('production');
+      expect(result.productionSafety?.healthValidation).toBe(true);
+      expect(result.productionSafety?.rollbackOnFailure).toBe(true);
+      expect(result.productionSafety?.maxConcurrentReloads).toBe(2);
+    });
+
+    it('should apply production safety defaults', () => {
+      const config = {
+        enabled: true,
+        environment: 'production' as const,
+        productionSafety: {},
+      };
+      const result = PluginHotReloadSchema.parse(config);
+      expect(result.productionSafety?.healthValidation).toBe(true);
+      expect(result.productionSafety?.rollbackOnFailure).toBe(true);
+      expect(result.productionSafety?.drainConnections).toBe(true);
+      expect(result.productionSafety?.maxConcurrentReloads).toBe(1);
+      expect(result.productionSafety?.minReloadInterval).toBe(5000);
+    });
+
+    it('should accept all environment values', () => {
+      const envs = ['development', 'staging', 'production'];
+      envs.forEach((env) => {
+        const result = PluginHotReloadSchema.parse({ environment: env });
+        expect(result.environment).toBe(env);
+      });
     });
   });
 
@@ -269,6 +314,49 @@ describe('Plugin Loading Protocol', () => {
       const result = PluginSandboxingSchema.parse({});
       expect(result.enabled).toBe(false);
       expect(result.isolationLevel).toBe('none');
+      expect(result.scope).toBe('automation-only');
+    });
+
+    it('should accept all isolation scope values', () => {
+      const scopes = ['automation-only', 'untrusted-only', 'all-plugins'];
+      scopes.forEach((scope) => {
+        const result = PluginSandboxingSchema.parse({ scope });
+        expect(result.scope).toBe(scope);
+      });
+    });
+
+    it('should accept full plugin isolation with IPC', () => {
+      const config = {
+        enabled: true,
+        scope: 'all-plugins' as const,
+        isolationLevel: 'process' as const,
+        ipc: {
+          enabled: true,
+          transport: 'unix-socket' as const,
+          maxMessageSize: 2097152,
+          timeout: 15000,
+          allowedServices: ['metadata', 'data', 'auth'],
+        },
+      };
+      const result = PluginSandboxingSchema.parse(config);
+      expect(result.scope).toBe('all-plugins');
+      expect(result.ipc?.enabled).toBe(true);
+      expect(result.ipc?.transport).toBe('unix-socket');
+      expect(result.ipc?.allowedServices).toHaveLength(3);
+    });
+
+    it('should apply IPC defaults', () => {
+      const config = {
+        enabled: true,
+        scope: 'all-plugins' as const,
+        isolationLevel: 'process' as const,
+        ipc: {},
+      };
+      const result = PluginSandboxingSchema.parse(config);
+      expect(result.ipc?.enabled).toBe(true);
+      expect(result.ipc?.transport).toBe('message-port');
+      expect(result.ipc?.maxMessageSize).toBe(1048576);
+      expect(result.ipc?.timeout).toBe(30000);
     });
   });
 
@@ -401,6 +489,9 @@ describe('Plugin Loading Protocol', () => {
         'cache-hit',
         'cache-miss',
         'hot-reload',
+        'dynamic-load',
+        'dynamic-unload',
+        'dynamic-discover',
       ];
 
       types.forEach((type) => {
@@ -438,6 +529,8 @@ describe('Plugin Loading Protocol', () => {
         'ready',
         'failed',
         'reloading',
+        'unloading',
+        'unloaded',
       ];
 
       states.forEach((stateValue) => {

packages/spec/src/kernel/plugin-loading.zod.ts

@@ -291,14 +291,26 @@ export const PluginDependencyResolutionSchema = z.object({
 
 /**
  * Plugin Hot Reload Configuration
- * Enables hot module replacement for development
+ * Enables hot module replacement for development and production environments.
+ * 
+ * Production mode adds safety features: health validation, rollback on failure,
+ * connection draining, and concurrency control for zero-downtime reloads.
  */
 export const PluginHotReloadSchema = z.object({
   /**
    * Enable hot reload
    */
   enabled: z.boolean().default(false),
 
+  /**
+   * Target environment for hot reload behavior
+   */
+  environment: z.enum([
+    'development',   // Fast reload with relaxed safety (file watchers, no health validation)
+    'staging',       // Production-like reload with validation but relaxed rollback
+    'production',    // Full safety: health validation, rollback, connection draining
+  ]).default('development').describe('Target environment controlling safety level'),
+  
   /**
    * Hot reload strategy
    */
@@ -347,6 +359,54 @@ export const PluginHotReloadSchema = z.object({
     afterReload: z.string().optional().describe('Function to call after reload'),
     onError: z.string().optional().describe('Function to call on reload error'),
   }).optional(),
+  
+  /**
+   * Production safety configuration
+   * Applied when environment is 'staging' or 'production'
+   */
+  productionSafety: z.object({
+    /**
+     * Validate plugin health before completing reload
+     */
+    healthValidation: z.boolean().default(true)
+      .describe('Run health checks after reload before accepting traffic'),
+    
+    /**
+     * Automatically rollback to previous version on reload failure
+     */
+    rollbackOnFailure: z.boolean().default(true)
+      .describe('Auto-rollback if reloaded plugin fails health check'),
+    
+    /**
+     * Maximum time to wait for health validation after reload (ms)
+     */
+    healthTimeout: z.number().int().min(1000).default(30000)
+      .describe('Health check timeout after reload in ms'),
+    
+    /**
+     * Drain active connections before reload
+     */
+    drainConnections: z.boolean().default(true)
+      .describe('Gracefully drain active requests before reloading'),
+    
+    /**
+     * Maximum time to wait for connection draining (ms)
+     */
+    drainTimeout: z.number().int().min(0).default(15000)
+      .describe('Max wait time for connection draining in ms'),
+    
+    /**
+     * Maximum number of concurrent plugin reloads
+     */
+    maxConcurrentReloads: z.number().int().min(1).default(1)
+      .describe('Limit concurrent reloads to prevent system instability'),
+    
+    /**
+     * Minimum interval between reloads of the same plugin (ms)
+     */
+    minReloadInterval: z.number().int().min(1000).default(5000)
+      .describe('Cooldown period between reloads of the same plugin'),
+  }).optional(),
 }).describe('Plugin hot reload configuration');
 
 /**
@@ -409,14 +469,26 @@ export const PluginCachingSchema = z.object({
 
 /**
  * Plugin Sandboxing Configuration
- * Security isolation for untrusted plugins
+ * Security isolation for plugins with configurable scope.
+ * 
+ * Supports isolation beyond automation scripts: any plugin can be sandboxed
+ * with process-level isolation and inter-plugin communication (IPC).
  */
 export const PluginSandboxingSchema = z.object({
   /**
    * Enable sandboxing
    */
   enabled: z.boolean().default(false),
 
+  /**
+   * Isolation scope - which plugins are subject to sandboxing
+   */
+  scope: z.enum([
+    'automation-only',  // Sandbox automation/scripting plugins only (current behavior)
+    'untrusted-only',   // Sandbox plugins below a trust threshold
+    'all-plugins',      // Sandbox all plugins (maximum isolation)
+  ]).default('automation-only').describe('Which plugins are subject to isolation'),
+  
   /**
    * Sandbox isolation level
    */
@@ -482,6 +554,47 @@ export const PluginSandboxingSchema = z.object({
      */
     allowedEnvVars: z.array(z.string()).optional(),
   }).optional(),
+  
+  /**
+   * Inter-Plugin Communication (IPC) configuration
+   * Enables isolated plugins to communicate with the kernel and other plugins
+   */
+  ipc: z.object({
+    /**
+     * Enable IPC for sandboxed plugins
+     */
+    enabled: z.boolean().default(true)
+      .describe('Allow sandboxed plugins to communicate via IPC'),
+    
+    /**
+     * IPC transport mechanism
+     */
+    transport: z.enum([
+      'message-port',   // MessagePort (worker threads / Web Workers)
+      'unix-socket',    // Unix domain sockets (process isolation)
+      'tcp',            // TCP sockets (container isolation)
+      'memory',         // Shared memory channel (in-process VM)
+    ]).default('message-port')
+      .describe('IPC transport for cross-boundary communication'),
+    
+    /**
+     * Maximum message size in bytes
+     */
+    maxMessageSize: z.number().int().min(1024).default(1048576)
+      .describe('Maximum IPC message size in bytes (default 1MB)'),
+    
+    /**
+     * Message timeout in milliseconds
+     */
+    timeout: z.number().int().min(100).default(30000)
+      .describe('IPC message response timeout in ms'),
+    
+    /**
+     * Allowed service calls through IPC
+     */
+    allowedServices: z.array(z.string()).optional()
+      .describe('Service names the sandboxed plugin may invoke via IPC'),
+  }).optional(),
 }).describe('Plugin sandboxing configuration');
 
 /**
@@ -579,7 +692,7 @@ export const PluginLoadingConfigSchema = z.object({
   dependencyResolution: PluginDependencyResolutionSchema.optional(),
 
   /**
-   * Hot reload configuration (development only)
+   * Hot reload configuration (development and production)
    */
   hotReload: PluginHotReloadSchema.optional(),
 
@@ -619,6 +732,9 @@ export const PluginLoadingEventSchema = z.object({
     'cache-hit',
     'cache-miss',
     'hot-reload',
+    'dynamic-load',       // Plugin loaded at runtime
+    'dynamic-unload',     // Plugin unloaded at runtime
+    'dynamic-discover',   // Plugin discovered via registry
   ]),
 
   /**
@@ -672,6 +788,8 @@ export const PluginLoadingStateSchema = z.object({
     'ready',         // Fully initialized and ready
     'failed',        // Failed to load or initialize
     'reloading',     // Hot reloading in progress
+    'unloading',     // Being unloaded at runtime
+    'unloaded',      // Successfully unloaded (dynamic loading)
   ]),
 
   /**