Skip to content

feat: configurable external middleware pipeline#1687

Open
mcarlson wants to merge 2 commits into
Portkey-AI:2.0.0from
mcarlson:feat/configurable-middleware-pipeline
Open

feat: configurable external middleware pipeline#1687
mcarlson wants to merge 2 commits into
Portkey-AI:2.0.0from
mcarlson:feat/configurable-middleware-pipeline

Conversation

@mcarlson

Copy link
Copy Markdown

Summary

Adds a configurable middleware pipeline that lets users compose request/response transformations without forking the gateway. Builds on the external loading concept from #1563 with ordering guarantees, per-middleware config, and error reporting.

  • Middleware are standard Hono middleware loaded from a directory and configured in conf.json
  • Array order = execution order — first middleware is outermost (wraps everything downstream)
  • Factory pattern — each file exports a function that receives config and returns Hono middleware
  • Fail-fast startup — gateway refuses to start if any middleware is missing or fails to initialize
  • Error attribution — runtime errors are prefixed with the middleware name for easy debugging

Config schema

{
  "middleware_dir": "./middleware",
  "middleware": [
    { "name": "request-logger", "config": { "verbose": true } },
    { "name": "cache-optimizer", "enabled": false }
  ]
}

Middleware authoring

// middleware/request-logger.js
module.exports = function(config) {
  return async (c, next) => {
    const start = Date.now();
    await next();
    console.log(`${c.req.method} ${c.req.path}${c.res.status} (${Date.now() - start}ms)`);
  };
};

Files changed

File Purpose
src/types/middlewareConfig.ts Config parser (middleware array, dir, enabled flag)
src/middlewares/external.ts Loader: import factories, call with config, register in order
src/index.ts Wire loader before hooks pipeline
conf.json Add middleware_dir and middleware fields (empty by default)
examples/middleware/request-logger.js Example middleware
src/types/__tests__/middlewareConfig.test.ts Config parser tests (6 tests)
src/middlewares/__tests__/external.test.ts Loader tests (7 tests)

Relationship to existing systems

Concern Mechanism
HTTP-level (compression, CORS, auth) Built-in Hono middleware
Request/response transformation External middleware (this PR)
Content filtering, safety checks Plugins/guardrails (hooks pipeline)

External middleware wraps the hooks pipeline — it runs before guardrails see the request and after they produce the response.

Limitations (v1)

  • Node.js only — dynamic filesystem imports don't work on Workers
  • No hot reload — restart gateway to change middleware config
  • No per-request middleware selection — same pipeline for all requests

Test plan

  • Config parser: empty config, valid config, disabled entries, CLI override, invalid entries (6 tests)
  • Loader: ordering, config passing, missing file, factory error, empty entries, async factory, error wrapping (7 tests)
  • Manual: add example middleware to conf.json, verify logs on startup and per-request

🤖 Generated with Claude Code

mcarlson and others added 2 commits June 11, 2026 21:23
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Adds a middleware pipeline that lets users compose request/response
transformations without forking the gateway.

Middleware are standard Hono middleware loaded from a configurable
directory and registered in conf.json order. Each middleware file
exports a factory function that receives a config object and returns
a Hono middleware.

Config schema (conf.json):
  - middleware_dir: path to middleware directory (default: ./middleware)
  - middleware: ordered array of { name, config?, enabled? } entries

Features:
  - Array order = execution order (first is outermost)
  - Per-middleware config object passed to factory
  - enabled: false to disable without removing from array
  - Async factory support for middleware needing initialization
  - Error reporting names the failing middleware at startup and runtime
  - Supports .js (CJS) and .mjs (ESM) middleware files
  - CLI override via MIDDLEWARES_DIR env var

Gateway refuses to start if any middleware file is missing or its
factory throws.

Builds on the external loading concept from Portkey-AI#1563 with ordering
guarantees, per-middleware config, and error reporting.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant