Skip to content

Commit 633e833

Browse files
perf: optimize hot path iteration and destructuring
Co-authored-by: google-labs-jules[bot] <161369871+google-labs-jules[bot]@users.noreply.github.com> Co-authored-by: shenald-dev <245350826+shenald-dev@users.noreply.github.com>
1 parent 92588fc commit 633e833

6 files changed

Lines changed: 27 additions & 7 deletions

File tree

.jules/bolt.md

Lines changed: 9 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -176,7 +176,7 @@ Optimized validation helper logic in `src/index.js` to strictly rely on explicit
176176
Learning: Global `express.json()` middleware forces the application to buffer and parse request bodies even for unknown or non-existent routes, exposing a Denial of Service (DoS) vulnerability via large payloads to 404 endpoints.
177177
Action: Apply `express.json()` strictly as route-specific middleware to the exact endpoints that require body parsing, and ensure dependent JSON error handlers are positioned correctly after those specific route definitions in the middleware chain.
178178

179-
## $(date +%Y-%m-%d) — Optimize Health Check Placement
179+
## 2026-04-29 — Optimize Health Check Placement
180180

181181
Learning: In Express API gateways, declaring high-frequency, simple endpoints (like `/health`) below global middleware such as `helmet` and `cors` introduces significant and unnecessary CPU parsing overhead for every ping, even if the ping does not require CORS or security headers.
182182
Action: Moved the `/health` endpoint definition above `helmet()` and `cors()` in `src/index.js`, while manually explicitly setting the `Content-Type` header. This drastically reduces CPU overhead and latency for load balancer pings while maintaining correct response headers.
@@ -188,3 +188,11 @@ In highly trafficked functions like `heavyComputation` that rely on an LRU map c
188188

189189
Action:
190190
Added an L1 cache using module-scoped variables (`lastIterations` and `lastResult`) to `heavyComputation` in `src/index.js`. This avoids redundant Map lookups and mutations for consecutive identical calls, transforming a hot path from an (1)$ Map operation to a much faster strict equality check.
191+
192+
## 2026-04-29 — Optimize Hot Path Iteration and Destructuring
193+
194+
Learning:
195+
In highly trafficked endpoints like `/v1/chat/completions`, destructuring properties directly from potentially undefined objects (e.g., `const { model, messages } = req.body || {}`) creates unnecessary object allocations for the fallback `{}` and is slower than explicitly checking for the object's existence and accessing properties directly. Additionally, using `for...of` loops incurs iterator allocation and traversal overhead compared to classic `for` loops with a cached array length, especially for large arrays like `messages`.
196+
197+
Action:
198+
Replaced the object destructuring and `|| {}` fallback with direct property access after an explicit check of `req.body`. Replaced the `for...of` loop with a classic `for` loop caching the `messages.length` in a local variable `messagesLen` before the loop. This reduces garbage collection pressure and significantly improves execution speed on hot paths.

.jules/warden.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,8 @@
1+
## 2026-04-29 — Assessment & Lifecycle
2+
Observation / Pruned:
3+
Assessed JULES/BOLT's optimization replacing object destructuring with explicit property access and fallback handling on \`req.body\`. Replaced \`for...of\` loops with classic \`for\` loops and length caching to avoid iterator allocation. Checked dependencies, no unused found. Pruned scratchpad files used for local benchmarks.
4+
Alignment / Deferred:
5+
Appended release notes to CHANGELOG.md specifying the performance improvement. Version bumped to 1.1.27.
16

27
2024-06-21 — Assessment & Lifecycle
38
Observation / Pruned:

CHANGELOG.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,7 @@
1+
## [1.1.27] - 2026-04-29
2+
### Changed
3+
- Optimized `/v1/chat/completions` parsing and validation loops.
4+
15
## [1.1.26] - 2026-04-28
26
### Changed
37
* **[Reliability]:** Fixed an issue in `heavyComputation` where the L1 cache was incorrectly returning false cache hits when `undefined` was passed as a parameter. The cache is now properly initialized with a unique `Symbol`. Zero dead code pruned.

package-lock.json

Lines changed: 2 additions & 2 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,6 @@
11
{
22
"name": "one-api",
3-
"version": "1.1.26",
3+
"version": "1.1.27",
44
"description": "One API to rule them all. Unified gateway for 20+ LLM providers. OpenAI-compatible, single binary, zero config.",
55
"main": "src/index.js",
66
"scripts": {

src/index.js

Lines changed: 6 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -96,7 +96,9 @@ const ERROR_MALFORMED_MESSAGE = Buffer.from(JSON.stringify({ error: 'Malformed m
9696
const jsonParser = express.json({ limit: '10mb' });
9797

9898
app.post('/v1/chat/completions', jsonParser, (req, res) => {
99-
const { model, messages } = req.body || {};
99+
const body = req.body;
100+
const model = body ? body.model : undefined;
101+
const messages = body ? body.messages : undefined;
100102
if (!isValidModel(model)) {
101103
return res.status(400).send(ERROR_MISSING_MODEL);
102104
}
@@ -107,8 +109,9 @@ app.post('/v1/chat/completions', jsonParser, (req, res) => {
107109
return res.status(400).send(ERROR_TOO_MANY_MESSAGES);
108110
}
109111

110-
for (const msg of messages) {
111-
if (!isValidMessage(msg)) {
112+
const messagesLen = messages.length;
113+
for (let i = 0; i < messagesLen; i++) {
114+
if (!isValidMessage(messages[i])) {
112115
return res.status(400).send(ERROR_MALFORMED_MESSAGE);
113116
}
114117
}

0 commit comments

Comments
 (0)