Skip to content

fix(hub): raise Socket.IO maxHttpBufferSize for file downloads#516

Open
GeT-LeFt wants to merge 2 commits intotiann:mainfrom
GeT-LeFt:fix/socket-maxhttpbuffersize
Open

fix(hub): raise Socket.IO maxHttpBufferSize for file downloads#516
GeT-LeFt wants to merge 2 commits intotiann:mainfrom
GeT-LeFt:fix/socket-maxhttpbuffersize

Conversation

@GeT-LeFt
Copy link
Copy Markdown
Contributor

Summary

Socket.IO Engine defaults maxHttpBufferSize to 1MB. File downloads in HAPI go through WebSocket RPC (the CLI reads a file → base64-encodes it → returns it via Socket.IO ack), so any file larger than ~750KB (after base64 inflation) silently drops the response. The frontend download spinner then hangs indefinitely with no error surfaced.

Changes

  • Raise the default maxHttpBufferSize to 100MB in hub/src/socket/server.ts.
  • Expose it via a new HAPI_SOCKET_MAX_BUFFER_SIZE env var (using the existing resolveEnvNumber helper) so operators can tune it up or down based on their deployment.

Test plan

  • bun run typecheck passes
  • Manually reproduced the hang on main with a ~2MB file download
  • Verified downloads of files up to 50MB succeed after the change

Socket.IO Engine defaults maxHttpBufferSize to 1MB. File downloads go
through WebSocket RPC (CLI reads file -> base64 -> Socket.IO ack), so
any file >750KB (after base64 inflation) silently drops the response,
causing the frontend download spinner to hang indefinitely.

Raise the default to 100MB and expose it via the
HAPI_SOCKET_MAX_BUFFER_SIZE env var so operators can tune it.
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • [Major] maxHttpBufferSize is now 100 MiB before any namespace auth runs, while allowRequest still accepts requests with no Origin. That means an unauthenticated client can force the hub to buffer and parse much larger /socket.io/ frames than before. The existing product cap is still 50 MiB for file payloads, so the default can stay much tighter. Evidence: hub/src/socket/server.ts:67, hub/src/socket/server.ts:68, hub/src/socket/server.ts:101, hub/src/socket/server.ts:123, hub/src/web/routes/sessions.ts:43.
    Suggested fix:
    const DEFAULT_MAX_HTTP_BUFFER_SIZE = Math.ceil((50 * 1024 * 1024 * 4) / 3)
    
    function resolveEnvNumber(name: string, fallback: number, max = fallback): number {
        const raw = process.env[name]
        if (!raw) {
            return fallback
        }
        const parsed = Number.parseInt(raw, 10)
        return Number.isFinite(parsed) && parsed > 0 ? Math.min(parsed, max) : fallback
    }

Summary

  • Review mode: initial
  • One major issue found in the new socket buffer default: it expands the unauthenticated Engine.IO message budget by 100x with no matching auth/rate-limit change.
  • HAPI Bot

Testing

  • Not run (automation)

Comment thread hub/src/socket/server.ts
const engine = new Engine({
path: '/socket.io/',
cors: corsOptions,
maxHttpBufferSize: resolveEnvNumber('HAPI_SOCKET_MAX_BUFFER_SIZE', DEFAULT_MAX_HTTP_BUFFER_SIZE),
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] This raises the Engine.IO frame limit to 100 MiB before any namespace auth runs. allowRequest still accepts requests with no Origin, and the token/JWT checks only happen later in cliNs.use(...) / terminalNs.use(...), so an unauthenticated client can now make the hub buffer and parse much larger /socket.io/ frames than before.

We only need enough headroom for the existing 50 MiB file cap (hub/src/web/routes/sessions.ts:43), which is about 67 MiB after base64 inflation. Please keep the default near that ceiling and clamp the env override instead of opening a 100 MiB unauthenticated buffer.

Suggested fix:

const DEFAULT_MAX_HTTP_BUFFER_SIZE = Math.ceil((50 * 1024 * 1024 * 4) / 3)

function resolveEnvNumber(name: string, fallback: number, max = fallback): number {
    const raw = process.env[name]
    if (!raw) {
        return fallback
    }
    const parsed = Number.parseInt(raw, 10)
    return Number.isFinite(parsed) && parsed > 0 ? Math.min(parsed, max) : fallback
}

Address review feedback:
- Lower default from 100MB to ~68MB (ceil(50MB * 4/3)) to just cover
  base64 inflation of the 50MB product file cap.
- Add a max parameter to resolveEnvNumber so the env override cannot
  exceed the default, preventing unauthenticated frame budget abuse.
Copy link
Copy Markdown

@github-actions github-actions Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Findings

  • [Major] resolveEnvNumber now clamps unrelated numeric env overrides to their default values. Because the new default max = fallback applies to the existing terminal settings, HAPI_TERMINAL_IDLE_TIMEOUT_MS can no longer be raised above 15 minutes and HAPI_TERMINAL_MAX_TERMINALS can no longer be raised above 4, even though the unchanged call sites still use this helper for those knobs. Evidence: hub/src/socket/server.ts:31, hub/src/socket/server.ts:80, hub/src/socket/server.ts:81.
    Suggested fix:
    function resolveEnvNumber(name: string, fallback: number, max?: number): number {
        const raw = process.env[name]
        if (!raw) {
            return fallback
        }
        const parsed = Number.parseInt(raw, 10)
        if (!Number.isFinite(parsed) || parsed <= 0) {
            return fallback
        }
        return max === undefined ? parsed : Math.min(parsed, max)
    }
    
    maxHttpBufferSize: resolveEnvNumber(
        'HAPI_SOCKET_MAX_BUFFER_SIZE',
        DEFAULT_MAX_HTTP_BUFFER_SIZE,
        DEFAULT_MAX_HTTP_BUFFER_SIZE
    ),

Questions

  • None.

Summary

  • Review mode: follow-up after new commits
  • The prior unauthenticated 100 MiB socket buffer issue is addressed, but the follow-up introduced a broader env parsing regression for terminal hub settings.

Testing

  • Not run (automation)

HAPI Bot

Comment thread hub/src/socket/server.ts
}
const parsed = Number.parseInt(raw, 10)
return Number.isFinite(parsed) && parsed > 0 ? parsed : fallback
return Number.isFinite(parsed) && parsed > 0 ? Math.min(parsed, max) : fallback
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[MAJOR] This clamps every existing call to resolveEnvNumber to its fallback because max defaults to fallback. The socket buffer needs that cap, but the same helper still feeds HAPI_TERMINAL_IDLE_TIMEOUT_MS and HAPI_TERMINAL_MAX_TERMINALS, so users can no longer raise the hub terminal idle timeout above 15 minutes or terminal count above 4.

Suggested fix:

function resolveEnvNumber(name: string, fallback: number, max?: number): number {
    const raw = process.env[name]
    if (!raw) {
        return fallback
    }
    const parsed = Number.parseInt(raw, 10)
    if (!Number.isFinite(parsed) || parsed <= 0) {
        return fallback
    }
    return max === undefined ? parsed : Math.min(parsed, max)
}

maxHttpBufferSize: resolveEnvNumber(
    'HAPI_SOCKET_MAX_BUFFER_SIZE',
    DEFAULT_MAX_HTTP_BUFFER_SIZE,
    DEFAULT_MAX_HTTP_BUFFER_SIZE
),

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant