Skip to content

feat(files): use multipart/form-data for /api/files/create#9521

Merged
mscolnick merged 5 commits into
mainfrom
ms/feature/files-create-multipart-upload
May 12, 2026
Merged

feat(files): use multipart/form-data for /api/files/create#9521
mscolnick merged 5 commits into
mainfrom
ms/feature/files-create-multipart-upload

Conversation

@mscolnick
Copy link
Copy Markdown
Contributor

@mscolnick mscolnick commented May 12, 2026

Switches the HTTP upload path from base64-in-JSON to multipart/form-data,
eliminating the 33% size overhead and pairing naturally with File/Blob on
the frontend. The WASM/Pyodide bridge keeps its JSON+base64 wire format
since the JS<->Py RPC boundary cannot carry multipart; both transports
share a single client-side FileCreateInput interface.

Also adds:

  • path-traversal hardening in OSFileSystem.create_file_or_directory
  • parse_multipart_request helper in marimo/_server/api/utils.py

Two followups:

  • streaming so the full file doesn't need to be loaded into memory in the server
  • batching uploads on the frontend

Copilot AI review requested due to automatic review settings May 12, 2026 18:36
@vercel
Copy link
Copy Markdown

vercel Bot commented May 12, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
marimo-docs Ready Ready Preview, Comment May 12, 2026 7:24pm

Request Review

@github-actions github-actions Bot added the bash-focus Area to focus on during release bug bash label May 12, 2026
@github-actions
Copy link
Copy Markdown

Breaking changes detected in the OpenAPI specification!

==========================================================================
==                            API CHANGE LOG                            ==
==========================================================================
                                marimo API                                
--------------------------------------------------------------------------
--                            What's Changed                            --
--------------------------------------------------------------------------
- POST   /api/files/create
  Request:
        - Added multipart/form-data
        - Deleted application/json
--------------------------------------------------------------------------
--                                Result                                --
--------------------------------------------------------------------------
                 API changes broke backward compatibility                 
--------------------------------------------------------------------------

Switches the HTTP upload path from base64-in-JSON to multipart/form-data,
eliminating the 33% size overhead and pairing naturally with File/Blob on
the frontend. The WASM/Pyodide bridge keeps its JSON+base64 wire format
since the JS<->Py RPC boundary cannot carry multipart; both transports
share a single client-side FileCreateInput interface.

Also adds:
- path-traversal hardening in OSFileSystem.create_file_or_directory
- parse_multipart_request helper in marimo/_server/api/utils.py
@mscolnick mscolnick force-pushed the ms/feature/files-create-multipart-upload branch from 3001aa8 to dcbc32c Compare May 12, 2026 18:38
@mscolnick mscolnick added the enhancement New feature or request label May 12, 2026
@github-actions
Copy link
Copy Markdown

Breaking changes detected in the OpenAPI specification!

==========================================================================
==                            API CHANGE LOG                            ==
==========================================================================
                                marimo API                                
--------------------------------------------------------------------------
--                            What's Changed                            --
--------------------------------------------------------------------------
- POST   /api/files/create
  Request:
        - Added multipart/form-data
        - Deleted application/json
--------------------------------------------------------------------------
--                                Result                                --
--------------------------------------------------------------------------
                 API changes broke backward compatibility                 
--------------------------------------------------------------------------

Copy link
Copy Markdown
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR migrates the /api/files/create HTTP endpoint from JSON+base64 uploads to multipart/form-data to reduce overhead and better support File/Blob uploads from the frontend, while keeping the WASM/Pyodide JSON+base64 path intact. It also adds filename/path-traversal hardening at the filesystem layer and introduces a shared multipart parsing utility.

Changes:

  • Switch /api/files/create to accept multipart/form-data and update frontend networking to send FormData.
  • Add parse_multipart_request() helper and corresponding server-side tests.
  • Harden OSFileSystem.create_file_or_directory() against path traversal / separator escaping and add tests.

Reviewed changes

Copilot reviewed 17 out of 17 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
tests/_server/files/test_os_file_system.py Adds parametrized tests to ensure traversal-ish names are rejected.
tests/_server/api/test_utils.py New tests for parse_multipart_request behavior (fields + file bytes).
tests/_server/api/endpoints/test_file_explorer.py Updates /api/files/create tests to use form data + multipart file upload.
pyproject.toml Adds python-multipart dependency required for Starlette form parsing.
packages/openapi/src/api.ts Updates generated OpenAPI TS types to reflect multipart request body schema.
packages/openapi/api.yaml Updates OpenAPI spec for /api/files/create to multipart/form-data and adds multipart schema.
marimo/_server/models/files.py Introduces FileCreateMultipartRequest schema and shared FileCreateType.
marimo/_server/files/os_file_system.py Adds separator/null-byte checks to prevent name-based path traversal.
marimo/_server/api/utils.py Adds parse_multipart_request() helper returning typed fields + raw file bytes.
marimo/_server/api/endpoints/file_explorer.py Switches create endpoint to multipart parsing + raw bytes handling.
marimo/_cli/development/commands.py Includes new multipart request schema in server API schema generation.
frontend/src/core/wasm/bridge.ts Keeps WASM bridge JSON by base64-encoding file bytes before RPC.
frontend/src/core/network/types.ts Adds FileCreateInput interface used by both HTTP and WASM transports.
frontend/src/core/network/requests-network.ts Sends FormData for create requests via openapi-fetch (custom init/serializer).
frontend/src/core/codemirror/markdown/commands.ts Updates image upload flow to send File instead of base64 contents.
frontend/src/core/codemirror/markdown/tests/commands.test.ts Updates expectations to assert file: expect.any(File) instead of base64 contents.
frontend/src/components/editor/file-tree/upload.tsx Updates file-tree upload to send File directly (no base64 serialization).

Comment thread pyproject.toml
Comment thread marimo/_server/files/os_file_system.py
Comment thread marimo/_server/models/files.py
Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 17 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="marimo/_server/api/utils.py">

<violation number="1" location="marimo/_server/api/utils.py:66">
P2: Use `request.form()` as an async context manager so multipart file resources are closed after parsing.</violation>
</file>
Architecture diagram
sequenceDiagram
    participant FE as Frontend (React)
    participant HTTP as HTTP Transport (requests-network.ts)
    participant WASM as WASM Bridge (bridge.ts)
    participant Server as Starlette Server (endpoint)
    participant Utils as parse_multipart_request
    participant FS as OSFileSystem
    participant Test as Test Client

    Note over FE,FS: File Upload Flow – HTTP Transport (NEW: multipart/form-data)

    FE->>FE: User drops file(s) in file tree
    FE->>FE: sendCreateFileOrFolder({ path, type, name, file })
    Note over FE,FE: NEW: FileCreateInput interface – shared by both transports
    FE->>HTTP: sendCreateFileOrFolder()
    HTTP->>HTTP: Build FormData from request fields
    Note over HTTP,HTTP: NEW: appends file as Blob with original filename
    HTTP->>Server: POST /api/files/create (multipart/form-data)
    Note over HTTP,Server: Bypass openapi-fetch JSON body, use FormData directly
    Server->>Utils: parse_multipart_request(request, FileCreateMultipartRequest)
    Utils->>Utils: request.form() -> separate string fields from UploadFile parts
    alt Valid multipart data
        Utils-->>Server: MultipartRequest(body, files dict with raw bytes)
        Server->>FS: create_file_or_directory(path, type, name, file_bytes)
        alt Valid name (no path traversal)
            FS->>FS: Validate name against "/", "\\", "..", ".", null bytes
            alt Name valid
                FS->>FS: Generate unique path via _generate_unique_path
                FS-->>Server: FileInfo
                Server-->>HTTP: 200 FileCreateResponse{ success: true }
                HTTP-->>FE: handleResponse()
                FE-->>FE: Update file tree
            else Name invalid
                FS-->>Server: ValueError
                Server-->>HTTP: 200 FileCreateResponse{ success: false }
                HTTP-->>FE: Error toast
            end
        else Missing required string fields
            Utils-->>Server: msgspec.ValidationError
            Server-->>HTTP: 200 FileCreateResponse{ success: false }
        end
    end

    Note over FE,FS: File Upload Flow – WASM/Pyodide (UNCHANGED format, updated interface)

    FE->>WASM: sendCreateFileOrFolder()
    WASM->>WASM: CHANGED: base64-encode file bytes to contents string
    Note over WASM,WASM: WASM RPC cannot carry multipart → revert to JSON+base64
    WASM->>WASM: rpc.proxy.request.bridge({ functionName, payload })
    WASM-->>FE: FileCreateResponse{ success: true }

    Note over Server,FS: Path Traversal Hardening (NEW in OSFileSystem)

    Test->>FS: create_file_or_directory("path", "file", "../bad.txt", b"data")
    FS-->>Test: ValueError("Invalid name …")
    Note over Test,FS: Also rejects names with "\", null bytes, ".", ".."
Loading

Reply with feedback, questions, or to request a fix. Tag @cubic-dev-ai to re-run a review.

Comment thread marimo/_server/api/utils.py Outdated
Addresses Copilot review on #9521 — `python-multipart>=0.0.18` was
added to `project.dependencies` but the snapshot test was not updated.
@mscolnick mscolnick requested a review from akshayka as a code owner May 12, 2026 18:54
@github-actions
Copy link
Copy Markdown

Breaking changes detected in the OpenAPI specification!

==========================================================================
==                            API CHANGE LOG                            ==
==========================================================================
                                marimo API                                
--------------------------------------------------------------------------
--                            What's Changed                            --
--------------------------------------------------------------------------
- POST   /api/files/create
  Request:
        - Added multipart/form-data
        - Deleted application/json
--------------------------------------------------------------------------
--                                Result                                --
--------------------------------------------------------------------------
                 API changes broke backward compatibility                 
--------------------------------------------------------------------------

Addresses cubic review on #9521 — `request.form()` should be used as an
async context manager so any spooled temp files backing UploadFile
parts are closed after parsing.
@github-actions
Copy link
Copy Markdown

Breaking changes detected in the OpenAPI specification!

==========================================================================
==                            API CHANGE LOG                            ==
==========================================================================
                                marimo API                                
--------------------------------------------------------------------------
--                            What's Changed                            --
--------------------------------------------------------------------------
- POST   /api/files/create
  Request:
        - Added multipart/form-data
        - Deleted application/json
--------------------------------------------------------------------------
--                                Result                                --
--------------------------------------------------------------------------
                 API changes broke backward compatibility                 
--------------------------------------------------------------------------

Top-level `from starlette.datastructures import UploadFile` broke the
pyodide bootstrap path — `parse_title` is imported transitively by
`marimo._server.templates.templates`, which runs in environments
without starlette installed. Moved the import inside
`parse_multipart_request` where it's actually used.
@mscolnick mscolnick requested a review from kirangadhave May 12, 2026 19:02
@github-actions
Copy link
Copy Markdown

Breaking changes detected in the OpenAPI specification!

==========================================================================
==                            API CHANGE LOG                            ==
==========================================================================
                                marimo API                                
--------------------------------------------------------------------------
--                            What's Changed                            --
--------------------------------------------------------------------------
- POST   /api/files/create
  Request:
        - Added multipart/form-data
        - Deleted application/json
--------------------------------------------------------------------------
--                                Result                                --
--------------------------------------------------------------------------
                 API changes broke backward compatibility                 
--------------------------------------------------------------------------

`tests/_ai/tools/test_utils.py` already exists; pytest's rootdir-based
module discovery clashes when two test files share a basename and
neither directory has an `__init__.py`. Renamed to
`tests/_server/api/test_api_utils.py`.
@github-actions
Copy link
Copy Markdown

Breaking changes detected in the OpenAPI specification!

==========================================================================
==                            API CHANGE LOG                            ==
==========================================================================
                                marimo API                                
--------------------------------------------------------------------------
--                            What's Changed                            --
--------------------------------------------------------------------------
- POST   /api/files/create
  Request:
        - Added multipart/form-data
        - Deleted application/json
--------------------------------------------------------------------------
--                                Result                                --
--------------------------------------------------------------------------
                 API changes broke backward compatibility                 
--------------------------------------------------------------------------

Copy link
Copy Markdown
Member

@kirangadhave kirangadhave left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🚀

formData.append("path", request.path);
formData.append("type", request.type);
formData.append("name", request.name);
if (request.file) {
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we want to make the request if there is no file?

@mscolnick mscolnick merged commit 16d224b into main May 12, 2026
55 of 56 checks passed
@mscolnick mscolnick deleted the ms/feature/files-create-multipart-upload branch May 12, 2026 19:49
@github-actions
Copy link
Copy Markdown

🚀 Development release published. You may be able to view the changes at https://marimo.app?v=0.23.7-dev8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bash-focus Area to focus on during release bug bash dependencies enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants