Skip to content

feat: add form data and file upload support#21

Merged
vigo merged 11 commits into
mainfrom
issue-4
Jan 23, 2026
Merged

feat: add form data and file upload support#21
vigo merged 11 commits into
mainfrom
issue-4

Conversation

@vigo

@vigo vigo commented Jan 23, 2026

Copy link
Copy Markdown
Member

Summary

  • Add application/x-www-form-urlencoded content type support
  • Add multipart/form-data content type support for file uploads
  • Display form data and file metadata in terminal and web dashboard
  • Show image preview in web dashboard for uploaded images (JPEG, PNG, GIF, etc.)
  • Sanitize binary content in raw HTTP output to avoid terminal garbage

Changes

  • httpserver.go: Parse form data and multipart requests, extract files with base64 encoding for images
  • requeststore/store.go: Add FileAttachment struct and Files field to Request
  • webui/static/index.html: Display form data tables, file info, and image previews
  • README.md: Document new features with curl examples

Test plan

  • Run go test -race ./... - all tests pass
  • Run golangci-lint run ./... - no issues
  • Manual test: form urlencoded data displays correctly
  • Manual test: multipart form data with files displays correctly
  • Manual test: image preview shows in web dashboard
  • Manual test: binary data shows as [binary data: X KB] in terminal

Closes #4

🤖 Generated with Claude Code

vigo and others added 3 commits January 23, 2026 13:08
- Add application/x-www-form-urlencoded content type support
- Parse and display form data in table format (CLI and WebUI)
- Multiple values for same key shown comma-separated
- URL-encoded characters automatically decoded
- Fix data race in run.go (stopErr variable)
- Fix SSE connection initialization (send connected comment)
- Fix SSE event ordering in WebUI (load after connect)
- Update header padding alignment in WebUI
- Update tests for SSE changes (io.Pipe unbuffered reads)

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- Switch pre-commit-golang repo to TekWizely/pre-commit-golang
- Use golangci-lint-mod for full module linting
- Update golangci.yml errcheck exclusions for fmt.Fprint

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
- add multipart/form-data content type support for file uploads
- display file metadata (filename, size, content-type) in terminal
- show content for small text files (under 1KB)
- sanitize binary content in raw HTTP output ([binary data: X KB])
- add image preview in web dashboard for uploaded images
- store file attachments with base64-encoded data for WebUI
- add comprehensive tests for multipart handling

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@codecov-commenter

codecov-commenter commented Jan 23, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 84.54936% with 36 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.46%. Comparing base (90ad0ce) to head (c9736c3).

Files with missing lines Patch % Lines
internal/httpserver/httpserver.go 85.15% 26 Missing and 8 partials ⚠️
internal/httpserver/run.go 0.00% 2 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main      #21      +/-   ##
==========================================
+ Coverage   76.06%   78.46%   +2.39%     
==========================================
  Files           9        9              
  Lines         631      859     +228     
==========================================
+ Hits          480      674     +194     
- Misses        136      162      +26     
- Partials       15       23       +8     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

- add tests for PNG, JPEG, GIF image uploads
- add test for mixed files and fields with binary detection
- improve coverage for image content type detection
- improve coverage for binary data sanitization

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b5b96895e6

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/webui/static/index.html Outdated
Comment on lines +973 to +976
eventSource.onopen = () => {
setStatus(true);
// Load initial requests after SSE is connected
loadInitialRequests();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Load initial requests even if SSE never opens

Because loadInitialRequests() is only invoked inside eventSource.onopen, the dashboard now depends on the SSE connection opening before any historical requests are fetched. If EventSource is blocked by a proxy/CSP or the SSE endpoint fails to connect, the request list stays empty even though /api/requests would still be reachable, which is a regression from the prior unconditional load. Consider calling loadInitialRequests() on page load (or in onerror as a fallback) so the UI still works without SSE.

Useful? React with 👍 / 👎.

Comment on lines +573 to +575
if len(fi.RawData) > 0 {
sf.Data = base64.StdEncoding.EncodeToString(fi.RawData)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Cap base64 image data to avoid unbounded memory

For image parts, the full file bytes are base64‑encoded and stored without any size guard. A large upload (e.g., tens/hundreds of MB) will be retained in memory and broadcast over SSE to every dashboard client, which can significantly increase memory/CPU usage and stall the UI. Consider skipping Data or enforcing a maximum size before encoding to keep worst‑case payloads bounded.

Useful? React with 👍 / 👎.

vigo and others added 2 commits January 23, 2026 15:22
- Load initial requests on page load instead of only on SSE open
- Add fallback to load requests on SSE error
- Cap image preview base64 encoding to 5MB max to prevent memory issues

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
Add a note explaining that the server captures all incoming requests
regardless of the URL path. The paths in examples are for illustration only.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@vigo

vigo commented Jan 23, 2026

Copy link
Copy Markdown
Member Author

@codex review

Allows manually triggering the test workflow from GitHub Actions UI
or via gh CLI.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5e80c0aead

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/httpserver/httpserver.go Outdated
Comment on lines +486 to +487
partData, _ := io.ReadAll(part)
_ = part.Close()

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid reading entire multipart parts into memory

When handling multipart uploads, this io.ReadAll(part) loads each file part into a new buffer even though the full request body was already read into body. For large uploads this doubles memory usage (body + partData) and can easily exhaust memory, especially since only size and small previews are needed. Consider streaming/discarding while counting size and only buffering up to the preview limits (e.g., 1KB for text, 5MB for images) to avoid OOM on multi‑MB/GB files.

Useful? React with 👍 / 👎.

- Only buffer what's needed for preview (1KB for text, 5MB for images)
- Discard remaining bytes while counting total size
- Prevents OOM on large file uploads

Addresses Codex review feedback.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@vigo

vigo commented Jan 23, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 533c76929d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/webui/static/index.html Outdated
Comment on lines +973 to +976
eventSource.onopen = () => {
setStatus(true);
// Load initial requests after SSE is connected
loadInitialRequests();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Avoid overwriting SSE updates on connect

When the SSE connection opens, the handler calls loadInitialRequests() which replaces the in-memory requests array. If a new request arrives via SSE between the onopen call and the async fetch resolving, addRequest will prepend it, then loadInitialRequests will overwrite the list and silently drop that new request. This makes the dashboard miss requests under normal traffic. Consider loading initial requests before opening SSE, or merging fetched data instead of replacing the array.

Useful? React with 👍 / 👎.

- Load initial requests BEFORE connecting SSE
- Remove redundant load from onopen to avoid overwriting SSE data
- Track initialLoadDone to only fallback-load once on SSE failure

Prevents requests arriving via SSE from being silently dropped
when loadInitialRequests() overwrites the array.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@vigo

vigo commented Jan 23, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: ebdad3c092

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +667 to +669
if (file.data && isImageContentType(file.contentType)) {
const dataUrl = `data:${file.contentType};base64,${file.data}`;
fileInfoContent += `<div class="image-preview"><img src="${dataUrl}" alt="${escapeHtml(file.filename)}" /></div>`;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Sanitize image content-type before building data URL

file.contentType is taken from the multipart part headers (attacker-controlled) and is interpolated directly into the src attribute without escaping. If a requester sends a file part with a header like Content-Type: image/png" onerror="alert(1) the generated HTML becomes <img src="data:image/png" onerror="alert(1);base64,...">, which executes script when the preview renders. Consider whitelisting/normalizing content types to a safe set (e.g., image/png, image/jpeg, image/gif) or escaping the value before embedding it in the data URL.

Useful? React with 👍 / 👎.

Add whitelist of safe image MIME types (png, jpeg, gif, webp, bmp, x-icon).
Content-Type header is attacker-controlled and was interpolated directly
into data URL, allowing XSS via headers like:
  Content-Type: image/png" onerror="alert(1)

Now only whitelisted types are used; others fall back to application/octet-stream.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@vigo

vigo commented Jan 23, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4c9a5b2507

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread internal/httpserver/httpserver.go Outdated
Comment on lines +643 to +645
if bodyAsString != "" {
fmt.Fprintf(mwr, "\n%s\n", bodyAsString)
sanitizedBody := sanitizeBodyForDisplay(bodyAsString, r.Header.Get(headerContentType))
fmt.Fprintf(mwr, "\n%s\n", sanitizedBody)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve raw body when saving raw HTTP requests

When -save-raw-http-request is enabled, the raw request file is written through mwr, and the body now goes through sanitizeBodyForDisplay. That means multipart uploads or binary bodies are replaced with placeholders like [binary data: …] in the saved file, so the file is no longer a faithful capture and can't be replayed with nc as documented. This change only affects saves when the body contains binary data, but in that case the saved payload is corrupted. Consider writing the unsanitized body to rawHRw while keeping the sanitized output only for the terminal writer.

Useful? React with 👍 / 👎.

Terminal output gets sanitized body (no binary garbage).
Raw file gets unsanitized body for faithful capture and replay with nc.

Previously both outputs received sanitized body, corrupting saved files
and breaking the documented `nc localhost 9002 < file.raw` workflow.

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>
@vigo

vigo commented Jan 23, 2026

Copy link
Copy Markdown
Member Author

@codex review

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: c9736c31a8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +671 to +674
const fileRows = files.map(file => {
const sizeStr = formatFileSize(file.size);
let fileInfoContent = `${escapeHtml(file.filename)} | ${sizeStr} | ${escapeHtml(file.contentType)}`;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Render text file contents in Web UI

When multipart uploads include files, formatBody always routes to formatMultipartWithFiles, but this code path only renders filename | size | contentType (and optionally an image preview). It never renders inline text content for small text files, even though the fallback multipart renderer below does, so users uploading a small .txt/.json file will see metadata only in the web dashboard. This diverges from the “small text files display content” behavior and makes the UI inconsistent with terminal output. Consider either sending text content in the files payload or reusing the parsed multipart content in this path.

Useful? React with 👍 / 👎.

@vigo vigo merged commit 12678c0 into main Jan 23, 2026
2 checks passed
@vigo vigo deleted the issue-4 branch January 23, 2026 13:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature Requests

2 participants