feat(http_server source): add configuration for custom HTTP responses#25108
feat(http_server source): add configuration for custom HTTP responses#25108stigglor wants to merge 12 commits into
Conversation
github-actions
Bot
added
domain: sources
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4325165f04
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
Hi there, I've added a Thanks! |
pront
added
the
meta: awaiting author
github-actions
Bot
removed
the
meta: awaiting author
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a6f8cb6840
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a43240ae4e
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Adds a `response_source` configuration option to the `http_server` source that accepts a VRL program to generate a custom HTTP response for each request. Closes: vectordotdev#21013
stigglor
changed the title
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 27d967bf57
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
- Added a note for Non-log events being dropped - Added a note for EventMetadata::default()` loses vector-namespace metadata - Added fix for u16 truncation wraps out-of-range status codes - Added fix for JSON abort message without control keys silently drops body
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 689682438b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 05b45681be
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
github-actions
Bot
added
the
docs review on hold
The master merge added response_source and reject_code to SimpleHttpSource. This sets both fields in the three enrich_events tests so the test build compiles, fixes import ordering in the http prelude, and regenerates the http and http_server component docs.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a7c0cea50a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
… of dropping events A response_source program builds response headers from event data, so an invalid header name or value (for example a value containing CR/LF) put the response builder into an error state. build_vrl_response then returned a 500 before send_batch ran, turning a malformed optional header into dropped input events. Validate each header name and value up front and insert via headers_mut, skipping any invalid header with a warning. The accepted batch is always forwarded, and the rest of the response is returned intact.
…at build compile_response_source only checked that the VRL program compiled, so a program like `response_source = "1"` was accepted even though the runtime can only turn a string or object into a response. Such a value hit the `unexpected type` branch at request time, returned a 500, and dropped the batch. Validate the program's result type at build, rejecting anything that cannot be a string or object. Programs that always `abort` have a `never` result type and are still allowed.
The response_source program builds its input from log events only. When a decoder that emits metric or trace events (native, native_json, otlp) is configured, those events are still forwarded to the sink but are absent from `.`, so logic such as `length(.)` accounts only for log events. Document this so the limitation is explicit.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 966bff655c
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
… build
The return-type check accepted any program whose result contained a supported kind, so a
union like `if cond { "ok" } else { 1 }` passed and then dropped the batch with a 500 on
the integer branch. Require the result to be a subset of the supported kinds (string,
object, or an abort-only `never`).
3 participants
By default the
http_serversource returns a fixed status code with an empty body. This PR lets users define a VRL program that builds the response per request, so the source can return custom status codes, bodies, and headers, for example to validate payloads and reject bad input with a meaningful error.The program receives the decoded, enriched events as input (
.is an array of event objects) and returns either:response_code, orstatus,body, andheadersfields.abort:abortsuppresses event forwarding and responds immediately.500or400.abortresponses are sent before events reach the sink and are never overridden.How did you test this PR?
Added unit tests which cover the response shapes (string and object returns, custom headers), build-time validation of the VRL program,
abort-based early rejection (including JSON object messages andreject_codefallbacks), and the acknowledgement interaction (sink failure overrides a normal response, but never anabortresponse).Also verified end-to-end with a pipeline using a
response_sourceprogram with if/else branches returning different status codes, headers, and bodies.Change Type
Is this a breaking change?
Does this PR include user facing changes?
no-changeloglabel to this PR.References
Closes: #21013