doris: add stream load output#4218
Conversation
There was a problem hiding this comment.
Pull request overview
This PR introduces a new Apache Doris Stream Load output (doris_stream_load) and wires it into the community component bundle so it’s available in standard builds.
Changes:
- Adds a new
doris_stream_loadbatch output implementation with FE→BE redirect handling, JSON/CSV encoding, and FE failover support. - Registers the Doris component in
public/componentsand the plugin catalog. - Adds unit tests covering encoding, redirects/failover, header mapping, and connection tests.
Reviewed changes
Copilot reviewed 5 out of 5 changed files in this pull request and generated 2 comments.
Show a summary per file
| File | Description |
|---|---|
| public/components/doris/package.go | Adds the public component package that side-effect imports the internal Doris implementation. |
| public/components/community/package.go | Wires the new Doris public component into the community build import set. |
| internal/plugins/info.csv | Registers doris_stream_load in the plugin inventory. |
| internal/impl/doris/output_stream_load.go | Implements the Doris Stream Load output, config parsing/spec, request building, redirect/failover logic, and connection tests. |
| internal/impl/doris/output_stream_load_test.go | Adds unit tests for encoding, redirects, failover, connection tests, and promoted header behavior. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
|
|
1 similar comment
|
|
Summary
doris_stream_loadoutputExpect: 100-continue, query-port based connection tests, and FE failover viafe_urlsValidation
go test ./internal/impl/dorisgo build -o target/redpanda-connect ./cmd/redpanda-connectNotes