Skip to content

jw/doristemp#4457

Draft
josephwoodward wants to merge 7 commits into
mainfrom
jw/doristemp
Draft

jw/doristemp#4457
josephwoodward wants to merge 7 commits into
mainfrom
jw/doristemp

Conversation

@josephwoodward
Copy link
Copy Markdown
Contributor

  • doris: add stream load output
  • doris: go mod tidy
  • doris: updating docs

Comment thread internal/impl/doris/integration_test.go Outdated
@claude
Copy link
Copy Markdown

claude Bot commented May 26, 2026

Commits

  1. The third commit message doris: updating docs uses gerund (updating) instead of imperative mood. Per the commit policy, messages must use imperative mood (e.g. update, not updating/updated).
  2. The PR title jw/doristemp appears to be a placeholder/branch name rather than a descriptive title. Consider renaming to something like doris: add stream load output.

Review

Solid new output for Apache Doris Stream Load with FE→BE redirect handling, label-based idempotency, optional query-port reachability check, multi-FE failover, and good unit-test coverage of the HTTP behavior.

  1. The integration test in internal/impl/doris/integration_test.go is built on github.com/ory/dockertest/v3 instead of testcontainers-go, which is required by the project's test patterns (see inline comment).

@claude
Copy link
Copy Markdown

claude Bot commented May 26, 2026

Commits

  1. doris: updating docs uses gerund "updating" instead of imperative mood. Per CLAUDE.md commit policy, messages must use imperative mood (e.g. "add", "fix", not "added", "updating"). Suggest renaming to doris: update docs.

Review
New doris_stream_load batch output (community-tier) with FE→BE redirect handling, multi-FE failover, label-based idempotency, group_commit awareness, and integration test using testcontainers-go. Registration, license headers, public wrapper, bundle import, and info.csv entry all follow project patterns correctly.

LGTM

Comment on lines +641 to +657
cfg.Timeout = 5 * time.Second
cfg.ReadTimeout = 5 * time.Second
cfg.WriteTimeout = 5 * time.Second

db, err := sql.Open("mysql", cfg.FormatDSN())
if err != nil {
return fmt.Errorf("opening Doris query_port connection to %s:%d: %w", host, queryPort, err)
}
defer db.Close()

qctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()

if err := db.PingContext(qctx); err != nil {
return fmt.Errorf("pinging Doris query_port %s:%d: %w", host, queryPort, err)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardcoded timeouts in connectionCheck. The MySQL driver Dial/Read/Write timeouts and the context.WithTimeout deadline are all hardcoded to 5 * time.Second, while the component already exposes a configurable timeout field (dsFieldTimeout, d.conf.Timeout).

Per the project Go patterns in .claude/agents/godev.md:

Configurable Time Parameters: Every time-related value (timeouts, backoffs, intervals, retry delays) must be exposed as a YAML-configurable field. Do not hardcode durations.

Either reuse d.conf.Timeout here (consistent with the HTTP client) or add a dedicated configurable field (e.g. query_port_timeout).

Comment thread internal/impl/doris/integration_test.go
@claude
Copy link
Copy Markdown

claude Bot commented May 26, 2026

Commits

  1. Commit doris: updating docs (fbd0b8c) — message is not in imperative mood. Should be doris: update docs per the commit policy (message uses imperative mood, e.g., "add", "fix", not "added", "fixes").
  2. Two commits with the identical message doris: migrate integration test to testcontainers are present (6cb29075 at 15:37:04 and e7adc1ac at 15:40:10, three minutes apart). Looks like an unsquashed fixup — please squash before merge.

Review

The new doris_stream_load output is well-structured and follows most project patterns (config spec, field constants, registration via MustRegisterBatchOutput, public wrapper + community bundle import, info.csv entry, Apache 2.0 headers, ConnectionTest, BatchPolicy and MaxInFlight exposure, table-driven config tests, testcontainers-based integration test). Two issues worth fixing before merge:

  1. Compilation failure in internal/impl/doris/integration_test.go"github.com/moby/moby/api/types/container" is imported twice (lines 29 and 31). The file will not build. See the inline comment.
  2. Hardcoded timeouts in connectionCheck (internal/impl/doris/output_stream_load.go lines 647–657) — the MySQL DSN dial/read/write timeouts and the context.WithTimeout deadline are all hardcoded to 5s even though the component exposes a configurable timeout field. This conflicts with the project rule in .claude/agents/godev.md: "Every time-related value (timeouts, backoffs, intervals, retry delays) must be exposed as a YAML-configurable field. Do not hardcode durations." See the inline comment.

Comment thread internal/impl/doris/integration_test.go
Comment on lines +641 to +657
cfg.Timeout = 5 * time.Second
cfg.ReadTimeout = 5 * time.Second
cfg.WriteTimeout = 5 * time.Second

db, err := sql.Open("mysql", cfg.FormatDSN())
if err != nil {
return fmt.Errorf("opening Doris query_port connection to %s:%d: %w", host, queryPort, err)
}
defer db.Close()

qctx, cancel := context.WithTimeout(ctx, 5*time.Second)
defer cancel()

if err := db.PingContext(qctx); err != nil {
return fmt.Errorf("pinging Doris query_port %s:%d: %w", host, queryPort, err)
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hardcoded timeouts in connectionCheck. The MySQL driver Dial/Read/Write timeouts and the context.WithTimeout deadline are all hardcoded to 5 * time.Second, while the component already exposes a configurable timeout field (dsFieldTimeout, d.conf.Timeout).

Per the project Go patterns in .claude/agents/godev.md:

Configurable Time Parameters: Every time-related value (timeouts, backoffs, intervals, retry delays) must be exposed as a YAML-configurable field. Do not hardcode durations.

Either reuse d.conf.Timeout here (consistent with the HTTP client) or add a dedicated configurable field (e.g. query_port_timeout).

@claude
Copy link
Copy Markdown

claude Bot commented May 26, 2026

Commits

  1. Duplicate commit messages: doris: migrate integration test to testcontainers appears twice (6cb29075… and e7adc1ac…), and doris: fix duplicates is the only follow-up — this looks like an unsquashed rebase artifact. Consider squashing the integration-test related commits together.
  2. doris: updating docs uses non-imperative mood. Per the commit policy, messages should be imperative (e.g. doris: update docs).
  3. doris: fix duplicates is vague — the message does not describe what was duplicated or how it was resolved. A more descriptive message would help future readers.

Review

New doris_stream_load batch output connector. Implementation, unit tests, integration tests, public wrapper, bundle registration, and info.csv entry all follow the established project patterns.

LGTM

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants