[BugFix] Fix ORC file metric construction in AddFilesProcedure by Guosmilesmile · Pull Request #71220 · StarRocks/starrocks

Guosmilesmile · 2026-04-02T09:44:44Z

Why I'm doing:

In AddFilesProcedure extractOrcMetrics, ORC's Reader.getStatistics() returns file-level stats at index 0, with actual column stats starting from index 1. The current code iterates from index 0, causing all column metrics to be shifted by one position.

What I'm doing:

Fix the off-by-one bug by starting the loop from colId = 1 and passing colId - 1 to getColumnNameFromOrcSchem to create correct metric in orc

Fixes #71222
Issue: #71222

What type of PR is this:

Does this PR entail a change in behavior?

Yes, this PR will result in a change in behavior.
No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

Interface/UI changes: syntax, type conversion, expression evaluation, display information
Parameter changes: default values, similar parameters but with different default values
Policy changes: use new policy to replace old one, functionality automatically enabled
Feature removed
Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

I have added test cases for my bug fix or my new feature
This pr needs user documentation (for new or modified features or behaviors)
- I have added documentation for my new feature or new function
- This pr needs auto generate documentation
This is a backport pr

Bugfix cherry-pick branch check:

Guosmilesmile · 2026-04-02T10:23:30Z

Hi @Youngwb , please help to
review if you have time. thx

Signed-off-by: chrisyguo <511955993@qq.com>

CelerData-Reviewer · 2026-04-02T10:53:57Z

@codex review

chatgpt-codex-connector · 2026-04-02T10:55:35Z

Codex Review: Didn't find any major issues. Swish!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

github-actions · 2026-04-02T12:10:56Z

[Java-Extensions Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

github-actions · 2026-04-02T12:12:23Z

[FE Incremental Coverage Report]

✅ pass : 2 / 2 (100.00%)

file detail

	path	covered_line	new_line	coverage	not_covered_line_detail
🔵	com/starrocks/connector/iceberg/procedure/AddFilesProcedure.java	2	2	100.00%	[]

github-actions · 2026-04-02T12:19:21Z

[BE Incremental Coverage Report]

✅ pass : 0 / 0 (0%)

sonarqubecloud · 2026-04-02T12:29:03Z

Quality Gate failed

Failed conditions
E Maintainability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

Copilot

Pull request overview

Fixes ORC metric extraction in the Iceberg add_files procedure by correcting how ORC Reader.getStatistics() entries are mapped to Iceberg fields, and adds a unit test to validate the expected mapping.

Changes:

Adjust ORC statistics iteration to skip the file-level/root statistics entry at index 0.
Offset ORC schema field-name lookup by -1 to align statistics indexes with top-level columns.
Add a unit test covering ORC metric extraction for a simple struct schema.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 2 comments.

File	Description
fe/fe-core/src/main/java/com/starrocks/connector/iceberg/procedure/AddFilesProcedure.java	Updates ORC metrics extraction loop and column-name mapping to address off-by-one indexing.
fe/fe-core/src/test/java/com/starrocks/connector/iceberg/procedure/AddFilesProcedureTest.java	Adds a reflective unit test validating ORC metrics (recordCount/valueCounts/nullValueCounts) mapping.

Guosmilesmile · 2026-04-07T09:26:06Z

Hey @kevincai, thanks for the feedback! What should I do to help push this PR forward?

kevincai · 2026-04-07T09:29:48Z

Hey @kevincai, thanks for the feedback! What should I do to help push this PR forward?

@Guosmilesmile
~~I am trying to figure out the ORC spec that stats about the columns statistics structure where the stats[0] is the file stat instead of column stats~~

kevincai · 2026-04-07T12:05:54Z

@Mergifyio backport branch-4.1 branch-4.0

mergify · 2026-04-07T12:06:11Z

backport branch-4.1 branch-4.0

✅ Backports have been created

Details

#71379 [BugFix] Fix ORC file metric construction in AddFilesProcedure (backport #71220) has been created for branch branch-4.1 but encountered conflicts

Cherry-pick of ca4bd59 has failed:

On branch mergify/bp/branch-4.1/pr-71220
Your branch is up to date with 'origin/branch-4.1'.

You are currently cherry-picking commit ca4bd593ee.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Changes to be committed:
	modified:   fe/fe-core/src/main/java/com/starrocks/connector/iceberg/procedure/AddFilesProcedure.java

Unmerged paths:
  (use "git add <file>..." to mark resolution)
	both modified:   fe/fe-core/src/test/java/com/starrocks/connector/iceberg/procedure/AddFilesProcedureTest.java

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

#71380 [BugFix] Fix ORC file metric construction in AddFilesProcedure (backport #71220) has been created for branch branch-4.0 but encountered conflicts

Cherry-pick of ca4bd59 has failed:

On branch mergify/bp/branch-4.0/pr-71220
Your branch is up to date with 'origin/branch-4.0'.

You are currently cherry-picking commit ca4bd593ee.
  (fix conflicts and run "git cherry-pick --continue")
  (use "git cherry-pick --skip" to skip this patch)
  (use "git cherry-pick --abort" to cancel the cherry-pick operation)

Unmerged paths:
  (use "git add/rm <file>..." as appropriate to mark resolution)
	deleted by us:   fe/fe-core/src/main/java/com/starrocks/connector/iceberg/procedure/AddFilesProcedure.java
	deleted by us:   fe/fe-core/src/test/java/com/starrocks/connector/iceberg/procedure/AddFilesProcedureTest.java

no changes added to commit (use "git add" and/or "git commit -a")

To fix up this pull request, you can check it out locally. See documentation: https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/reviewing-changes-in-pull-requests/checking-out-pull-requests-locally

Signed-off-by: chrisyguo <511955993@qq.com> (cherry picked from commit ca4bd59) # Conflicts: # fe/fe-core/src/test/java/com/starrocks/connector/iceberg/procedure/AddFilesProcedureTest.java

Signed-off-by: chrisyguo <511955993@qq.com> (cherry picked from commit ca4bd59) # Conflicts: # fe/fe-core/src/main/java/com/starrocks/connector/iceberg/procedure/AddFilesProcedure.java # fe/fe-core/src/test/java/com/starrocks/connector/iceberg/procedure/AddFilesProcedureTest.java

Guosmilesmile · 2026-04-07T12:13:29Z

@kevincai @stephen-shelby Thanks for the review！

Do I need to backport it manually?

kevincai · 2026-04-07T12:27:20Z

@kevincai @stephen-shelby Thanks for the review！

Do I need to backport it manually?

I will take care of them.

kevincai · 2026-04-07T12:45:53Z

ignore backport branch-4.0, the code is only added since 4.1

…ort #71220) (#71379) Signed-off-by: Kevin Cai <kevin.cai@celerdata.com> Co-authored-by: GuoYu <511955993@qq.com> Co-authored-by: Kevin Cai <kevin.cai@celerdata.com>

Guosmilesmile requested a review from a team as a code owner April 2, 2026 09:44

github-actions Bot added the 4.1 label Apr 2, 2026

mergify Bot assigned Guosmilesmile Apr 2, 2026

Guosmilesmile closed this Apr 2, 2026

Guosmilesmile reopened this Apr 2, 2026

Guosmilesmile closed this Apr 2, 2026

Guosmilesmile reopened this Apr 2, 2026

Guosmilesmile closed this Apr 2, 2026

Guosmilesmile reopened this Apr 2, 2026

Guosmilesmile closed this Apr 2, 2026

Guosmilesmile reopened this Apr 2, 2026

github-actions Bot added behavior_changed and removed behavior_changed 4.1 labels Apr 2, 2026

[BugFix] Fix ORC file metric construction in AddFilesProcedure

1994e9a

Signed-off-by: chrisyguo <511955993@qq.com>

Guosmilesmile force-pushed the fix_orc_metric branch from 80b7252 to 1994e9a Compare April 2, 2026 10:52

stephen-shelby approved these changes Apr 3, 2026

View reviewed changes

This was referenced Apr 4, 2026

AddFilesProcedure incorrectly retrieves metrics for ORC files #71222

Closed

[BugFix] Fix off-by-one in ORC column statistics extraction #71300

Closed

kevincai requested review from Copilot and kevincai April 7, 2026 06:53

Copilot started reviewing on behalf of kevincai April 7, 2026 06:54 View session

Copilot AI reviewed Apr 7, 2026

View reviewed changes

Comment thread fe/fe-core/src/main/java/com/starrocks/connector/iceberg/procedure/AddFilesProcedure.java

Comment thread fe/fe-core/src/main/java/com/starrocks/connector/iceberg/procedure/AddFilesProcedure.java

kevincai approved these changes Apr 7, 2026

View reviewed changes

Comment thread fe/fe-core/src/main/java/com/starrocks/connector/iceberg/procedure/AddFilesProcedure.java

kevincai merged commit ca4bd59 into StarRocks:main Apr 7, 2026
90 checks passed

github-actions Bot added 4.0 4.1 labels Apr 7, 2026

mergify Bot mentioned this pull request Apr 7, 2026

[BugFix] Fix ORC file metric construction in AddFilesProcedure (backport #71220) #71379

Merged

24 tasks

mergify Bot mentioned this pull request Apr 7, 2026

[BugFix] Fix ORC file metric construction in AddFilesProcedure (backport #71220) #71380

Closed

24 tasks

github-actions Bot removed the 4.0 label Apr 7, 2026

Guosmilesmile deleted the fix_orc_metric branch April 7, 2026 13:16

github-actions Bot added the 4.1-merged label Apr 7, 2026

Conversation

Guosmilesmile commented Apr 2, 2026 • edited by kevincai Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why I'm doing:

What I'm doing:

What type of PR is this:

Checklist:

Bugfix cherry-pick branch check:

Uh oh!

Guosmilesmile commented Apr 2, 2026

Uh oh!

CelerData-Reviewer commented Apr 2, 2026

Uh oh!

chatgpt-codex-connector Bot commented Apr 2, 2026

Uh oh!

github-actions Bot commented Apr 2, 2026

[Java-Extensions Incremental Coverage Report]

Uh oh!

github-actions Bot commented Apr 2, 2026

[FE Incremental Coverage Report]

file detail

Uh oh!

github-actions Bot commented Apr 2, 2026

[BE Incremental Coverage Report]

Uh oh!

sonarqubecloud Bot commented Apr 2, 2026

Quality Gate failed

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Guosmilesmile commented Apr 7, 2026

Uh oh!

kevincai commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

kevincai commented Apr 7, 2026

Uh oh!

mergify Bot commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Backports have been created

Uh oh!

Guosmilesmile commented Apr 7, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kevincai commented Apr 7, 2026

Uh oh!

kevincai commented Apr 7, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Guosmilesmile commented Apr 2, 2026 •

edited by kevincai

Loading

kevincai commented Apr 7, 2026 •

edited

Loading

mergify Bot commented Apr 7, 2026 •

edited

Loading

Guosmilesmile commented Apr 7, 2026 •

edited

Loading