Skip to content

HIVE-29670: Avoid overwriting read-only DB_NOTIFICATION_EVENT_ID_KEY_NAME in DbNotificationListener when processing batched partition events#6548

Open
Aggarwal-Raghav wants to merge 1 commit into
apache:masterfrom
Aggarwal-Raghav:HIVE-29670

Conversation

@Aggarwal-Raghav

@Aggarwal-Raghav Aggarwal-Raghav commented Jun 17, 2026

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

Check JIRA HIVE-29670 for stacktrace.

Why are the changes needed?

Alter partitions fails/warns to update stats for more than 1 batch (metastore.jdbc.max.batch.size) of partitions. local-setup-hs2.log

Does this PR introduce any user-facing change?

No

How was this patch tested?

Using q file and local hive setup testing

mvn clean test -Dtest=TestMiniLlapLocalCliDriver -Dqfile=db_notification_batch_insert.q -Drat.skip -Pitests -pl itests/qtest

…NAME in DbNotificationListener when processing batched partition events
@Aggarwal-Raghav

Aggarwal-Raghav commented Jun 17, 2026

Copy link
Copy Markdown
Contributor Author

CC @dengzhhu653 , can you please help with the feedback here! Related to HIVE-27746

On debugging, I found the same event object is being overridden in while loop in DbNotificationListener#onAlterPartitions() .

  1. For 1st batch of partitions process() is triggered and set's the DB_NOTIFICATION_EVENT_ID_KEY_NAME

  2. But for next batch same event object is passed to process() and putParameterIfAbsent() throws IllegalStateException ❌

Before:
BEFORE_FIX

After:
After_FIX

@sonarqubecloud

Copy link
Copy Markdown

@dengzhhu653 dengzhhu653 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1 looks good to me. Thanks for the PR!
The DB_NOTIFICATION_EVENT_ID_KEY_NAME looks no sense to me, even I went through the HIVE-16164.

@Aggarwal-Raghav Aggarwal-Raghav marked this pull request as ready for review June 18, 2026 04:31
@Aggarwal-Raghav

Aggarwal-Raghav commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

Thanks for the review @dengzhhu653 .

More context, this was reported by the user, where they were performing INSERT INTO SELECT * and dynamically created more than 1000 partitions (more than default value of metastore.jdbc.max.batch.size) and the WARN log was observed on the beeline as well during stats gathering (hive.stats.autogather)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants