Skip to content

[Enhancement] Skip predicate column vacuuming when TTL is negative or usage is empty (backport #71290)#71777

Merged
wanpengfei-git merged 2 commits into
branch-4.1from
mergify/bp/branch-4.1/pr-71290
May 9, 2026
Merged

[Enhancement] Skip predicate column vacuuming when TTL is negative or usage is empty (backport #71290)#71777
wanpengfei-git merged 2 commits into
branch-4.1from
mergify/bp/branch-4.1/pr-71290

Conversation

@mergify
Copy link
Copy Markdown
Contributor

@mergify mergify Bot commented Apr 16, 2026

Why I'm doing:

I'm using StarRocks with enable_predicate_columns_collection=false and have truncated the _statistics_.predicate_columns table. However, the vacuum process with TTL is still running and generating useless audit logs. I want to suppress predicate column vacuuming by disabling predicate column TTL.

What I'm doing:

When statistic_predicate_columns_ttl_hours is set to a negative value (e.g. -1), skip vacuuming entirely.

Additionally, skip vacuuming id2columnUsage when it is already empty before removal:

    public void vacuum() {
        long ttlHour = Config.statistic_predicate_columns_ttl_hours;
        if (ttlHour < 0) { // HERE
            return;
        }
        LocalDateTime ttlTime = TimeUtils.getSystemNow().minusHours(ttlHour);
        Predicate<ColumnUsage> outdated = x -> x.getLastUsed().isBefore(ttlTime);

        long before = id2columnUsage.size();
        if (before > 0 && id2columnUsage.values().removeIf(outdated)) { // HERE
            long after = id2columnUsage.size();
            LOG.info("removed {} objects from predicate columns because of ttl {}", before - after,
                    Config.statistic_predicate_columns_ttl_hours);
        }

        // If the process crashed before vacuum the storage, the storage may be different from in-memory state,
        // but it doesn't matter. Because we will remove them finally.
        getStorage().vacuum(ttlTime);
    }

Already, reflect the changes on documents also.

Fixes #71291

What type of PR is this:

  • BugFix
  • Feature
  • Enhancement
  • Refactor
  • UT
  • Doc
  • Tool

Does this PR entail a change in behavior?

  • Yes, this PR will result in a change in behavior.
  • No, this PR will not result in a change in behavior.

If yes, please specify the type of change:

  • Interface/UI changes: syntax, type conversion, expression evaluation, display information
  • Parameter changes: default values, similar parameters but with different default values
  • Policy changes: use new policy to replace old one, functionality automatically enabled
  • Feature removed
  • Miscellaneous: upgrade & downgrade compatibility, etc.

Checklist:

  • I have added test cases for my bug fix or my new feature
  • This pr needs user documentation (for new or modified features or behaviors)
    • I have added documentation for my new feature or new function
    • This pr needs auto generate documentation
  • This is a backport pr

Bugfix cherry-pick branch check:

  • I have checked the version labels which the pr will be auto-backported to the target branch
    • 4.1
    • 4.0
    • 3.5
    • 3.4

This is an automatic backport of pull request #71290 done by [Mergify](https://mergify.com).

… usage is empty (#71290)

Signed-off-by: seokyun.ha <seokyun.ha@toss.im>
Signed-off-by: 絵空事スピリット <richard.wang@celerdata.com>
Co-authored-by: 絵空事スピリット <richard.wang@celerdata.com>
(cherry picked from commit e3510e9)
@github-actions github-actions Bot added documentation Improvements or additions to documentation automerge labels Apr 16, 2026
@wanpengfei-git wanpengfei-git enabled auto-merge (squash) April 16, 2026 02:47
@github-actions
Copy link
Copy Markdown
Contributor

github-actions Bot commented Apr 16, 2026

🌎 Translation Required?

All translation files are up to date.
Great job! No translation actions are required for this PR.

🕒 Last updated: Sat, 09 May 2026 08:12:37 GMT

@seokyun-ha-toss
Copy link
Copy Markdown
Contributor

Good morning @stephen-shelby, could you let me know when this backport PR can be merged? Thanks!

@kevincai
Copy link
Copy Markdown
Contributor

kevincai commented May 9, 2026

@Mergifyio update

@mergify
Copy link
Copy Markdown
Contributor Author

mergify Bot commented May 9, 2026

update

✅ Branch has been successfully updated

@kevincai kevincai self-assigned this May 9, 2026
@wanpengfei-git wanpengfei-git merged commit f19bfea into branch-4.1 May 9, 2026
30 checks passed
@wanpengfei-git wanpengfei-git deleted the mergify/bp/branch-4.1/pr-71290 branch May 9, 2026 08:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

automerge documentation Improvements or additions to documentation version:4.1.1

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants