Skip to content

Add APPROX_PERCENTILE aggregation function#16041

Merged
JackieTien97 merged 3 commits into
apache:masterfrom
FearfulTomcat27:feat/percentile
Aug 26, 2025
Merged

Add APPROX_PERCENTILE aggregation function#16041
JackieTien97 merged 3 commits into
apache:masterfrom
FearfulTomcat27:feat/percentile

Conversation

@FearfulTomcat27

Copy link
Copy Markdown
Contributor

Add comprehensive implementation of the APPROX_PERCENTILE function for efficient percentile calculation using T-Digest algorithm:

Core Components

  • TDigest algorithm implementation with memory optimization and reset functionality
  • Abstract base classes for both regular and grouped accumulator patterns
  • Support for both basic and weighted percentile calculations

Features Added

  • Efficient quantile estimation with configurable compression
  • Memory usage tracking with RamUsageEstimator integration
  • Type-safe conversions for int, long, float, and double data types
  • Support for grouped aggregation operations
  • Reset functionality for memory reuse

Performance Optimizations

  • Code structure improvements and method extraction for better maintainability
  • Reduced code duplication through inheritance hierarchy
  • Memory-efficient array management with TDigestBigArray

FearfulTomcat27 and others added 2 commits July 27, 2025 17:35
Add comprehensive implementation of the APPROX_PERCENTILE function for efficient
percentile calculation using T-Digest algorithm:

## Core Components
- TDigest algorithm implementation with memory optimization and reset functionality
- Abstract base classes for both regular and grouped accumulator patterns
- Support for both basic and weighted percentile calculations

## Features Added
- Efficient quantile estimation with configurable compression
- Memory usage tracking with RamUsageEstimator integration
- Type-safe conversions for int, long, float, and double data types
- Support for grouped aggregation operations
- Reset functionality for memory reuse

## Performance Optimizations
- Code structure improvements and method extraction for better maintainability
- Reduced code duplication through inheritance hierarchy
- Memory-efficient array management with TDigestBigArray

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@JackieTien97 JackieTien97 requested a review from Copilot August 7, 2025 08:42

This comment was marked as outdated.

@JackieTien97 JackieTien97 requested a review from Copilot August 8, 2025 07:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements the APPROX_PERCENTILE aggregation function in IoTDB, providing efficient percentile calculation using the T-Digest algorithm. The implementation includes comprehensive support for both regular and weighted percentile calculations with memory optimizations.

  • Adds T-Digest algorithm implementation with configurable compression and reset functionality
  • Introduces abstract base classes for both regular and grouped accumulator patterns
  • Provides support for all numeric data types (int, long, float, double) and grouped aggregations

Reviewed Changes

Copilot reviewed 16 out of 16 changed files in this pull request and generated 9 comments.

Show a summary per file
File Description
common.thrift Adds APPROX_PERCENTILE to TAggregationType enum
TableBuiltinAggregationFunction.java Registers APPROX_PERCENTILE as builtin function
SqlConstant.java Defines APPROX_PERCENTILE constant
AstBuilder.java Adds validation for APPROX_PERCENTILE function arguments
TableMetadataImpl.java Implements type validation and return type logic
TDigest.java Core T-Digest algorithm implementation with serialization
TDigestBigArray.java Memory-efficient array management for grouped operations
AbstractApproxPercentileAccumulator.java Base class for single accumulator pattern
ApproxPercentileAccumulator.java Regular percentile accumulator implementation
ApproxPercentileWithWeightAccumulator.java Weighted percentile accumulator implementation
AbstractGroupedApproxPercentileAccumulator.java Base class for grouped accumulator pattern
GroupedApproxPercentileAccumulator.java Grouped regular percentile accumulator
GroupedApproxPercentileWithWeightAccumulator.java Grouped weighted percentile accumulator
TDigestStateFactory.java Factory for creating T-Digest state objects
AccumulatorFactory.java Factory integration for accumulator creation
IoTDBTableAggregationIT.java Integration tests for APPROX_PERCENTILE functionality

@JackieTien97 JackieTien97 merged commit 922330e into apache:master Aug 26, 2025
32 of 33 checks passed
@FearfulTomcat27 FearfulTomcat27 deleted the feat/percentile branch October 9, 2025 14:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants