Skip to content

Commit e2678a1

Browse files
selsongSelina Song
andauthored
Support timechart command with Calcite (opensearch-project#3993)
* WIP: Support timechart grammar / AST Signed-off-by: Selina Song <selsong@amazon.com> * WIP: Support span=unit in timechart Signed-off-by: Selina Song <selsong@amazon.com> * Return correct column format after span=unit Signed-off-by: Selina Song <selsong@amazon.com> * sort by @timestamp, group by aggregate function Signed-off-by: Selina Song <selsong@amazon.com> * WIP: pivot table by field Signed-off-by: Selina Song <selsong@amazon.com> * Correct pivot format for by field Signed-off-by: Selina Song <selsong@amazon.com> * Fix aggregation composite bucket limit Signed-off-by: Selina Song <selsong@amazon.com> * Add timechart.rst doc and PPLQueryAnonymizer test Signed-off-by: Selina Song <selsong@amazon.com> * Add ExplainIT for timechart Signed-off-by: Selina Song <selsong@amazon.com> * Restore reverse ExplainIT Signed-off-by: Selina Song <selsong@amazon.com> * Update explainIT timechart Signed-off-by: Selina Song <selsong@amazon.com> * spotlessApply formatting Signed-off-by: Selina Song <selsong@amazon.com> * format IT Signed-off-by: Selina Song <selsong@amazon.com> * Add limit parameter Signed-off-by: Selina Song <selsong@amazon.com> * Add limit=0 means no limit, show all values, no other Signed-off-by: Selina Song <selsong@amazon.com> * Add useother parameter Signed-off-by: Selina Song <selsong@amazon.com> * clean up format, fix column order Signed-off-by: Selina Song <selsong@amazon.com> * add test for formatter Signed-off-by: Selina Song <selsong@amazon.com> * Increase test coverage for formatter Signed-off-by: Selina Song <selsong@amazon.com> * Rename bin option expression, modify constructor Signed-off-by: Selina Song <selsong@amazon.com> * Make test data smaller, update IT, rst Signed-off-by: Selina Song <selsong@amazon.com> * add explain output to IT Signed-off-by: Selina Song <selsong@amazon.com> * Add limitation to rst, format, comment Signed-off-by: Selina Song <selsong@amazon.com> * Add test coverage formatter Signed-off-by: Selina Song <selsong@amazon.com> * Refactor formatter for clarity Signed-off-by: Selina Song <selsong@amazon.com> * fix NPE Signed-off-by: Selina Song <selsong@amazon.com> * add import Signed-off-by: Selina Song <selsong@amazon.com> * update mapping to match smaller dataset Signed-off-by: Selina Song <selsong@amazon.com> * update explainIT Signed-off-by: Selina Song <selsong@amazon.com> * Fix explainIT, improve code structure Signed-off-by: Selina Song <selsong@amazon.com> * spotless Signed-off-by: Selina Song <selsong@amazon.com> * Support diff position parameters, count default 0 not null, update doc rst Signed-off-by: Selina Song <selsong@amazon.com> * fix count aggr type int Signed-off-by: Selina Song <selsong@amazon.com> * rename @timestamp column Signed-off-by: Selina Song <selsong@amazon.com> * move parameter extraction to PPLService Signed-off-by: Selina Song <selsong@amazon.com> * clean up PPLService Signed-off-by: Selina Song <selsong@amazon.com> * fix count type test, doc rst format Signed-off-by: Selina Song <selsong@amazon.com> * add test coverage Signed-off-by: Selina Song <selsong@amazon.com> * WIP: SQL query with limit useother Signed-off-by: Selina Song <selsong@amazon.com> * revert QueryService Signed-off-by: Selina Song <selsong@amazon.com> * add limit useother to SQL query Signed-off-by: Selina Song <selsong@amazon.com> * SQL query working, WIP column rename Signed-off-by: Selina Song <selsong@amazon.com> * use loadIndex in IT Signed-off-by: Selina Song <selsong@amazon.com> * Rename fields to match, update doc Signed-off-by: Selina Song <selsong@amazon.com> * revert QueryResult Signed-off-by: Selina Song <selsong@amazon.com> * revert gradle build Signed-off-by: Selina Song <selsong@amazon.com> * fix format Signed-off-by: Selina Song <selsong@amazon.com> * Add count fill zero, update toSQL tests, doc Signed-off-by: Selina Song <selsong@amazon.com> * fix nits Signed-off-by: Selina Song <selsong@amazon.com> * Fix rename aggregation only no by field Signed-off-by: Selina Song <selsong@amazon.com> * Update SQL test Signed-off-by: Selina Song <selsong@amazon.com> * revert OS Exec Engine edits Signed-off-by: Selina Song <selsong@amazon.com> * correct OS Exec Engine revert to 5ec9603 Signed-off-by: Selina Song <selsong@amazon.com> * restore OS Exec Engine Signed-off-by: Selina Song <selsong@amazon.com> * update QueryAnonymizer Test to reflect default Signed-off-by: Selina Song <selsong@amazon.com> * WIP: Add null=1, Other fill zero wip Signed-off-by: Selina Song <selsong@amazon.com> * Add doctest and update ExplainIT Signed-off-by: Selina Song <selsong@amazon.com> * Replace detectFieldIndices function Signed-off-by: Selina Song <selsong@amazon.com> * update explainIT Signed-off-by: Selina Song <selsong@amazon.com> * Fix Other in zero fill case Signed-off-by: Selina Song <selsong@amazon.com> * Fix OTHER in zero fill case Signed-off-by: Selina Song <selsong@amazon.com> * spotless format Signed-off-by: Selina Song <selsong@amazon.com> * update doc with null example Signed-off-by: Selina Song <selsong@amazon.com> * null not included in limit calc Signed-off-by: Selina Song <selsong@amazon.com> * update SQL test Signed-off-by: Selina Song <selsong@amazon.com> * update SQL test format Signed-off-by: Selina Song <selsong@amazon.com> * Update ExplainIT with count Signed-off-by: Selina Song <selsong@amazon.com> * remove unused code Signed-off-by: Selina Song <selsong@amazon.com> * Fix SQL tests nullable Signed-off-by: Selina Song <selsong@amazon.com> * spotless format Signed-off-by: Selina Song <selsong@amazon.com> --------- Signed-off-by: Selina Song <selsong@amazon.com> Signed-off-by: Selina Song <selinasong6@gmail.com> Co-authored-by: Selina Song <selsong@amazon.com>
1 parent 8638460 commit e2678a1

25 files changed

Lines changed: 1898 additions & 2 deletions

core/src/main/java/org/opensearch/sql/analysis/Analyzer.java

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -88,6 +88,7 @@
8888
import org.opensearch.sql.ast.tree.Sort.SortOption;
8989
import org.opensearch.sql.ast.tree.SubqueryAlias;
9090
import org.opensearch.sql.ast.tree.TableFunction;
91+
import org.opensearch.sql.ast.tree.Timechart;
9192
import org.opensearch.sql.ast.tree.Trendline;
9293
import org.opensearch.sql.ast.tree.UnresolvedPlan;
9394
import org.opensearch.sql.ast.tree.Values;
@@ -739,6 +740,11 @@ public LogicalPlan visitReverse(Reverse node, AnalysisContext context) {
739740
throw getOnlyForCalciteException("Reverse");
740741
}
741742

743+
@Override
744+
public LogicalPlan visitTimechart(Timechart node, AnalysisContext context) {
745+
throw getOnlyForCalciteException("Timechart");
746+
}
747+
742748
@Override
743749
public LogicalPlan visitRegex(Regex node, AnalysisContext context) {
744750
throw getOnlyForCalciteException("Regex");

core/src/main/java/org/opensearch/sql/ast/AbstractNodeVisitor.java

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -76,6 +76,7 @@
7676
import org.opensearch.sql.ast.tree.Sort;
7777
import org.opensearch.sql.ast.tree.SubqueryAlias;
7878
import org.opensearch.sql.ast.tree.TableFunction;
79+
import org.opensearch.sql.ast.tree.Timechart;
7980
import org.opensearch.sql.ast.tree.Trendline;
8081
import org.opensearch.sql.ast.tree.Values;
8182
import org.opensearch.sql.ast.tree.Window;
@@ -260,6 +261,10 @@ public T visitReverse(Reverse node, C context) {
260261
return visitChildren(node, context);
261262
}
262263

264+
public T visitTimechart(Timechart node, C context) {
265+
return visitChildren(node, context);
266+
}
267+
263268
public T visitRegex(Regex node, C context) {
264269
return visitChildren(node, context);
265270
}
Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
/*
2+
* Copyright OpenSearch Contributors
3+
* SPDX-License-Identifier: Apache-2.0
4+
*/
5+
6+
package org.opensearch.sql.ast.tree;
7+
8+
import com.google.common.collect.ImmutableList;
9+
import java.util.List;
10+
import lombok.AllArgsConstructor;
11+
import lombok.EqualsAndHashCode;
12+
import lombok.Getter;
13+
import lombok.ToString;
14+
import org.opensearch.sql.ast.AbstractNodeVisitor;
15+
import org.opensearch.sql.ast.expression.UnresolvedExpression;
16+
17+
/** AST node represent Timechart operation. */
18+
@Getter
19+
@ToString
20+
@EqualsAndHashCode(callSuper = false)
21+
@AllArgsConstructor
22+
@lombok.Builder(toBuilder = true)
23+
public class Timechart extends UnresolvedPlan {
24+
private UnresolvedPlan child;
25+
private UnresolvedExpression binExpression;
26+
private UnresolvedExpression aggregateFunction;
27+
private UnresolvedExpression byField;
28+
private Integer limit;
29+
private Boolean useOther;
30+
31+
public Timechart(UnresolvedPlan child, UnresolvedExpression aggregateFunction) {
32+
this(child, null, aggregateFunction, null, null, true);
33+
}
34+
35+
public Timechart span(UnresolvedExpression binExpression) {
36+
return toBuilder().binExpression(binExpression).build();
37+
}
38+
39+
public Timechart by(UnresolvedExpression byField) {
40+
return toBuilder().byField(byField).build();
41+
}
42+
43+
public Timechart limit(Integer limit) {
44+
return toBuilder().limit(limit).build();
45+
}
46+
47+
public Timechart useOther(Boolean useOther) {
48+
return toBuilder().useOther(useOther).build();
49+
}
50+
51+
@Override
52+
public Timechart attach(UnresolvedPlan child) {
53+
return toBuilder().child(child).build();
54+
}
55+
56+
@Override
57+
public List<UnresolvedPlan> getChild() {
58+
return ImmutableList.of(child);
59+
}
60+
61+
@Override
62+
public <T, C> T accept(AbstractNodeVisitor<T, C> nodeVisitor, C context) {
63+
return nodeVisitor.visitTimechart(this, context);
64+
}
65+
}

0 commit comments

Comments
 (0)