Skip to content

Commit 068a381

Browse files
Update the limitation docs (#3801) (#3837)
(cherry picked from commit 39eefc8) Signed-off-by: Lantao Jin <ltjin@amazon.com> Signed-off-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
1 parent 98c0c21 commit 068a381

3 files changed

Lines changed: 60 additions & 17 deletions

File tree

docs/dev/intro-v3-engine.md

Lines changed: 4 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,4 @@
1-
# PPL Engine V3 (for 3.0.0-beta)
1+
# PPL Engine V3 (for 3.0.0)
22

33
---
44
## 1. Motivations
@@ -24,7 +24,7 @@ Find more details in [V3 Architecture](./intro-v3-architecture.md).
2424
---
2525
## 2. What's New
2626

27-
In the initial release of the V3 engine (3.0.0-beta), the main new features focus on enhancing the PPL language while maintaining maximum compatibility with V2 behavior.
27+
In the initial release of the V3 engine (3.0.0), the main new features focus on enhancing the PPL language while maintaining maximum compatibility with V2 behavior.
2828

2929
* **[Join](../user/ppl/cmd/join.rst) Command**
3030
* **[Lookup](../user/ppl/cmd/lookup.rst) Command**
@@ -35,7 +35,7 @@ In the initial release of the V3 engine (3.0.0-beta), the main new features focu
3535

3636
### 3.1 Breaking Changes
3737

38-
Because of implementation changed internally, following behaviors are changed from 3.0.0-beta. (Behaviors in V3 is correct)
38+
Because of implementation changed internally, following behaviors are changed from 3.0.0. (Behaviors in V3 is correct)
3939

4040
| Item | V2 | V3 |
4141
|:------------------------------------------------:|:---------:|:--------------------:|
@@ -51,7 +51,7 @@ Because of implementation changed internally, following behaviors are changed fr
5151

5252
### 3.2 Fallback Mechanism
5353

54-
As v3 engine is experimental in 3.0.0-beta, not all PPL commands could work under this new engine. Those unsupported queries will be forwarded to V2 engine by fallback mechanism. To avoid impact on your side, normally you won't see any difference in a query response. If you want to check if and why your query falls back to be handled by V2 engine, please check OpenSearch log for "Fallback to V2 query engine since ...".
54+
As v3 engine is experimental in 3.0.0, not all PPL commands could work under this new engine. Those unsupported queries will be forwarded to V2 engine by fallback mechanism. To avoid impact on your side, normally you won't see any difference in a query response. If you want to check if and why your query falls back to be handled by V2 engine, please check OpenSearch log for "Fallback to V2 query engine since ...".
5555

5656
### 3.3 Limitations
5757

@@ -66,24 +66,12 @@ For the following functionalities in V3 engine, the query will be forwarded to t
6666

6767
#### Unsupported functionalities
6868
- All SQL queries
69-
- `trendline`
70-
- `show datasource`
71-
- `explain`
72-
- `describe`
73-
- `top` and `rare`
74-
- `fillnull`
75-
- `patterns`
7669
- `dedup` with `consecutive=true`
7770
- Search relevant commands
7871
- AD
7972
- ML
8073
- Kmeans
8174
- Commands with `fetch_size` parameter
82-
- query with metadata fields, `_id`, `_doc`, etc.
83-
- Json relevant functions
84-
- cast to json
85-
- json
86-
- json_valid
8775
- Search relevant functions
8876
- match
8977
- match_phrase
@@ -105,6 +93,5 @@ If you're interested in the new query engine, please find more details in [V3 Ar
10593
The following items are on our roadmap with high priority:
10694
- Resolve the [V3 limitation](#33-limitations).
10795
- Advancing pushdown optimization and benchmarking
108-
- Backport to 2.19.x
10996
- Unified the PPL syntax between [PPL-on-OpenSearch](https://github.com/opensearch-project/sql/blob/main/ppl/src/main/antlr/OpenSearchPPLParser.g4) and [PPL-on-Spark](https://github.com/opensearch-project/opensearch-spark/blob/main/ppl-spark-integration/src/main/antlr4/OpenSearchPPLParser.g4)
11097
- Support more DSL aggregation

docs/user/limitations/limitations.rst

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -130,3 +130,31 @@ The response in JSON format is::
130130
},
131131
"status": 400
132132
}
133+
134+
Limitations on Calcite Engine
135+
=============================
136+
137+
Since 3.0.0, we introduce Apache Calcite as an experimental query engine. Please see `introduce v3 engine <../../../dev/intro-v3-engine.md>`_.
138+
For the following functionalities, the query will be forwarded to the V2 query engine.
139+
140+
* All SQL queries
141+
142+
* ``dedup`` with ``consecutive=true``
143+
144+
* Search relevant commands
145+
146+
* AD
147+
* ML
148+
* Kmeans
149+
150+
* Commands with ``fetch_size`` parameter
151+
152+
* Search relevant functions
153+
154+
* match
155+
* match_phrase
156+
* match_bool_prefix
157+
* match_phrase_prefix
158+
* simple_query_string
159+
* query_string
160+
* multi_match

docs/user/ppl/limitations/limitations.rst

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -84,3 +84,31 @@ plugins.query.field_type_tolerance setting is enabled, the SQL/PPL plugin will h
8484
scalar data types, allowing basic queries (e.g., source = tbl | where condition). However, using multi-value
8585
fields in expressions or functions will result in exceptions. If this setting is disabled or absent, only the
8686
first element of an array is returned, preserving the default behavior.
87+
88+
Unsupported Functionalities in Calcite Engine
89+
=============================================
90+
91+
Since 3.0.0, we introduce Apache Calcite as an experimental query engine. Please see `introduce v3 engine <../../../dev/intro-v3-engine.md>`_.
92+
For the following functionalities, the query will be forwarded to the V2 query engine.
93+
94+
* All SQL queries
95+
96+
* ``dedup`` with ``consecutive=true``
97+
98+
* Search relevant commands
99+
100+
* AD
101+
* ML
102+
* Kmeans
103+
104+
* Commands with ``fetch_size`` parameter
105+
106+
* Search relevant functions
107+
108+
* match
109+
* match_phrase
110+
* match_bool_prefix
111+
* match_phrase_prefix
112+
* simple_query_string
113+
* query_string
114+
* multi_match

0 commit comments

Comments
 (0)