Skip to content

Commit 90e38c4

Browse files
leto-bbqleto-bbq
authored
add set operations in table from 2091 (#1094)
1 parent faf6a2c commit 90e38c4

12 files changed

Lines changed: 2472 additions & 0 deletions

File tree

‎src/.vuepress/sidebar/V2.0.x/en-Table.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -233,6 +233,7 @@ export const enSidebar = {
233233
{ text: 'ORDER BY Clause', link: 'OrderBy-Clause' },
234234
{ text: 'LIMIT&OFFSET Clause', link: 'Limit-Offset-Clause' },
235235
{ text: 'Nested Queries', link: 'Nested-Queries' },
236+
{ text: 'Set Operations', link: 'Set-Operations_apache' },
236237
],
237238
},
238239
{ text: 'Maintenance Statements', link: 'SQL-Maintenance-Statements_apache' },

‎src/.vuepress/sidebar/V2.0.x/zh-Table.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -230,6 +230,7 @@ export const zhSidebar = {
230230
{ text: 'ORDER BY子句', link: 'OrderBy-Clause' },
231231
{ text: 'LIMIT&OFFSET子句', link: 'Limit-Offset-Clause' },
232232
{ text: '嵌套查询', link: 'Nested-Queries' },
233+
{ text: '集合操作', link: 'Set-Operations_apache' },
233234
],
234235
},
235236
{ text: '运维语句', link: 'SQL-Maintenance-Statements_apache' },

‎src/.vuepress/sidebar_timecho/V2.0.x/en-Table.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -265,6 +265,7 @@ export const enSidebar = {
265265
{ text: 'LIMIT&OFFSET Clause', link: 'Limit-Offset-Clause' },
266266
{ text: 'Nested Queries', link: 'Nested-Queries' },
267267
{ text: 'Pattern Query', link: 'Row-Pattern-Recognition_timecho' },
268+
{ text: 'Set Operations', link: 'Set-Operations_timecho' },
268269
],
269270
},
270271
{ text: 'Maintenance Statements', link: 'SQL-Maintenance-Statements_timecho' },

‎src/.vuepress/sidebar_timecho/V2.0.x/zh-Table.ts‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -256,6 +256,7 @@ export const zhSidebar = {
256256
{ text: 'LIMIT&OFFSET子句', link: 'Limit-Offset-Clause' },
257257
{ text: '嵌套查询', link: 'Nested-Queries' },
258258
{ text: '模式查询', link: 'Row-Pattern-Recognition_timecho' },
259+
{ text: '集合操作', link: 'Set-Operations_timecho' },
259260
],
260261
},
261262
{ text: '运维语句', link: 'SQL-Maintenance-Statements_timecho' },

‎src/UserGuide/Master/Table/SQL-Manual/Set-Operations_apache.md‎

Lines changed: 295 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,295 @@
1+
<!--
2+
3+
Licensed to the Apache Software Foundation (ASF) under one
4+
or more contributor license agreements. See the NOTICE file
5+
distributed with this work for additional information
6+
regarding copyright ownership. The ASF licenses this file
7+
to you under the Apache License, Version 2.0 (the
8+
"License"); you may not use this file except in compliance
9+
with the License. You may obtain a copy of the License at
10+
11+
http://www.apache.org/licenses/LICENSE-2.0
12+
13+
Unless required by applicable law or agreed to in writing,
14+
software distributed under the License is distributed on an
15+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
KIND, either express or implied. See the License for the
17+
specific language governing permissions and limitations
18+
under the License.
19+
20+
-->
21+
# Set Operations
22+
23+
IoTDB natively supports standard SQL set operations, including three core operators: **UNION**, **INTERSECT**, and **EXCEPT**. These operations enable seamless merging, comparison, and filtering of query results from multiple time-series data sources, greatly improving the flexibility and efficiency of time-series data analysis.
24+
25+
> Note: This feature is available since version 2.0.9-beta.
26+
27+
## 1. UNION
28+
### 1.1 Overview
29+
The UNION operator combines all rows from two result sets (order not guaranteed), supporting both duplicate elimination (default) and duplicate retention modes.
30+
31+
### 1.2 Syntax
32+
```sql
33+
query UNION (ALL | DISTINCT) query
34+
```
35+
36+
**Description**
37+
1. **Duplicate Handling**
38+
- Default (`UNION` or `UNION DISTINCT`): Automatically removes duplicate rows.
39+
- `UNION ALL`: Preserves all rows (including duplicates) with higher performance.
40+
41+
2. **Input Requirements**
42+
- The two queries must return the same number of columns.
43+
- Corresponding columns must have compatible data types:
44+
- Numeric compatibility: `INT32`, `INT64`, `FLOAT`, and `DOUBLE` are fully compatible with each other.
45+
- String compatibility: `TEXT` and `STRING` are fully compatible.
46+
- Special rule: `INT64` is compatible with `TIMESTAMP`.
47+
48+
3. **Result Set Rules**
49+
- Column names and order are inherited from the first query.
50+
51+
### 1.3 Examples
52+
Using the [sample data](../Reference/Sample-Data.md):
53+
54+
1. Get distinct non-null device and temperature records from `table1` and `table2`
55+
```sql
56+
SELECT device_id, temperature FROM table1 WHERE temperature IS NOT NULL
57+
UNION
58+
SELECT device_id, temperature FROM table2 WHERE temperature IS NOT NULL;
59+
60+
-- Equivalent to:
61+
SELECT device_id, temperature FROM table1 WHERE temperature IS NOT NULL
62+
UNION DISTINCT
63+
SELECT device_id, temperature FROM table2 WHERE temperature IS NOT NULL;
64+
```
65+
66+
Result:
67+
```
68+
+---------+-----------+
69+
|device_id|temperature|
70+
+---------+-----------+
71+
| 101| 90.0|
72+
| 101| 85.0|
73+
| 100| 90.0|
74+
| 100| 85.0|
75+
| 100| 88.0|
76+
+---------+-----------+
77+
Total line number = 5
78+
It costs 0.074s
79+
```
80+
81+
2. Get all non-null device and temperature records from `table1` and `table2` (including duplicates)
82+
```sql
83+
SELECT device_id, temperature FROM table1 WHERE temperature IS NOT NULL
84+
UNION ALL
85+
SELECT device_id, temperature FROM table2 WHERE temperature IS NOT NULL;
86+
```
87+
88+
Result:
89+
```
90+
+---------+-----------+
91+
|device_id|temperature|
92+
+---------+-----------+
93+
| 101| 90.0|
94+
| 101| 90.0|
95+
| 101| 85.0|
96+
| 101| 85.0|
97+
| 101| 85.0|
98+
| 101| 85.0|
99+
| 100| 90.0|
100+
| 100| 85.0|
101+
| 100| 85.0|
102+
| 100| 88.0|
103+
| 100| 90.0|
104+
| 100| 90.0|
105+
| 101| 90.0|
106+
| 101| 85.0|
107+
| 101| 85.0|
108+
| 100| 85.0|
109+
| 100| 90.0|
110+
+---------+-----------+
111+
Total line number = 17
112+
It costs 0.108s
113+
```
114+
115+
> **Notes**
116+
> - Set operations **do not guarantee result order**; actual output may differ from examples.
117+
118+
119+
## 2. INTERSECT
120+
### 2.1 Overview
121+
The INTERSECT operator returns rows that exist in both result sets (order not guaranteed), supporting both duplicate elimination (default) and duplicate retention modes.
122+
123+
### 2.2 Syntax
124+
```sql
125+
query1 INTERSECT [ALL | DISTINCT] query2
126+
```
127+
128+
**Description**
129+
1. **Duplicate Handling**
130+
- Default (`INTERSECT` or `INTERSECT DISTINCT`): Automatically removes duplicate rows.
131+
- `INTERSECT ALL`: Preserves duplicate rows, with slightly lower performance.
132+
133+
2. **Precedence Rules**
134+
- `INTERSECT` has higher precedence than `UNION` and `EXCEPT`
135+
(e.g., `A UNION B INTERSECT C` is equivalent to `A UNION (B INTERSECT C)`).
136+
- Evaluation is left-to-right
137+
(e.g., `A INTERSECT B INTERSECT C` is equivalent to `(A INTERSECT B) INTERSECT C`).
138+
139+
3. **Input Requirements**
140+
- The two queries must return the same number of columns.
141+
- Corresponding columns must have compatible data types (same rules as UNION).
142+
- NULL values are treated as equal (`NULL IS NOT DISTINCT FROM NULL`).
143+
- If the `time` column is not included in `SELECT`, it does not participate in comparison and will not appear in the result.
144+
145+
4. **Result Set Rules**
146+
- Column names and order are inherited from the first query.
147+
148+
### 2.3 Examples
149+
Using the [sample data](../Reference/Sample-Data.md):
150+
151+
1. Get distinct common device and temperature records from `table1` and `table2`
152+
```sql
153+
SELECT device_id, temperature FROM table1
154+
INTERSECT
155+
SELECT device_id, temperature FROM table2;
156+
157+
-- Equivalent to:
158+
SELECT device_id, temperature FROM table1
159+
INTERSECT DISTINCT
160+
SELECT device_id, temperature FROM table2;
161+
```
162+
163+
Result:
164+
```
165+
+---------+-----------+
166+
|device_id|temperature|
167+
+---------+-----------+
168+
| 101| 90.0|
169+
| 101| 85.0|
170+
| 100| null|
171+
| 100| 90.0|
172+
| 100| 85.0|
173+
+---------+-----------+
174+
Total line number = 5
175+
It costs 0.087s
176+
```
177+
178+
2. Get all common device and temperature records from `table1` and `table2` (including duplicates)
179+
```sql
180+
SELECT device_id, temperature FROM table1
181+
INTERSECT ALL
182+
SELECT device_id, temperature FROM table2;
183+
```
184+
185+
Result:
186+
```
187+
+---------+-----------+
188+
|device_id|temperature|
189+
+---------+-----------+
190+
| 100| 85.0|
191+
| 100| 90.0|
192+
| 100| null|
193+
| 101| 85.0|
194+
| 101| 85.0|
195+
| 101| 90.0|
196+
+---------+-----------+
197+
Total line number = 6
198+
It costs 0.139s
199+
```
200+
201+
> **Notes**
202+
> - Set operations **do not guarantee result order**.
203+
> - When mixed with `UNION`/`EXCEPT`, use parentheses to explicitly specify precedence
204+
> (e.g., `A INTERSECT (B UNION C)`).
205+
206+
207+
## 3. EXCEPT
208+
### 3.1 Overview
209+
The EXCEPT operator returns rows that exist in the first result set but **not** in the second (order not guaranteed), supporting both duplicate elimination (default) and duplicate retention modes.
210+
211+
### 3.2 Syntax
212+
```sql
213+
query1 EXCEPT [ALL | DISTINCT] query2
214+
```
215+
216+
**Description**
217+
1. **Duplicate Handling**
218+
- Default (`EXCEPT` or `EXCEPT DISTINCT`): Automatically removes duplicate rows.
219+
- `EXCEPT ALL`: Preserves duplicate rows, with slightly lower performance.
220+
221+
2. **Precedence Rules**
222+
- `EXCEPT` has the same precedence as `UNION`, and lower precedence than `INTERSECT`
223+
(e.g., `A INTERSECT B EXCEPT C` is equivalent to `(A INTERSECT B) EXCEPT C`).
224+
- Evaluation is left-to-right
225+
(e.g., `A EXCEPT B EXCEPT C` is equivalent to `(A EXCEPT B) EXCEPT C`).
226+
227+
3. **Input Requirements**
228+
- The two queries must return the same number of columns.
229+
- Corresponding columns must have compatible data types (same rules as UNION).
230+
- NULL values are treated as equal (`NULL IS NOT DISTINCT FROM NULL`).
231+
- If the `time` column is not included in `SELECT`, it does not participate in comparison and will not appear in the result.
232+
233+
4. **Result Set Rules**
234+
- Column names and order are inherited from the first query.
235+
236+
### 3.3 Examples
237+
Using the [sample data](../Reference/Sample-Data.md):
238+
239+
1. Get distinct records from `table1` that do not exist in `table2`
240+
```sql
241+
SELECT device_id, temperature FROM table1
242+
EXCEPT
243+
SELECT device_id, temperature FROM table2;
244+
245+
-- Equivalent to:
246+
SELECT device_id, temperature FROM table1
247+
EXCEPT DISTINCT
248+
SELECT device_id, temperature FROM table2;
249+
```
250+
251+
Result:
252+
```
253+
+---------+-----------+
254+
|device_id|temperature|
255+
+---------+-----------+
256+
| 101| null|
257+
| 100| 88.0|
258+
+---------+-----------+
259+
Total line number = 2
260+
It costs 0.173s
261+
```
262+
263+
2. Get all records from `table1` that do not exist in `table2` (including duplicates)
264+
```sql
265+
SELECT device_id, temperature FROM table1
266+
EXCEPT ALL
267+
SELECT device_id, temperature FROM table2;
268+
```
269+
270+
Result:
271+
```
272+
+---------+-----------+
273+
|device_id|temperature|
274+
+---------+-----------+
275+
| 100| 85.0|
276+
| 100| 88.0|
277+
| 100| 90.0|
278+
| 100| 90.0|
279+
| 100| null|
280+
| 101| 85.0|
281+
| 101| 85.0|
282+
| 101| 90.0|
283+
| 101| null|
284+
| 101| null|
285+
| 101| null|
286+
| 101| null|
287+
+---------+-----------+
288+
Total line number = 12
289+
It costs 0.155s
290+
```
291+
292+
> **Notes**
293+
> - Set operations **do not guarantee result order**.
294+
> - When mixed with `UNION`/`INTERSECT`, use parentheses to explicitly specify precedence
295+
> (e.g., `A EXCEPT (B INTERSECT C)`).

0 commit comments

Comments
 (0)