Skip to content

Commit 52d5a8b

Browse files
committed
Fix integration issues with clickhouse-server 25.10
1 parent 318ce15 commit 52d5a8b

27 files changed

Lines changed: 1923 additions & 351 deletions

File tree

distribution/proxy-native/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -718,7 +718,7 @@
718718
<dependency>
719719
<groupId>com.clickhouse</groupId>
720720
<artifactId>clickhouse-jdbc</artifactId>
721-
<classifier>http</classifier>
721+
<classifier>all</classifier>
722722
<scope>runtime</scope>
723723
</dependency>
724724
</dependencies>

distribution/proxy/pom.xml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -753,7 +753,7 @@
753753
<dependency>
754754
<groupId>com.clickhouse</groupId>
755755
<artifactId>clickhouse-jdbc</artifactId>
756-
<classifier>http</classifier>
756+
<classifier>all</classifier>
757757
<scope>runtime</scope>
758758
</dependency>
759759
</dependencies>

distribution/proxy/src/main/release-docs/LICENSE

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -324,7 +324,7 @@ BSD licenses
324324
The following components are provided under a BSD license. See project link for details.
325325
The text of each license is also included at licenses/LICENSE-[project].txt.
326326

327-
antlr4-runtime 4.10.1: https://github.com/antlr/antlr4, BSD-3-Clause
327+
antlr4-runtime 4.13.2: https://github.com/antlr/antlr4, BSD-3-Clause
328328
asm 9.3: https://github.com/llbit/ow2-asm, BSD-3-Clause
329329
commons-compiler 3.1.11: https://github.com/janino-compiler/janino, BSD-3-Clause
330330
janino 3.1.11: https://github.com/janino-compiler/janino, BSD-3-Clause

docs/document/content/user-manual/shardingsphere-jdbc/graalvm-native-image/_index.cn.md

Lines changed: 5 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -280,42 +280,13 @@ Caused by: java.io.UnsupportedEncodingException: Codepage Cp1252 is not supporte
280280
}
281281
```
282282

283-
6. 当需要通过 ShardingSphere JDBC 使用 ClickHouse 方言时,
284-
用户需要手动引入相关的可选模块和 classifier 为 `http` 的 ClickHouse JDBC 驱动。
285-
原则上,ShardingSphere 的 GraalVM Native Image 集成不希望使用 classifier 为 `all` 的 `com.clickhouse:clickhouse-jdbc`,
286-
因为 Uber Jar 会导致采集重复的 GraalVM Reachability Metadata。
287-
可能的配置例子如下,
288-
289-
```xml
290-
<project>
291-
<dependencies>
292-
<dependency>
293-
<groupId>org.apache.shardingsphere</groupId>
294-
<artifactId>shardingsphere-jdbc</artifactId>
295-
<version>${shardingsphere.version}</version>
296-
</dependency>
297-
<dependency>
298-
<groupId>org.apache.shardingsphere</groupId>
299-
<artifactId>shardingsphere-jdbc-dialect-clickhouse</artifactId>
300-
<version>${shardingsphere.version}</version>
301-
</dependency>
302-
<dependency>
303-
<groupId>com.clickhouse</groupId>
304-
<artifactId>clickhouse-jdbc</artifactId>
305-
<version>0.6.3</version>
306-
<classifier>http</classifier>
307-
</dependency>
308-
</dependencies>
309-
</project>
310-
```
311-
312-
7. ShardingSphere 的单元测试仅使用 Maven 模块 `io.github.linghengqian:hive-server2-jdbc-driver-thin` 来在 GraalVM Native Image 下验证 HiveServer2 集成的可用性。
283+
6. ShardingSphere 的单元测试仅使用 Maven 模块 `io.github.linghengqian:hive-server2-jdbc-driver-thin` 来在 GraalVM Native Image 下验证 HiveServer2 集成的可用性。
313284
如果开发者直接使用 `org.apache.hive:hive-jdbc`,则应自行处理依赖冲突和提供额外的 GraalVM Reachability Metadata 。
314285

315-
8. 由于 https://github.com/oracle/graal/issues/7979 的影响,
286+
7. 由于 https://github.com/oracle/graal/issues/7979 的影响,
316287
对应 `com.oracle.database.jdbc:ojdbc8` Maven 模块的 Oracle JDBC Driver 无法在 GraalVM Native Image 下使用。
317288

318-
9. 包括但不限于来自第三方依赖的 `com.mysql.cj.LocalizedErrorMessages`,
289+
8. 包括但不限于来自第三方依赖的 `com.mysql.cj.LocalizedErrorMessages`,
319290
`com.microsoft.sqlserver.jdbc.SQLServerResource`,
320291
`org.postgresql.translation.messages`,
321292
`org.opengauss.translation.messages` 等 `Resource Bundles` 在默认情况下会根据系统的默认语言环境加载 L10N 资源,
@@ -340,11 +311,11 @@ without it being registered as reachable. Add it to the resource metadata to sol
340311
com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:186)
341312
```
342313

343-
10. 受 `apache/calcite` 使用的 `janino-compiler/janino` 的影响,
314+
9. 受 `apache/calcite` 使用的 `janino-compiler/janino` 的影响,
344315
ShardingSphere 的 `SQL Federation` 功能在 GraalVM Native Image 下不可用。
345316
这同样导致 ShardingSphere Proxy Native 无法使用 OpenGauss 集成。
346317

347-
11. 受 https://github.com/oracle/graal/issues/11280 影响,
318+
10. 受 https://github.com/oracle/graal/issues/11280 影响,
348319
Etcd 的 Cluster 模式集成无法在通过 Windows 11 编译的 GraalVM Native Image 下使用,
349320
且 Etcd 的 Cluster 模式会与 GraalVM Tracing Agent 产生冲突。
350321
若开发者需要在通过 Linux 编译的 GraalVM Native Image 下使用 Etcd 的 Cluster 模式,

docs/document/content/user-manual/shardingsphere-jdbc/graalvm-native-image/_index.en.md

Lines changed: 5 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -291,42 +291,13 @@ Users need to define the following JSON in the `reachability-metadata.json` file
291291
}
292292
```
293293

294-
6. When using the ClickHouse dialect through ShardingSphere JDBC,
295-
users need to manually introduce the relevant optional modules and the ClickHouse JDBC driver with the classifier `http`.
296-
In principle, ShardingSphere's GraalVM Native Image integration does not want to use `com.clickhouse:clickhouse-jdbc` with classifier `all`,
297-
because Uber Jar will cause the collection of duplicate GraalVM Reachability Metadata.
298-
Possible configuration examples are as follows,
299-
300-
```xml
301-
<project>
302-
<dependencies>
303-
<dependency>
304-
<groupId>org.apache.shardingsphere</groupId>
305-
<artifactId>shardingsphere-jdbc</artifactId>
306-
<version>${shardingsphere.version}</version>
307-
</dependency>
308-
<dependency>
309-
<groupId>org.apache.shardingsphere</groupId>
310-
<artifactId>shardingsphere-jdbc-dialect-clickhouse</artifactId>
311-
<version>${shardingsphere.version}</version>
312-
</dependency>
313-
<dependency>
314-
<groupId>com.clickhouse</groupId>
315-
<artifactId>clickhouse-jdbc</artifactId>
316-
<version>0.6.3</version>
317-
<classifier>http</classifier>
318-
</dependency>
319-
</dependencies>
320-
</project>
321-
```
322-
323-
7. ShardingSphere's unit test only uses the Maven module `io.github.linghengqian:hive-server2-jdbc-driver-thin` to verify the availability of HiveServer2 integration under GraalVM Native Image.
294+
6. ShardingSphere's unit test only uses the Maven module `io.github.linghengqian:hive-server2-jdbc-driver-thin` to verify the availability of HiveServer2 integration under GraalVM Native Image.
324295
If developers use `org.apache.hive:hive-jdbc` directly, they should handle dependency conflicts and provide additional GraalVM Reachability Metadata by themselves.
325296

326-
8. Due to https://github.com/oracle/graal/issues/7979 ,
297+
7. Due to https://github.com/oracle/graal/issues/7979 ,
327298
the Oracle JDBC Driver corresponding to the `com.oracle.database.jdbc:ojdbc8` Maven module cannot be used under GraalVM Native Image.
328299

329-
9. Including but not limited to `com.mysql.cj.LocalizedErrorMessages`,
300+
8. Including but not limited to `com.mysql.cj.LocalizedErrorMessages`,
330301
`com.microsoft.sqlserver.jdbc.SQLServerResource`,
331302
`org.postgresql.translation.messages`,
332303
`org.opengauss.translation.messages` from third-party dependencies. By default, L10N resources are loaded according to the system's default locale,
@@ -351,11 +322,11 @@ without it being registered as reachable. Add it to the resource metadata to sol
351322
com.mysql.cj.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:186)
352323
```
353324

354-
10. Due to the use of `janino-compiler/janino` by `apache/calcite`,
325+
9. Due to the use of `janino-compiler/janino` by `apache/calcite`,
355326
ShardingSphere's `SQL Federation` feature is unavailable in the GraalVM Native Image.
356327
This also prevents ShardingSphere Proxy Native from integrating with OpenGauss.
357328

358-
11. Due to the issue at https://github.com/oracle/graal/issues/11280,
329+
10. Due to the issue at https://github.com/oracle/graal/issues/11280,
359330
Etcd's Cluster mode integration cannot be used on GraalVM Native Images compiled via Windows 11,
360331
and Etcd's Cluster mode will conflict with the GraalVM Tracing Agent.
361332
If developers need to use Etcd's Cluster mode on GraalVM Native Images compiled via Linux,

docs/document/content/user-manual/shardingsphere-jdbc/optional-plugins/clickhouse/_index.cn.md

Lines changed: 9 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,8 @@ ShardingSphere 对 ClickHouse JDBC Driver 的支持位于可选模块中。
2828
<dependency>
2929
<groupId>com.clickhouse</groupId>
3030
<artifactId>clickhouse-jdbc</artifactId>
31-
<classifier>http</classifier>
32-
<version>0.6.3</version>
31+
<classifier>all</classifier>
32+
<version>0.9.4</version>
3333
</dependency>
3434
</dependencies>
3535
```
@@ -43,7 +43,7 @@ ShardingSphere 对 ClickHouse JDBC Driver 的支持位于可选模块中。
4343
```yaml
4444
services:
4545
clickhouse-server:
46-
image: clickhouse/clickhouse-server:25.9.6.117
46+
image: clickhouse/clickhouse-server:25.10.3.100
4747
environment:
4848
CLICKHOUSE_SKIP_USER_SETUP: "1"
4949
ports:
@@ -167,24 +167,14 @@ public class ExampleUtils {
167167
### SQL 限制
168168

169169
ShardingSphere JDBC DataSource 尚不支持执行 ClickHouse 的 `create table``truncate table``drop table` 语句。
170-
用户应考虑为 ShardingSphere 提交包含单元测试的 PR。
170+
当前 ShardingSphere 对 ClickHouse 的 `INNER JOIN` 语法解析存在不足,
171+
`SELECT i.* FROM t_order o, t_order_item i WHERE o.order_id = i.order_id` 这类 SQL 可能返回错误的查询结果。
171172

172173
### 分布式序列限制
173174

174-
ClickHouse 自身的,对应分布式序列功能的列类型是 `UUID``UUID` 在 ClickHouse JDBC Driver 中接收为 `java.util.UUID`
175-
参考 https://github.com/ClickHouse/ClickHouse/issues/56228
176-
而 ShardingSphere 的 `SNOWFLAKE` 的分布式序列 SPI 实现对应的列类型是 `UInt64`
177-
在 ShardingSphere JDBC Driver 中接收为 `java.lang.Long`
178-
179-
当为 ShardingSphere 配置连接至 ClickHouse 时, 若同时配置了 ShardingSphere 使用 `SNOWFLAKE` 的分布式序列 SPI 实现,
180-
ShardingSphere 的分布式序列功能使用的 ClickHouse 真实数据库中的列类型不应该被设置为 `UUID`
181-
182-
由于 `com.clickhouse:clickhouse-jdbc:0.6.3:http` Maven 模块的 `com.clickhouse.jdbc.ClickHouseConnection#prepareStatement(String, int)`
183-
故意在 `autoGeneratedKeys``java.sql.Statement.RETURN_GENERATED_KEYS` 时抛出异常,
184-
以阻止 ShardingSphere 正常代理 `com.clickhouse.jdbc.internal.ClickHouseConnectionImpl`
185-
因此如果用户需要从 JDBC 业务代码获取 ShardingSphere 生成的分布式序列,需要将 `autoGeneratedKeys` 置为 `java.sql.Statement.NO_GENERATED_KEYS`
186-
187-
一个可能的示例如下,
175+
https://github.com/ClickHouse/ClickHouse/issues/21697 影响,
176+
由于 ClickHouse 不支持 `INSERT ... RETURNING` 语法,
177+
开发者无法在向 ShardingSphere 的逻辑数据源执行 `INSERT` SQL 后获得分布式序列。即,如下操作是不允许的,
188178

189179
```java
190180
import com.zaxxer.hikari.HikariConfig;
@@ -199,7 +189,7 @@ public class ExampleTest {
199189
Connection connection = dataSource.getConnection();
200190
PreparedStatement preparedStatement = connection.prepareStatement(
201191
"INSERT INTO t_order (user_id, order_type, address_id, status) VALUES (1, 1, 1, 'INSERT_TEST')",
202-
Statement.NO_GENERATED_KEYS
192+
Statement.RETURN_GENERATED_KEYS
203193
)) {
204194
preparedStatement.executeUpdate();
205195
try (ResultSet resultSet = preparedStatement.getGeneratedKeys()) {
@@ -227,9 +217,3 @@ ClickHouse 不支持 ShardingSphere 集成级别的本地事务,XA 事务或 S
227217
嵌入式 ClickHouse `chDB` 尚未发布 Java 客户端,
228218
ShardingSphere 不针对 SNAPSHOT 版本的 https://github.com/chdb-io/chdb-java 做集成测试。
229219
参考 https://github.com/chdb-io/chdb/issues/243
230-
231-
### ClickHouse JDBC Driver V2 限制
232-
233-
ClickHouse JDBC Driver V2 自 https://github.com/ClickHouse/clickhouse-java/pull/2368 所在的 `0.8.6` 里程碑开始,
234-
使用 `org.antlr:antlr4-maven-plugin:4.13.2`。这与 ShardingSphere 使用的 `org.antlr:antlr4-runtime:4.10.1` 产生冲突。
235-
ShardingSphere 仅使用 `com.clickhouse:clickhouse-jdbc:0.6.3:http` 测试 ClickHouse 集成。

docs/document/content/user-manual/shardingsphere-jdbc/optional-plugins/clickhouse/_index.en.md

Lines changed: 10 additions & 29 deletions
Original file line numberDiff line numberDiff line change
@@ -28,8 +28,8 @@ the possible Maven dependencies are as follows,
2828
<dependency>
2929
<groupId>com.clickhouse</groupId>
3030
<artifactId>clickhouse-jdbc</artifactId>
31-
<classifier>http</classifier>
32-
<version>0.6.3</version>
31+
<classifier>all</classifier>
32+
<version>0.9.4</version>
3333
</dependency>
3434
</dependencies>
3535
```
@@ -43,7 +43,7 @@ Write a Docker Compose file to start ClickHouse.
4343
```yaml
4444
services:
4545
clickhouse-server:
46-
image: clickhouse/clickhouse-server:25.9.6.117
46+
image: clickhouse/clickhouse-server:25.10.3.100
4747
environment:
4848
CLICKHOUSE_SKIP_USER_SETUP: "1"
4949
ports:
@@ -170,27 +170,15 @@ public class ExampleUtils {
170170

171171
ShardingSphere JDBC DataSource does not yet support executing ClickHouse's `create table`, `truncate table`,
172172
and `drop table` statements.
173-
Users should consider submitting a PR containing unit tests for ShardingSphere.
173+
The current ShardingSphere parsing of ClickHouse's `INNER JOIN` syntax has shortcomings,
174+
and may return incorrect query results for SQL statements such as `SELECT i.* FROM t_order o, t_order_item i WHERE o.order_id = i.order_id`.
174175

175176
### Key Generate restrictions
176177

177-
The column type corresponding to the Key Generate function of ClickHouse itself is `UUID`,
178-
and `UUID` is received as `java.util.UUID` in ClickHouse JDBC Driver,
179-
refer to https://github.com/ClickHouse/ClickHouse/issues/56228 .
180-
The column type corresponding to the Key Generate SPI implementation of ShardingSphere's `SNOWFLAKE` is `UInt64`,
181-
which is received as `java.lang.Long` in ShardingSphere JDBC Driver.
182-
183-
When configuring ShardingSphere to connect to ClickHouse,
184-
if ShardingSphere is also configured to use the Key Generate SPI implementation of `SNOWFLAKE`,
185-
the column type in the ClickHouse real database used by ShardingSphere's Key Generate function should not be set to `UUID`.
186-
187-
Because `com.clickhouse.jdbc.ClickHouseConnection#prepareStatement(String, int)` of `com.clickhouse:clickhouse-jdbc:0.6.3:http`
188-
Maven module intentionally throws an exception when `autoGeneratedKeys` is `java.sql.Statement.RETURN_GENERATED_KEYS`,
189-
to prevent ShardingSphere from proxying `com.clickhouse.jdbc.internal.ClickHouseConnectionImpl` normally,
190-
therefore, if users need to obtain the Key generated by ShardingSphere from the JDBC business code,
191-
they need to set `autoGeneratedKeys` to `java.sql.Statement.NO_GENERATED_KEYS`.
192-
193-
A possible example is as follows,
178+
Due to the issue mentioned in https://github.com/ClickHouse/ClickHouse/issues/21697 ,
179+
because ClickHouse does not support the `INSERT ... RETURNING` syntax,
180+
developers cannot obtain distributed sequences after executing `INSERT` SQL into ShardingSphere's logical data source.
181+
Specifically, the following operations are not allowed:
194182

195183
```java
196184
import com.zaxxer.hikari.HikariConfig;
@@ -205,7 +193,7 @@ public class ExampleTest {
205193
Connection connection = dataSource.getConnection();
206194
PreparedStatement preparedStatement = connection.prepareStatement(
207195
"INSERT INTO t_order (user_id, order_type, address_id, status) VALUES (1, 1, 1, 'INSERT_TEST')",
208-
Statement.NO_GENERATED_KEYS
196+
Statement.RETURN_GENERATED_KEYS
209197
)) {
210198
preparedStatement.executeUpdate();
211199
try (ResultSet resultSet = preparedStatement.getGeneratedKeys()) {
@@ -233,10 +221,3 @@ See https://github.com/ClickHouse/clickhouse-java/issues/2023 .
233221
The embedded ClickHouse `chDB` Java client has not been released yet.
234222
ShardingSphere does not do integration testing for the SNAPSHOT version of https://github.com/chdb-io/chdb-java .
235223
Refer to https://github.com/chdb-io/chdb/issues/243 .
236-
237-
### Limitations of ClickHouse JDBC Driver V2
238-
239-
Starting from the `0.8.6` milestone at https://github.com/ClickHouse/clickhouse-java/pull/2368 ,
240-
ClickHouse JDBC Driver V2 uses `org.antlr:antlr4-maven-plugin:4.13.2`.
241-
This conflicts with `org.antlr:antlr4-runtime:4.10.1` used by ShardingSphere.
242-
ShardingSphere only uses `com.clickhouse:clickhouse-jdbc:0.6.3:http` to test ClickHouse integration.

docs/document/content/user-manual/shardingsphere-jdbc/optional-plugins/hiveserver2/_index.cn.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -366,9 +366,9 @@ ShardingSphere 仅针对 HiveServer2 `4.0.1` 进行集成测试。
366366

367367
### SQL 限制
368368

369-
HiveServer2 并不能保证每一条 `insert` 相关的 DML SQL 都能成功执行,尽管可能没有任何异常被抛出。
370-
371369
ShardingSphere JDBC DataSource 尚不支持执行 HiveServer2 的 `set` 语句。
370+
当前 ShardingSphere 对 HiveServer2 的 `INNER JOIN` 语法解析存在不足,
371+
对 `SELECT i.* FROM t_order o, t_order_item i WHERE o.order_id = i.order_id` 这类 SQL 可能返回错误的查询结果。
372372

373373
#### 使用 `initFile` 参数部分绕开 SQL 限制
374374

docs/document/content/user-manual/shardingsphere-jdbc/optional-plugins/hiveserver2/_index.en.md

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -372,9 +372,9 @@ Reference https://issues.apache.org/jira/browse/HIVE-28418 .
372372

373373
### SQL Limitations
374374

375-
HiveServer2 does not guarantee that every `insert` related DML SQL can be executed successfully, although no exception may be thrown.
376-
377375
ShardingSphere JDBC DataSource does not yet support executing the `set` statement of HiveServer2.
376+
The current ShardingSphere parsing of HiveServer2's `INNER JOIN` syntax has shortcomings,
377+
and may return incorrect query results for SQL statements such as `SELECT i.* FROM t_order o, t_order_item i WHERE o.order_id = i.order_id`.
378378

379379
#### Use `initFile` parameter to partially bypass SQL restrictions
380380

infra/reachability-metadata/src/main/resources/META-INF/native-image/com.clickhouse/clickhouse-jdbc/0.6.3/reachability-metadata.json

Lines changed: 0 additions & 16 deletions
This file was deleted.

0 commit comments

Comments
 (0)