Skip to content

Commit 5465f08

Browse files
committed
Refactor ClickHouse's nativeTest
1 parent fd65142 commit 5465f08

13 files changed

Lines changed: 128 additions & 138 deletions

File tree

docs/document/content/user-manual/shardingsphere-jdbc/graalvm-native-image/_index.cn.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -47,7 +47,7 @@ java.beans.Introspector was unintentionally initialized at build time. To see wh
4747
<plugin>
4848
<groupId>org.graalvm.buildtools</groupId>
4949
<artifactId>native-maven-plugin</artifactId>
50-
<version>0.10.5</version>
50+
<version>0.10.6</version>
5151
<extensions>true</extensions>
5252
<configuration>
5353
<buildArgs>
@@ -85,12 +85,12 @@ java.beans.Introspector was unintentionally initialized at build time. To see wh
8585

8686
```groovy
8787
plugins {
88-
id 'org.graalvm.buildtools.native' version '0.10.5'
88+
id 'org.graalvm.buildtools.native' version '0.10.6'
8989
}
9090
9191
dependencies {
9292
implementation 'org.apache.shardingsphere:shardingsphere-jdbc:${shardingsphere.version}'
93-
implementation(group: 'org.graalvm.buildtools', name: 'graalvm-reachability-metadata', version: '0.10.5', classifier: 'repository', ext: 'zip')
93+
implementation(group: 'org.graalvm.buildtools', name: 'graalvm-reachability-metadata', version: '0.10.6', classifier: 'repository', ext: 'zip')
9494
}
9595
9696
graalvmNative {

docs/document/content/user-manual/shardingsphere-jdbc/graalvm-native-image/_index.en.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ and the documentation of GraalVM Native Build Tools shall prevail.
4949
<plugin>
5050
<groupId>org.graalvm.buildtools</groupId>
5151
<artifactId>native-maven-plugin</artifactId>
52-
<version>0.10.5</version>
52+
<version>0.10.6</version>
5353
<extensions>true</extensions>
5454
<configuration>
5555
<buildArgs>
@@ -89,12 +89,12 @@ Reference https://github.com/graalvm/native-build-tools/issues/572 .
8989

9090
```groovy
9191
plugins {
92-
id 'org.graalvm.buildtools.native' version '0.10.5'
92+
id 'org.graalvm.buildtools.native' version '0.10.6'
9393
}
9494
9595
dependencies {
9696
implementation 'org.apache.shardingsphere:shardingsphere-jdbc:${shardingsphere.version}'
97-
implementation(group: 'org.graalvm.buildtools', name: 'graalvm-reachability-metadata', version: '0.10.5', classifier: 'repository', ext: 'zip')
97+
implementation(group: 'org.graalvm.buildtools', name: 'graalvm-reachability-metadata', version: '0.10.6', classifier: 'repository', ext: 'zip')
9898
}
9999
100100
graalvmNative {

docs/document/content/user-manual/shardingsphere-jdbc/graalvm-native-image/development/_index.cn.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,7 +27,7 @@ ShardingSphere 定义了,
2727

2828
1. GraalVM CE 22.0.2,或与 GraalVM CE 22.0.2 兼容的 GraalVM 下游发行版。以 [GraalVM Native Image](/cn/user-manual/shardingsphere-jdbc/graalvm-native-image) 为准。
2929
2. 编译 GraalVM Native Image 所需要的本地工具链。以 https://www.graalvm.org/latest/reference-manual/native-image/#prerequisites 为准。
30-
3. 可运行 Linux Containers 的 Docker Engine,或与 testcontainer-java 兼容的 Container Runtime。以 https://java.testcontainers.org/supported_docker_environment/ 为准。
30+
3. 可运行 Linux Containers 的 Docker Engine,或与 testcontainers-java 兼容的 Container Runtime。以 https://java.testcontainers.org/supported_docker_environment/ 为准。
3131

3232
本文不讨论 `LLVM Backend for Native Image`。下文分别讨论在 Ubuntu,Windows 与 Windows Server 下可能的所需操作。
3333

docs/document/content/user-manual/shardingsphere-jdbc/graalvm-native-image/development/_index.en.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -29,7 +29,7 @@ Developer must have installed on their devices,
2929

3030
1. GraalVM CE 22.0.2, or a GraalVM downstream distribution compatible with GraalVM CE 22.0.2. Refer to [GraalVM Native Image](/en/user-manual/shardingsphere-jdbc/graalvm-native-image).
3131
2. The native toolchain required to compile the GraalVM Native Image. Refer to https://www.graalvm.org/latest/reference-manual/native-image/#prerequisites .
32-
3. Docker Engine that can run Linux Containers, or a Container Runtime compatible with testcontainer-java. Refer to https://java.testcontainers.org/supported_docker_environment/ .
32+
3. Docker Engine that can run Linux Containers, or a Container Runtime compatible with testcontainers-java. Refer to https://java.testcontainers.org/supported_docker_environment/ .
3333

3434
This article does not discuss `LLVM Backend for Native Image`. The following sections discuss the possible required operations under Ubuntu, Windows, and Windows Server.
3535

docs/document/content/user-manual/shardingsphere-jdbc/optional-plugins/clickhouse/_index.cn.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ ShardingSphere 对 ClickHouse JDBC Driver 的支持位于可选模块中。
4343
```yaml
4444
services:
4545
clickhouse-server:
46-
image: clickhouse/clickhouse-server:24.11.1.2557
46+
image: clickhouse/clickhouse-server:25.4.5.24
4747
ports:
4848
- "8123:8123"
4949
```

docs/document/content/user-manual/shardingsphere-jdbc/optional-plugins/clickhouse/_index.en.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -43,7 +43,7 @@ Write a Docker Compose file to start ClickHouse.
4343
```yaml
4444
services:
4545
clickhouse-server:
46-
image: clickhouse/clickhouse-server:24.11.1.2557
46+
image: clickhouse/clickhouse-server:25.4.5.24
4747
ports:
4848
- "8123:8123"
4949
```

pom.xml

Lines changed: 9 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -148,7 +148,7 @@
148148
<hamcrest.version>3.0</hamcrest.version>
149149
<mockito.version>4.11.0</mockito.version>
150150
<awaitility.version>4.2.2</awaitility.version>
151-
<testcontainers.version>1.20.3</testcontainers.version>
151+
<testcontainers.version>1.21.0</testcontainers.version>
152152
<commons-csv.version>1.9.0</commons-csv.version>
153153

154154
<graal-sdk.version>24.1.2</graal-sdk.version>
@@ -158,7 +158,7 @@
158158
<protobuf-maven-plugin.version>0.6.1</protobuf-maven-plugin.version>
159159
<dockerfile-maven.version>1.4.13</dockerfile-maven.version>
160160
<os-detector-maven-plugin.version>0.3.1</os-detector-maven-plugin.version>
161-
<native-maven-plugin.version>0.10.5</native-maven-plugin.version>
161+
<native-maven-plugin.version>0.10.6</native-maven-plugin.version>
162162

163163
<!-- Compile plugin versions -->
164164
<templating-maven-plugin.version>1.0.0</templating-maven-plugin.version>
@@ -1147,6 +1147,13 @@
11471147
<quickBuild>true</quickBuild>
11481148
</configuration>
11491149
<executions>
1150+
<execution>
1151+
<id>build-native</id>
1152+
<goals>
1153+
<goal>compile-no-fork</goal>
1154+
</goals>
1155+
<phase>package</phase>
1156+
</execution>
11501157
<execution>
11511158
<id>test-native</id>
11521159
<goals>

test/native/pom.xml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -254,6 +254,11 @@
254254
<artifactId>mssqlserver</artifactId>
255255
<scope>test</scope>
256256
</dependency>
257+
<dependency>
258+
<groupId>org.testcontainers</groupId>
259+
<artifactId>clickhouse</artifactId>
260+
<scope>test</scope>
261+
</dependency>
257262
<dependency>
258263
<groupId>org.apache.curator</groupId>
259264
<artifactId>curator-test</artifactId>
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
/*
2+
* Licensed to the Apache Software Foundation (ASF) under one or more
3+
* contributor license agreements. See the NOTICE file distributed with
4+
* this work for additional information regarding copyright ownership.
5+
* The ASF licenses this file to You under the Apache License, Version 2.0
6+
* (the "License"); you may not use this file except in compliance with
7+
* the License. You may obtain a copy of the License at
8+
*
9+
* http://www.apache.org/licenses/LICENSE-2.0
10+
*
11+
* Unless required by applicable law or agreed to in writing, software
12+
* distributed under the License is distributed on an "AS IS" BASIS,
13+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
* See the License for the specific language governing permissions and
15+
* limitations under the License.
16+
*/
17+
18+
package org.apache.shardingsphere.test.natived.commons.algorithm.testcontainers;
19+
20+
import org.apache.shardingsphere.infra.database.core.type.DatabaseType;
21+
import org.apache.shardingsphere.infra.spi.type.typed.TypedSPILoader;
22+
23+
import java.util.Collection;
24+
import java.util.Collections;
25+
import java.util.Optional;
26+
27+
/**
28+
* Database type of Clickhouse in testcontainers.
29+
*/
30+
public final class TcClickhouseDatabaseType implements TestcontainersDatabaseType {
31+
32+
@Override
33+
public Collection<String> getJdbcUrlPrefixes() {
34+
return Collections.singleton("jdbc:tc:clickhouse:");
35+
}
36+
37+
@Override
38+
public Optional<DatabaseType> getTrunkDatabaseType() {
39+
return Optional.of(TypedSPILoader.getService(DatabaseType.class, "ClickHouse"));
40+
}
41+
42+
@Override
43+
public String getType() {
44+
return "TC-Clickhouse";
45+
}
46+
}

test/native/src/test/java/org/apache/shardingsphere/test/natived/jdbc/databases/ClickHouseTest.java

Lines changed: 6 additions & 114 deletions
Original file line numberDiff line numberDiff line change
@@ -24,54 +24,22 @@
2424
import org.apache.shardingsphere.infra.metadata.database.resource.unit.StorageUnit;
2525
import org.apache.shardingsphere.mode.manager.ContextManager;
2626
import org.apache.shardingsphere.test.natived.commons.TestShardingService;
27-
import org.awaitility.Awaitility;
2827
import org.junit.jupiter.api.AfterEach;
29-
import org.junit.jupiter.api.BeforeEach;
3028
import org.junit.jupiter.api.Test;
3129
import org.junit.jupiter.api.condition.EnabledInNativeImage;
32-
import org.testcontainers.containers.GenericContainer;
33-
import org.testcontainers.junit.jupiter.Container;
30+
import org.testcontainers.jdbc.ContainerDatabaseDriver;
3431
import org.testcontainers.junit.jupiter.Testcontainers;
3532

3633
import javax.sql.DataSource;
3734
import java.sql.Connection;
38-
import java.sql.DriverManager;
3935
import java.sql.SQLException;
40-
import java.sql.Statement;
41-
import java.time.Duration;
42-
import java.util.Properties;
43-
import java.util.stream.Stream;
4436

45-
import static org.hamcrest.MatcherAssert.assertThat;
46-
import static org.hamcrest.Matchers.is;
47-
import static org.hamcrest.Matchers.nullValue;
48-
49-
/**
50-
* Cannot use testcontainers-java style jdbcURL for Clickhouse Server due to unresolved
51-
* <a href="https://github.com/testcontainers/testcontainers-java/issues/8736">testcontainers/testcontainers-java#8736</a>.
52-
*/
53-
@SuppressWarnings({"SqlNoDataSourceInspection", "resource"})
5437
@EnabledInNativeImage
5538
@Testcontainers
5639
class ClickHouseTest {
5740

58-
@Container
59-
private final GenericContainer<?> container = new GenericContainer<>("clickhouse/clickhouse-server:24.11.1.2557")
60-
.withExposedPorts(8123);
61-
62-
private final String systemPropKeyPrefix = "fixture.test-native.yaml.database.clickhouse.";
63-
6441
private DataSource logicDataSource;
6542

66-
private String jdbcUrlPrefix;
67-
68-
@BeforeEach
69-
void beforeEach() {
70-
assertThat(System.getProperty(systemPropKeyPrefix + "ds0.jdbc-url"), is(nullValue()));
71-
assertThat(System.getProperty(systemPropKeyPrefix + "ds1.jdbc-url"), is(nullValue()));
72-
assertThat(System.getProperty(systemPropKeyPrefix + "ds2.jdbc-url"), is(nullValue()));
73-
}
74-
7543
@AfterEach
7644
void afterEach() throws SQLException {
7745
try (Connection connection = logicDataSource.getConnection()) {
@@ -81,92 +49,16 @@ void afterEach() throws SQLException {
8149
}
8250
contextManager.close();
8351
}
84-
System.clearProperty(systemPropKeyPrefix + "ds0.jdbc-url");
85-
System.clearProperty(systemPropKeyPrefix + "ds1.jdbc-url");
86-
System.clearProperty(systemPropKeyPrefix + "ds2.jdbc-url");
52+
ContainerDatabaseDriver.killContainers();
8753
}
8854

8955
@Test
9056
void assertShardingInLocalTransactions() throws SQLException {
91-
jdbcUrlPrefix = "jdbc:ch://localhost:" + container.getMappedPort(8123) + "/";
92-
logicDataSource = createDataSource();
93-
TestShardingService testShardingService = new TestShardingService(logicDataSource);
94-
testShardingService.processSuccessInClickHouse();
95-
}
96-
97-
private Connection openConnection(final String databaseName) throws SQLException {
98-
Properties props = new Properties();
99-
props.setProperty("user", "default");
100-
props.setProperty("password", "");
101-
return DriverManager.getConnection(jdbcUrlPrefix + databaseName, props);
102-
}
103-
104-
private DataSource createDataSource() throws SQLException {
105-
Awaitility.await().atMost(Duration.ofMinutes(1L)).ignoreExceptions().until(() -> {
106-
openConnection("default").close();
107-
return true;
108-
});
109-
try (
110-
Connection connection = openConnection("default");
111-
Statement statement = connection.createStatement()) {
112-
statement.executeUpdate("CREATE DATABASE demo_ds_0");
113-
statement.executeUpdate("CREATE DATABASE demo_ds_1");
114-
statement.executeUpdate("CREATE DATABASE demo_ds_2");
115-
}
116-
Stream.of("demo_ds_0", "demo_ds_1", "demo_ds_2").parallel().forEach(this::initTable);
11757
HikariConfig config = new HikariConfig();
11858
config.setDriverClassName("org.apache.shardingsphere.driver.ShardingSphereDriver");
119-
config.setJdbcUrl("jdbc:shardingsphere:classpath:test-native/yaml/jdbc/databases/clickhouse.yaml?placeholder-type=system_props");
120-
System.setProperty(systemPropKeyPrefix + "ds0.jdbc-url", jdbcUrlPrefix + "demo_ds_0");
121-
System.setProperty(systemPropKeyPrefix + "ds1.jdbc-url", jdbcUrlPrefix + "demo_ds_1");
122-
System.setProperty(systemPropKeyPrefix + "ds2.jdbc-url", jdbcUrlPrefix + "demo_ds_2");
123-
return new HikariDataSource(config);
124-
}
125-
126-
/**
127-
* ClickHouse does not support `AUTO_INCREMENT`,
128-
* refer to <a href="https://github.com/ClickHouse/ClickHouse/issues/56228">ClickHouse/ClickHouse#56228</a> .
129-
* TODO The {@code shardingsphere-parser-sql-clickhouse} module needs to be fixed to use SQL like `create table`,
130-
* `truncate table` and `drop table`.
131-
*
132-
* @param databaseName database name
133-
* @throws RuntimeException SQL exception
134-
*/
135-
private void initTable(final String databaseName) {
136-
try (
137-
Connection connection = openConnection(databaseName);
138-
Statement statement = connection.createStatement()) {
139-
statement.execute("create table IF NOT EXISTS t_order\n"
140-
+ "(\n"
141-
+ " order_id Int64 NOT NULL,\n"
142-
+ " order_type Int32,\n"
143-
+ " user_id Int32 NOT NULL,\n"
144-
+ " address_id Int64 NOT NULL,\n"
145-
+ " status VARCHAR(50)\n"
146-
+ ") engine = MergeTree\n"
147-
+ " primary key (order_id)\n"
148-
+ " order by (order_id)");
149-
statement.execute("create table IF NOT EXISTS t_order_item\n"
150-
+ "(\n"
151-
+ " order_item_id Int64 NOT NULL,\n"
152-
+ " order_id Int64 NOT NULL,\n"
153-
+ " user_id Int32 NOT NULL,\n"
154-
+ " phone VARCHAR(50),\n"
155-
+ " status VARCHAR(50)\n"
156-
+ ") engine = MergeTree\n"
157-
+ " primary key (order_item_id)\n"
158-
+ " order by (order_item_id)");
159-
statement.execute("CREATE TABLE IF NOT EXISTS t_address\n"
160-
+ "(\n"
161-
+ " address_id BIGINT NOT NULL,\n"
162-
+ " address_name VARCHAR(100) NOT NULL,\n"
163-
+ " PRIMARY KEY (address_id)\n"
164-
+ ")");
165-
statement.execute("TRUNCATE TABLE t_order");
166-
statement.execute("TRUNCATE TABLE t_order_item");
167-
statement.execute("TRUNCATE TABLE t_address");
168-
} catch (final SQLException exception) {
169-
throw new RuntimeException(exception);
170-
}
59+
config.setJdbcUrl("jdbc:shardingsphere:classpath:test-native/yaml/jdbc/databases/clickhouse.yaml");
60+
logicDataSource = new HikariDataSource(config);
61+
TestShardingService testShardingService = new TestShardingService(logicDataSource);
62+
testShardingService.processSuccessInClickHouse();
17163
}
17264
}

0 commit comments

Comments
 (0)