Merge branch 'master' into multi-path

Author: VGalaxies

CONTRIBUTING.md

@@ -27,7 +27,7 @@ IoTDB official website：https://iotdb.apache.org/
 
 Code library：https://github.com/apache/iotdb/tree/master
 
-Get started quickly：https://iotdb.apache.org/UserGuide/Master/QuickStart/QuickStart.html
+Get started quickly：https://iotdb.apache.org/UserGuide/latest/QuickStart/QuickStart.html
 
 ## Subscribe to mailing list
 

README.md

@@ -83,7 +83,7 @@ For the latest information about IoTDB, please visit [IoTDB official website](ht
 
 # Quick Start
 
-This short guide will walk you through the basic process of using IoTDB. For a more detailed introduction, please visit our website's [User Guide](https://iotdb.apache.org/UserGuide/Master/QuickStart/QuickStart.html).
+This short guide will walk you through the basic process of using IoTDB. For a more detailed introduction, please visit our website's [User Guide](https://iotdb.apache.org/UserGuide/latest/QuickStart/QuickStart.html).
 
 ## Prerequisites
 
@@ -220,7 +220,7 @@ IoTDB provides three installation methods, you can refer to the following sugges
 * Using Docker：The path to the dockerfile is [here](https://github.com/apache/iotdb/tree/master/docker/src/main).
 
 
-Here in the Quick Start, we give a brief introduction of using source code to install IoTDB. For further information, please refer to [User Guide](https://iotdb.apache.org/UserGuide/Master/QuickStart/QuickStart.html).
+Here in the Quick Start, we give a brief introduction of using source code to install IoTDB. For further information, please refer to [User Guide](https://iotdb.apache.org/UserGuide/latest/QuickStart/QuickStart.html).
 
 ## Build from source
 
@@ -307,10 +307,10 @@ you run `mvn package` successfully.**
 Configuration files are under the "conf" folder.
 
   * environment config module (`datanode-env.bat`, `datanode-env.sh`),
-  * system config module (`iotdb-datanode.properties`)
+  * system config module (`iotdb-system.properties`)
   * log config module (`logback.xml`).
 
-For more information, please see [Config Manual](https://iotdb.apache.org/UserGuide/Master/Reference/DataNode-Config-Manual.html).
+For more information, please see [Config Manual](https://iotdb.apache.org/UserGuide/latest/Reference/DataNode-Config-Manual.html).
 
 ## Start
 
@@ -482,7 +482,7 @@ or
 IoTDB> exit
 ```
 
-For more information about the commands supported by IoTDB SQL, please see [User Guide](https://iotdb.apache.org/UserGuide/Master/QuickStart/QuickStart.html).
+For more information about the commands supported by IoTDB SQL, please see [User Guide](https://iotdb.apache.org/UserGuide/latest/QuickStart/QuickStart.html).
 
 ### Stop IoTDB
 
@@ -496,12 +496,14 @@ The server can be stopped with "ctrl-C" or the following script:
 > sbin\stop-standalone.bat
 ```
 
-# The use of CSV Import and Export Tool
+# The use of Data Import and Export Tool
+
+see [The use of Data Import Tool](https://iotdb.apache.org/UserGuide/latest/Tools-System/Data-Import-Tool.html)
+see [The use of Data Export Tool](https://iotdb.apache.org/UserGuide/latest/Tools-System/Data-Export-Tool.html)
 
-see [The use of CSV Import and Export Tool](https://iotdb.apache.org/UserGuide/latest/Tools-System/Import-Export-Tool.html)
 
 # Frequent Questions for Compiling
-see [Frequent Questions when Compiling the Source Code](https://iotdb.apache.org/Development/ContributeGuide.html#_Frequent-Questions-when-Compiling-the-Source-Code)
+see [Frequent Questions when Compiling the Source Code](https://iotdb.apache.org/Community/Development-Guide.html#frequently-asked-questions)
 
 # Contact Us
 

README_ZH.md

@@ -81,7 +81,7 @@ IoTDB的主要特点如下:
 
 # 快速开始
 
-这篇简短的指南将带您了解使用IoTDB的基本过程。如需更详细的介绍，请访问我们的网站[用户指南](https://iotdb.apache.org/zh/UserGuide/Master/QuickStart/QuickStart.html)。
+这篇简短的指南将带您了解使用IoTDB的基本过程。如需更详细的介绍，请访问我们的网站[用户指南](https://iotdb.apache.org/zh/UserGuide/latest/QuickStart/QuickStart.html)。
 
 ## 环境准备
 
@@ -108,7 +108,7 @@ IoTDB提供了三种安装方法，您可以参考以下建议，选择最适合
 
 * 使用Docker: dockerfile的路径是https://github.com/apache/iotdb/tree/master/docker/src/main
 
-在这篇《快速入门》中，我们简要介绍如何使用源代码安装IoTDB。如需进一步资料，请参阅官网[用户指南](https://iotdb.apache.org/zh/UserGuide/Master/QuickStart/QuickStart.html)。
+在这篇《快速入门》中，我们简要介绍如何使用源代码安装IoTDB。如需进一步资料，请参阅官网[用户指南](https://iotdb.apache.org/zh/UserGuide/latest/QuickStart/QuickStart.html)。
 
 ## 从源码构建
 
@@ -182,10 +182,10 @@ git checkout rel/x.x
 
 配置文件在"conf"文件夹下
 * 环境配置模块(`datanode-env.bat`, `datanode-env.sh`),
-* 系统配置模块(`iotdb-datanode.properties`)
+* 系统配置模块(`iotdb-system.properties`)
 * 日志配置模块(`logback.xml`)。
 
-有关详细信息，请参见[配置参数](https://iotdb.apache.org/zh/UserGuide/Master/Reference/DataNode-Config-Manual.html)。
+有关详细信息，请参见[配置参数](https://iotdb.apache.org/zh/UserGuide/latest/Reference/DataNode-Config-Manual.html)。
 
 ## 开始
 
@@ -364,7 +364,7 @@ or
 IoTDB> exit
 ```
 
-有关IoTDB SQL支持的命令的更多信息，请参见[用户指南](https://iotdb.apache.org/zh/UserGuide/Master/QuickStart/QuickStart.html)。
+有关IoTDB SQL支持的命令的更多信息，请参见[用户指南](https://iotdb.apache.org/zh/UserGuide/latest/QuickStart/QuickStart.html)。
 
 ### 停止 IoTDB
 
@@ -378,12 +378,13 @@ server 可以使用 "ctrl-C" 或者执行下面的脚本:
 > sbin\stop-standalone.bat
 ```
 
-# 导入导出CSV工具
+# 数据导入导出工具
 
-查看 [导入导出CSV工具](https://iotdb.apache.org/zh/UserGuide/latest/Tools-System/Import-Export-Tool.html)
+查看 [数据导入工具](https://iotdb.apache.org/zh/UserGuide/latest/Tools-System/Data-Import-Tool.html)
+查看 [数据导出工具](https://iotdb.apache.org/zh/UserGuide/latest/Tools-System/Data-Export-Tool.html)
 
 # 常见编译错误
-查看 [常见编译错误](https://iotdb.apache.org/zh/Community/Development-Guide.html)
+查看 [常见编译错误](https://iotdb.apache.org/zh/Community/Development-Guide.html#%E5%B8%B8%E8%A7%81%E9%97%AE%E9%A2%98)
 
 # 联系我们
 ### QQ群

docker/ReadMe.md

@@ -81,7 +81,7 @@ docker run -d --name iotdb -p 6667:6667 -p 31999:31999 -p 8181:8181 -p 5555:5555
 ```shell
 docker run -d --name iotdb -p 6667:6667 -p 31999:31999 -p 8181:8181 -p 5555:5555 -p 9003:9003 -p 40010:40010 apache/iotdb:<version>
 ```
-Since 1.0.0, see [offical documents.](https://iotdb.apache.org/UserGuide/Master/QuickStart/WayToGetIoTDB.html)
+Since 1.0.0, see [offical documents.](https://iotdb.apache.org/UserGuide/latest/QuickStart/QuickStart.html)
 
 # How to register AINode to IoTDB cluster
 

integration-test/src/main/java/org/apache/iotdb/it/env/cluster/env/AbstractEnv.java

@@ -679,7 +679,7 @@ public Connection getConnection(
         ? new ClusterTestConnection(
             getWriteConnection(version, username, password, sqlDialect),
             getReadConnections(version, username, password, sqlDialect))
-        : getWriteConnection(version, username, password, sqlDialect).getUnderlyingConnecton();
+        : getWriteConnection(version, username, password, sqlDialect).getUnderlyingConnection();
   }
 
   @Override

integration-test/src/main/java/org/apache/iotdb/itbase/runtime/ClusterTestConnection.java

@@ -43,8 +43,8 @@
 /** The implementation of {@link Connection} in cluster test. */
 public class ClusterTestConnection implements Connection {
 
-  private final NodeConnection writeConnection;
-  private final List<NodeConnection> readConnections;
+  public final NodeConnection writeConnection;
+  public final List<NodeConnection> readConnections;
   private boolean isClosed;
 
   public ClusterTestConnection(
@@ -110,7 +110,7 @@ public boolean isClosed() {
 
   @Override
   public DatabaseMetaData getMetaData() throws SQLException {
-    return writeConnection.getUnderlyingConnecton().getMetaData();
+    return writeConnection.getUnderlyingConnection().getMetaData();
   }
 
   @Override
@@ -269,9 +269,9 @@ public boolean isValid(int timeout) {
 
   @Override
   public void setClientInfo(String name, String value) throws SQLClientInfoException {
-    writeConnection.getUnderlyingConnecton().setClientInfo(name, value);
+    writeConnection.getUnderlyingConnection().setClientInfo(name, value);
     for (NodeConnection conn : readConnections) {
-      conn.getUnderlyingConnecton().setClientInfo(name, value);
+      conn.getUnderlyingConnection().setClientInfo(name, value);
     }
   }
 

integration-test/src/main/java/org/apache/iotdb/itbase/runtime/ClusterTestStatement.java

@@ -51,7 +51,7 @@ public class ClusterTestStatement implements Statement {
   public ClusterTestStatement(
       NodeConnection writeConnection, List<NodeConnection> readConnections) {
     try {
-      this.writeStatement = writeConnection.getUnderlyingConnecton().createStatement();
+      this.writeStatement = writeConnection.getUnderlyingConnection().createStatement();
       updateConfig(writeStatement, 0);
       writEndpoint = writeConnection.toString();
     } catch (SQLException e) {
@@ -60,7 +60,7 @@ public ClusterTestStatement(
 
     for (NodeConnection readConnection : readConnections) {
       try {
-        Statement readStatement = readConnection.getUnderlyingConnecton().createStatement();
+        Statement readStatement = readConnection.getUnderlyingConnection().createStatement();
         this.readStatements.add(readStatement);
         this.readEndpoints.add(readConnection.toString());
         updateConfig(readStatement, queryTimeout);

integration-test/src/main/java/org/apache/iotdb/itbase/runtime/NodeConnection.java

@@ -33,17 +33,17 @@ public class NodeConnection {
   private final String endpoint;
   private final NodeRole nodeRole;
   private final ConnectionRole connectionRole;
-  private final Connection underlyingConnecton;
+  private final Connection underlyingConnection;
 
   public NodeConnection(
       String endpoint,
       NodeRole nodeRole,
       ConnectionRole connectionRole,
-      Connection underlyingConnecton) {
+      Connection underlyingConnection) {
     this.endpoint = endpoint;
     this.nodeRole = nodeRole;
     this.connectionRole = connectionRole;
-    this.underlyingConnecton = underlyingConnecton;
+    this.underlyingConnection = underlyingConnection;
   }
 
   public String getEndpoint() {
@@ -58,13 +58,13 @@ public ConnectionRole getConnectionRole() {
     return connectionRole;
   }
 
-  public Connection getUnderlyingConnecton() {
-    return underlyingConnecton;
+  public Connection getUnderlyingConnection() {
+    return underlyingConnection;
   }
 
   public void close() {
     try {
-      underlyingConnecton.close();
+      underlyingConnection.close();
     } catch (SQLException e) {
       logger.error("Close connection {} error", this, e);
     }

integration-test/src/test/java/org/apache/iotdb/ainode/it/AINodeConcurrentInferenceIT.java

@@ -22,22 +22,35 @@
 import org.apache.iotdb.it.env.EnvFactory;
 import org.apache.iotdb.itbase.env.BaseEnv;
 
+import com.google.common.collect.ImmutableMap;
+import com.google.common.collect.ImmutableSet;
 import org.junit.AfterClass;
+import org.junit.Assert;
 import org.junit.BeforeClass;
 import org.junit.Test;
 import org.slf4j.Logger;
 import org.slf4j.LoggerFactory;
 
 import java.sql.Connection;
+import java.sql.ResultSet;
 import java.sql.SQLException;
 import java.sql.Statement;
+import java.util.HashSet;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.TimeUnit;
 
 import static org.apache.iotdb.ainode.utils.AINodeTestUtils.concurrentInference;
 
 public class AINodeConcurrentInferenceIT {
 
   private static final Logger LOGGER = LoggerFactory.getLogger(AINodeConcurrentInferenceIT.class);
 
+  private static final Map<String, String> MODEL_ID_TO_TYPE_MAP =
+      ImmutableMap.of(
+          "timer_xl", "Timer-XL",
+          "sundial", "Timer-Sundial");
+
   @BeforeClass
   public static void setUp() throws Exception {
     // Init 1C1D1A cluster environment
@@ -91,12 +104,17 @@ private void concurrentCPUCallInferenceTest(String modelId)
         Statement statement = connection.createStatement()) {
       final int threadCnt = 4;
       final int loop = 10;
+      final int predictLength = 96;
       statement.execute(String.format("LOAD MODEL %s TO DEVICES \"cpu\"", modelId));
+      checkModelOnSpecifiedDevice(statement, MODEL_ID_TO_TYPE_MAP.get(modelId), "cpu");
       concurrentInference(
           statement,
-          String.format("CALL INFERENCE(%s, \"SELECT s FROM root.AI\")", modelId),
+          String.format(
+              "CALL INFERENCE(%s, \"SELECT s FROM root.AI\", predict_length=%d)",
+              modelId, predictLength),
           threadCnt,
-          loop);
+          loop,
+          predictLength);
       statement.execute(String.format("UNLOAD MODEL %s FROM DEVICES \"cpu\"", modelId));
     }
   }
@@ -111,14 +129,20 @@ private void concurrentGPUCallInferenceTest(String modelId)
       throws SQLException, InterruptedException {
     try (Connection connection = EnvFactory.getEnv().getConnection(BaseEnv.TREE_SQL_DIALECT);
         Statement statement = connection.createStatement()) {
-      final int threadCnt = 4;
-      final int loop = 10;
-      statement.execute(String.format("LOAD MODEL %s TO DEVICES \"0,1\"", modelId));
+      final int threadCnt = 10;
+      final int loop = 100;
+      final int predictLength = 512;
+      final String devices = "0,1";
+      statement.execute(String.format("LOAD MODEL %s TO DEVICES \"%s\"", modelId, devices));
+      checkModelOnSpecifiedDevice(statement, MODEL_ID_TO_TYPE_MAP.get(modelId), devices);
       concurrentInference(
           statement,
-          String.format("CALL INFERENCE(%s, \"SELECT s FROM root.AI\")", modelId),
+          String.format(
+              "CALL INFERENCE(%s, \"SELECT s FROM root.AI\", predict_length=%d)",
+              modelId, predictLength),
           threadCnt,
-          loop);
+          loop,
+          predictLength);
       statement.execute(String.format("UNLOAD MODEL %s FROM DEVICES \"0,1\"", modelId));
     }
   }
@@ -134,15 +158,18 @@ private void concurrentCPUForecastTest(String modelId) throws SQLException, Inte
         Statement statement = connection.createStatement()) {
       final int threadCnt = 4;
       final int loop = 10;
+      final int predictLength = 96;
       statement.execute(String.format("LOAD MODEL %s TO DEVICES \"cpu\"", modelId));
+      checkModelOnSpecifiedDevice(statement, MODEL_ID_TO_TYPE_MAP.get(modelId), "cpu");
       long startTime = System.currentTimeMillis();
       concurrentInference(
           statement,
           String.format(
-              "SELECT * FROM FORECAST(model_id=>'%s', input=>(SELECT time,s FROM root.AI) ORDER BY time)",
-              modelId),
+              "SELECT * FROM FORECAST(model_id=>'%s', input=>(SELECT time,s FROM root.AI) ORDER BY time), predict_length=>%d",
+              modelId, predictLength),
           threadCnt,
-          loop);
+          loop,
+          predictLength);
       long endTime = System.currentTimeMillis();
       LOGGER.info(
           String.format(
@@ -163,15 +190,19 @@ public void concurrentGPUForecastTest(String modelId) throws SQLException, Inter
         Statement statement = connection.createStatement()) {
       final int threadCnt = 10;
       final int loop = 100;
-      statement.execute(String.format("LOAD MODEL %s TO DEVICES \"0,1\"", modelId));
+      final int predictLength = 512;
+      final String devices = "0,1";
+      statement.execute(String.format("LOAD MODEL %s TO DEVICES \"%s\"", modelId, devices));
+      checkModelOnSpecifiedDevice(statement, MODEL_ID_TO_TYPE_MAP.get(modelId), devices);
       long startTime = System.currentTimeMillis();
       concurrentInference(
           statement,
           String.format(
-              "SELECT * FROM FORECAST(model_id=>'%s', input=>(SELECT time,s FROM root.AI) ORDER BY time)",
-              modelId),
+              "SELECT * FROM FORECAST(model_id=>'%s', input=>(SELECT time,s FROM root.AI) ORDER BY time), predict_length=>%d",
+              modelId, predictLength),
           threadCnt,
-          loop);
+          loop,
+          predictLength);
       long endTime = System.currentTimeMillis();
       LOGGER.info(
           String.format(
@@ -180,4 +211,29 @@ public void concurrentGPUForecastTest(String modelId) throws SQLException, Inter
       statement.execute(String.format("UNLOAD MODEL %s FROM DEVICES \"0,1\"", modelId));
     }
   }
+
+  private void checkModelOnSpecifiedDevice(Statement statement, String modelType, String device)
+      throws SQLException, InterruptedException {
+    for (int retry = 0; retry < 10; retry++) {
+      Set<String> targetDevices = ImmutableSet.copyOf(device.split(","));
+      Set<String> foundDevices = new HashSet<>();
+      try (final ResultSet resultSet =
+          statement.executeQuery(String.format("SHOW LOADED MODELS %s", device))) {
+        while (resultSet.next()) {
+          String deviceId = resultSet.getString(1);
+          String loadedModelType = resultSet.getString(2);
+          int count = resultSet.getInt(3);
+          if (loadedModelType.equals(modelType) && targetDevices.contains(deviceId)) {
+            Assert.assertTrue(count > 1);
+            foundDevices.add(deviceId);
+          }
+        }
+        if (foundDevices.containsAll(targetDevices)) {
+          return;
+        }
+      }
+      TimeUnit.SECONDS.sleep(3);
+    }
+    Assert.fail("Model " + modelType + " is not loaded on device " + device);
+  }
 }