Skip to content

Commit faba002

Browse files
authored
Refactor/cpp client sdk (#17801)
* Refactor client-cpp build to single CMake entry with portable third-party cache Move C++ client dependency resolution and compilation into one top-level CMakeLists with Fetch* modules, cache tarballs under third-party/<os>/ for offline copy-the-repo workflows, and slim the Maven POM to a CMake wrapper. Skip server CI when only client-cpp changes; gate multi-language-client jobs by language via paths-filter. * Fix macOS client-cpp CI bison version and add source-build workflow Install Homebrew bison on macOS package/IT runners so Thrift 0.21.0 builds (%code requires Bison 2.4+). Add client-cpp-source-build workflow that uses a minimal toolchain and online CMake fetch to verify zero-to-build compilation. * Fix client-cpp-source-build: install clang-format-17 on all runners Align Linux/macOS/Windows toolchain setup with client-cpp-package so Spotless clang-format check during mvn package succeeds. * Skip Spotless in client-cpp-source-build; drop clang-format install Source-build workflow focuses on minimal-toolchain CMake fetch; pass -Dspotless.skip=true instead of installing clang-format on runners. * Refactor C++ client SDK: isolate Thrift and ship Windows DLL. Reorganize sources into include/session/rpc with PIMPL so public headers no longer pull in Thrift or Boost. Embed Thrift in iotdb_session on all platforms; on Windows build a /MD shared library with import lib. Update examples, CI verification, and documentation for the slimmer SDK layout. * fix(client-cpp): include ctime for std::tm in Common.h Fixes Linux CI build where convertToTimestamp/int32ToDate declarations require std::tm from <ctime> (C++11). * fix(client-cpp): CMake 4 Thrift policy and Session.cpp cstring Pass CMAKE_POLICY_VERSION_MINIMUM=3.5 when configuring Thrift 0.21 on CMake 4.x (VS2026 CI). Include <cstring> in Session.cpp for memcpy/strlen/strstr on strict Linux builds. * fix(client-cpp): Linux Thrift link and default Boost 1.84 Drop --whole-archive on Linux to avoid libgcc morestack duplicate symbols; build Thrift with -fno-split-stack. Bump default Boost to 1.84.0 for modern Clang on macOS CI. * fix(client-cpp): Linux Thrift link and macOS Boost 1.84 Linux: keep --whole-archive for Thrift in libiotdb_session.so but add --allow-multiple-definition to avoid libgcc morestack duplicate symbols. macOS: default BOOST_VERSION to 1.84.0 for modern Clang enum checks (other platforms stay on 1.60.0). * ci(client-cpp): multi-toolchain package matrix for release zips Add per-classifier SDK packages (glibc 2.17 on CentOS 7, Windows VS2015-2026), workflow_dispatch variant filter, and documentation for choosing the right zip. * fix(ci): write windows_matrix to GITHUB_OUTPUT without invalid format * fix(ci): scope C++ triggers, glibc231 arm64, CentOS7 and VS2015 boost - paths-ignore C++ workflows/scripts on Java IT jobs; narrow C++ path filters (no root pom.xml) - aarch64 package on Ubuntu 20.04 container (linux-aarch64-glibc231) - CentOS7 script: vault repos, Adoptium API JDK, devtoolset deps - VS2015/2017: pin boost-msvc-14.2/14.1 versions instead of missing boost-msvc-14.0 * fix(ci): docker-based glibc builds, CMake 3.28 on arm64, drop VS2015 - glibc217/aarch64: checkout on host, build inside docker (fixes CentOS7 + Node 24) - aarch64: install Kitware CMake 3.28 (ARCHIVE_EXTRACT needs 3.18+) - Remove VS2015 package matrix and related docs * fix(client-cpp): flatten package zip name and layout - Zip: client-cpp-<version>-<classifier>.zip (remove redundant -cpp- segment) - Root folder client-cpp-<version>-<classifier>/ with include/ and lib/ inside - Update examples (strip one dir on unpack), distribution assembly, CI artifact paths - CentOS7 SCLo vault mirror fix; chown workspace after docker builds * fix(ci): pin CentOS7 SCLo repos to vault 7.9.2009 for glibc217 centos-sclo-sclo baseurl with releasever=7 404s on vault; rewrite SCLo repo files and refresh yum cache before devtoolset-8 install. * fix(ci): build glibc217 in manylinux2014 job container Replace docker run on ubuntu-22.04 with job-level container quay.io/pypa/manylinux2014_x86_64 and checkout@v4, matching PyPA manylinux CI style. * fix(ci): host checkout + docker manylinux2014 for glibc217 Job container cannot run Node-based actions on glibc 2.17. Run checkout/cache on ubuntu-latest and build inside manylinux2014 via docker run; use preinstalled devtoolset-10 in the image. * fix(ci): avoid SIGPIPE 141 from head/tail under pipefail Replace ldd | head -1 with sed -n 1p and find | head -1 with find -quit so glibc checks do not fail after a successful build. * fix(client-cpp): force x64 for Visual Studio Windows packages VS2017 generator defaults to Win32; pass -DCMAKE_GENERATOR_PLATFORM=x64 on Windows via Maven profile. Add CI PE check that iotdb_session.dll is x64. * fix(ci): build linux-aarch64 glibc217 with manylinux2014 Use manylinux2014_aarch64 and the shared packaging script instead of Ubuntu 20.04 glibc231. Extend the script for aarch64 CMake/JDK arch and classifier auto-detection. Update docs and remove the obsolete ubuntu20-arm script. * fix(client-cpp): PR review follow-ups for SDK branch Guard checkTemplateExists against a null dataset, document DataIterator lifetime, correct README Maven vs CMake option names, drop unused IotdbResolveTarball.cmake, and skip server CI when only multi-language-client workflow changes. * chore(ci): slim manylinux glibc217 package path Rename the Linux packaging helper to a manylinux-specific script, drop unused CentOS7 fallback branches, and keep only build plus GLIBC<=2.17 verification steps. Also add concise C++ workflow comments and unify client-cpp CMake minimum version to 3.15. * refactor(client-cpp): rename IoTDBDate and move tests Rename public Date type from IoTdbDate to IoTDBDate across headers, RPC/session code, and ITs. Move tests from src/test to a top-level test directory and update CMake, Maven, and README paths accordingly. * example: merge C Session demos into client-cpp-example Move tree_example.c and table_example.c into client-cpp-example, build them from the shared CMake entry, and update EN/ZH docs. Remove the standalone client-c-example module and drop it from the with-cpp profile. * style(client-cpp): apply spotless to C examples * fix(ci): use curl for downloads in manylinux glibc217 script * fix(ci): use curl directly in manylinux glibc217 script * fix(client-cpp): centralize SessionConfig defaults and align fetchSize with Java * ci(client-cpp): add glibc224 packages with explicit CXX11 ABI on manylinux_2_24 Add linux-*-glibc224 zips built on manylinux_2_24 with -D_GLIBCXX_USE_CXX11_ABI=1 wired through Session and Thrift. Document glibc224 as recommended over glibc217 for modern distros. * fix(client-cpp): correct glibc224 CXX11 ABI check and Maven flag wiring * fix(client-cpp): use CMake ABI tag file instead of nm for glibc224 CI * ci(client-cpp): remove source-build workflow Drop client-cpp-source-build.yml; packaging is covered by client-cpp-package.yml. * fix(ci): harden glibc checks and drop removed source-build workflow refs Fail manylinux package scripts when GLIBC symbols cannot be parsed, and remove stale client-cpp-source-build.yml path triggers left after that workflow was deleted. * fix(client-cpp): flatten package zip layout and add SHA512 checksums Unpack directly to include/ and lib/ at zip root, generate .sha512 via checksum-maven-plugin, and upload checksums from the package workflow. * fix sessionIT.cpp compilation * fix(client-cpp): download Catch2 at build time instead of vendoring Maven wget in generate-test-resources; CMake file(DOWNLOAD) fallback; remove test/catch2/catch.hpp from source tree. * fix(client-cpp): download Catch2 before CMake compile phase Run wget in generate-resources (not generate-test-resources, which runs after compile). CMake also downloads catch.hpp when the header is still missing. * refactor(client-cpp): move SessionPool implementation into session sources Place SessionPool.cpp under src/session to align with the existing source layout while keeping public headers in src/include. * refactor(client-cpp): ship SDK tarballs with cmake/pkgconfig and bundled examples Rename packages to iotdb-session-cpp-* classifiers, move examples into client-cpp, drop glibc2.24 release matrix, and add CI smoke tests that build and run examples against a standalone IoTDB server. * fix(ci): normalize RUNNER_TEMP for Windows tar extract in cpp package smoke test Git Bash on Windows runners passes D:\a\_temp to tar -C, which MSYS tar cannot open; convert via cygpath -u before unpack and example build paths. * chore(client-cpp): trim examples README and formatDatetime dead code Simplify the SDK layout note in examples README and drop unused-parameter casts for precision/zoneId in formatDatetime. * fix(ci): locate Windows standalone scripts under sbin/windows in cpp smoke test Distribution packages start/stop batch files at sbin/windows/, matching client-cpp pom.xml; normalize GITHUB_WORKSPACE via cygpath for find on Windows. * fix(ci): start IoTDB on Windows with cmd //c and wait for RPC port 6667 Git Bash mangles cmd.exe /c so start-standalone.bat never ran; use //c with quoted paths and poll Test-NetConnection before running packaged examples. * fix(ci): upload cpp SDK tarball before packaged example smoke tests Resolve and upload artifacts on macOS and Windows immediately after packaging so failed verify steps still retain the tarball; reuse pkg path in verify step. * refactor(client-cpp): use zip for SDK release packages instead of tar.gz Align Maven assembly, distribution bundle, CI resolve/upload, smoke-test unpack (unzip), and docs on iotdb-session-cpp-<version>-<classifier>.zip. * fix(ci): correct Git Bash quoting when starting IoTDB on Windows Nested escaped quotes made cmd.exe treat the bat path as the command name; use //c with a single quoted path. Add a local script to replay verify steps. * fix(ci): build packaged examples as x64 for VS2017 on Windows Visual Studio 15/16 generators default to Win32 while the SDK is x64, causing LNK1112. Pass -A x64 in the verify step and set CMAKE_GENERATOR_PLATFORM in examples CMakeLists. * Improve C++ session package documentation * Set CXX11 ABI for glibc217 C++ package * Fix C++ SDK CXX11 ABI propagation * Fix C++ SDK package ABI verification * Fix glibc217 C++ client ABI build * Use link test for glibc217 C++ ABI check * Rebuild C++ client when ABI changes * Fail C++ client build on ABI mismatch * Use old libstdc++ ABI for glibc217 package * Use manylinux_2_28 for C++ client packages * chore(ci): remove unused local Windows C++ example debug scripts These scripts were one-off helpers for VS2017 x64 CI fixes; verification logic already lives inline in client-cpp-package.yml. * Fix MSVC debug C++ client build * Document C++ client debug builds * Address C++ client packaging review comments
1 parent 3d98ea5 commit faba002

109 files changed

Lines changed: 6269 additions & 20613 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 140 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,140 @@
1+
#!/usr/bin/env bash
2+
# Licensed to the Apache Software Foundation (ASF) under one or more
3+
# contributor license agreements. See the NOTICE file distributed with this
4+
# work for additional information regarding copyright ownership. The ASF
5+
# licenses this file to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance with the
7+
# License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing, software
12+
# distributed under the License is distributed on an "AS IS" BASIS,
13+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
14+
# See the License for the specific language governing permissions and
15+
# limitations under the License.
16+
#
17+
# Build client-cpp in manylinux_2_28 and verify max required GLIBC symbol <= 2.28.
18+
set -euxo pipefail
19+
20+
MACHINE=$(uname -m)
21+
case "${MACHINE}" in
22+
x86_64)
23+
CMAKE_PKG_ARCH=linux-x86_64
24+
JDK_API_ARCH=linux/x64
25+
DEFAULT_CLASSIFIER=linux-x86_64-glibc2.28
26+
;;
27+
aarch64)
28+
CMAKE_PKG_ARCH=linux-aarch64
29+
JDK_API_ARCH=linux/aarch64
30+
DEFAULT_CLASSIFIER=linux-aarch64-glibc2.28
31+
;;
32+
*)
33+
echo "Unsupported architecture: ${MACHINE}" >&2
34+
exit 1
35+
;;
36+
esac
37+
38+
PACKAGE_CLASSIFIER="${PACKAGE_CLASSIFIER:-${DEFAULT_CLASSIFIER}}"
39+
40+
CMAKE_VERSION=3.28.4
41+
CMAKE_DIR="/opt/cmake-${CMAKE_VERSION}"
42+
if [[ ! -x "${CMAKE_DIR}/bin/cmake" ]]; then
43+
curl -fsSL -o /tmp/cmake.tar.gz "https://github.com/Kitware/CMake/releases/download/v${CMAKE_VERSION}/cmake-${CMAKE_VERSION}-${CMAKE_PKG_ARCH}.tar.gz"
44+
rm -rf "${CMAKE_DIR}"
45+
mkdir -p /opt
46+
tar xf /tmp/cmake.tar.gz -C /opt
47+
mv "/opt/cmake-${CMAKE_VERSION}-${CMAKE_PKG_ARCH}" "${CMAKE_DIR}"
48+
fi
49+
50+
JAVA_HOME=/opt/jdk-17
51+
if [[ ! -x "${JAVA_HOME}/bin/java" ]]; then
52+
curl -fsSL -o /tmp/jdk17.tar.gz "https://api.adoptium.net/v3/binary/latest/17/ga/${JDK_API_ARCH}/jdk/hotspot/normal/eclipse?project=jdk"
53+
rm -rf /opt/jdk-17*
54+
mkdir -p /opt
55+
tar xf /tmp/jdk17.tar.gz -C /opt
56+
JAVA_HOME=$(find /opt -maxdepth 1 -type d -name 'jdk-17*' -print -quit)
57+
ln -sfn "${JAVA_HOME}" /opt/jdk-17
58+
JAVA_HOME=/opt/jdk-17
59+
fi
60+
61+
export PATH="${CMAKE_DIR}/bin:${JAVA_HOME}/bin:${PATH}"
62+
export JAVA_HOME
63+
64+
gcc --version
65+
c++ --version
66+
gcc_major=$(gcc -dumpversion | cut -d. -f1)
67+
if (( gcc_major < 14 )); then
68+
echo "ERROR: GCC >= 14 is required; got $(gcc -dumpversion)"
69+
exit 1
70+
fi
71+
cmake --version
72+
java -version
73+
74+
cd "${GITHUB_WORKSPACE:?GITHUB_WORKSPACE is not set}"
75+
./mvnw clean package -P with-cpp -pl iotdb-client/client-cpp -am -DskipTests \
76+
-Dspotless.skip=true \
77+
-Dclient.cpp.package.classifier="${PACKAGE_CLASSIFIER}"
78+
79+
SO="iotdb-client/client-cpp/target/install/lib/libiotdb_session.so"
80+
test -f "${SO}"
81+
82+
echo "=== Build host glibc ==="
83+
ldd --version 2>&1 | sed -n '1p'
84+
85+
echo "=== Highest GLIBC_* symbols in libiotdb_session.so ==="
86+
objdump -T "${SO}" | grep GLIBC_ | sed "s/.*GLIBC_/GLIBC_/" | sort -Vu | tail -10
87+
88+
max_glibc=$(objdump -T "${SO}" | grep -oE "GLIBC_[0-9.]+" | sed "s/GLIBC_//" | sort -t. -k1,1n -k2,2n -k3,3n | tail -1)
89+
echo "max_glibc=${max_glibc}"
90+
91+
if [[ -z "${max_glibc}" ]]; then
92+
echo "ERROR: could not determine max GLIBC version from ${SO}"
93+
exit 1
94+
fi
95+
96+
if awk -v max="${max_glibc}" "BEGIN { exit !(max > 2.28) }"; then
97+
echo "ERROR: libiotdb_session.so requires glibc > 2.28 (max=${max_glibc})"
98+
exit 1
99+
fi
100+
101+
echo "glibc compatibility check passed (max=${max_glibc} <= 2.28)"
102+
103+
echo "=== Example package build/link/run smoke test ==="
104+
PKG_ZIP=$(find "${GITHUB_WORKSPACE}/iotdb-client/client-cpp/target" -maxdepth 1 -type f -name "iotdb-session-cpp-*-${PACKAGE_CLASSIFIER}.zip" -print -quit)
105+
if [[ -z "${PKG_ZIP}" ]]; then
106+
echo "ERROR: could not find package zip for ${PACKAGE_CLASSIFIER}"
107+
exit 1
108+
fi
109+
PKG_UNPACK="/tmp/client-cpp-package-smoke-glibc228"
110+
rm -rf "${PKG_UNPACK}"
111+
mkdir -p "${PKG_UNPACK}"
112+
unzip -q -o "${PKG_ZIP}" -d "${PKG_UNPACK}"
113+
PKG_ROOT=$(find "${PKG_UNPACK}" -mindepth 1 -maxdepth 1 -type d -name "iotdb-session-cpp-*-${PACKAGE_CLASSIFIER}" -print -quit)
114+
if [[ -z "${PKG_ROOT}" ]]; then
115+
echo "ERROR: could not find unpacked package directory for ${PACKAGE_CLASSIFIER}"
116+
exit 1
117+
fi
118+
EXAMPLE_BUILD="/tmp/client-cpp-example-smoke-glibc228"
119+
rm -rf "${EXAMPLE_BUILD}"
120+
mkdir -p "${EXAMPLE_BUILD}"
121+
unset CC CXX CFLAGS CXXFLAGS
122+
"${CMAKE_DIR}/bin/cmake" -S "${PKG_ROOT}/examples" -B "${EXAMPLE_BUILD}" \
123+
-DCMAKE_BUILD_TYPE=Release \
124+
-DIOTDB_SDK_ROOT="${PKG_ROOT}"
125+
"${CMAKE_DIR}/bin/cmake" --build "${EXAMPLE_BUILD}" -j"$(nproc)"
126+
127+
./mvnw -pl distribution -am -DskipTests -Dspotless.skip=true package
128+
SERVER_ROOT=$(find "${GITHUB_WORKSPACE}/distribution/target" -path '*/apache-iotdb-*-all-bin/sbin/start-standalone.sh' -print -quit | sed 's#/sbin/start-standalone.sh##')
129+
if [[ -z "${SERVER_ROOT}" ]]; then
130+
echo "ERROR: could not find IoTDB distribution under distribution/target"
131+
exit 1
132+
fi
133+
"${SERVER_ROOT}/sbin/start-standalone.sh"
134+
trap '"${SERVER_ROOT}/sbin/stop-standalone.sh" || true' EXIT
135+
sleep 30
136+
for example in SessionExample AlignedTimeseriesSessionExample TableModelSessionExample tree_example table_example; do
137+
test -x "${EXAMPLE_BUILD}/${example}"
138+
"${EXAMPLE_BUILD}/${example}"
139+
done
140+
echo "Example package smoke test passed"

0 commit comments

Comments
 (0)