Skip to content

Commit eaf2333

Browse files
alinaliBQjusting-bq
alinaliBQ
authored and
justing-bq
committed
ODBC Linux rpm installer support
* Implement RPM installer fix formatting Remove todos ODBC rpm Installer wrap up * installing `rpm` package fixed the `cpack` command issue * Fix cpack command on CI (merge into last commit when ready) Change to use `awk` so script works on both platforms Comment out `find` and `tree` steps Change build path to be under /arrow Since docker is mounted, changing build path to be in `/arrow` will allow host machine to access docker build contents TEMP - Disable non-ODBC items Add commands to find build folder Attempt to build RPM installer * not sure if `/build/cpp` can be accessed outside of docker. Hopefully it should. * add `rpm` dependency install Indicate RPM in workflow run Add rpm `postinstall` (not tested) Remove merge conflict code * Enable ODBC installer build in CI instead of `compose.yaml` Update workflow matrix Update cpp_extra.yml Fix package name
1 parent 46bb919 commit eaf2333

9 files changed

Lines changed: 118 additions & 22 deletions

File tree

‎.github/workflows/cpp_extra.yml‎

Lines changed: 22 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -372,7 +372,7 @@ jobs:
372372

373373
odbc-linux:
374374
needs: check-labels
375-
name: ODBC Linux
375+
name: ODBC ${{ matrix.build-type }} ${{ matrix.title }}
376376
runs-on: ubuntu-latest
377377
if: >-
378378
needs.check-labels.outputs.force == 'true' ||
@@ -381,6 +381,15 @@ jobs:
381381
timeout-minutes: 75
382382
strategy:
383383
fail-fast: false
384+
matrix:
385+
include:
386+
- image: ubuntu-cpp-odbc
387+
title: AMD64 Ubuntu RPM
388+
build-type: release
389+
format: rpm
390+
run-options: >-
391+
-e ARROW_FLIGHT_SQL_ODBC_INSTALLER=ON
392+
-e ODBC_PACKAGE_FORMAT=RPM
384393
env:
385394
ARCH: amd64
386395
ARCHERY_DEBUG: 1
@@ -408,7 +417,8 @@ jobs:
408417
uses: apache/infrastructure-actions/stash/restore@0ba14156c9f4c3cfbe4b0c9f36339ab0f8d81e53
409418
with:
410419
path: .docker
411-
key: ubuntu-cpp-odbc
420+
key: ${{ matrix.image }}-${{ matrix.format }}-${{ hashFiles('cpp/**') }}
421+
restore-keys: ${{ matrix.image }}-${{ matrix.format }}-
412422
- name: Execute Docker Build
413423
env:
414424
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
@@ -417,14 +427,21 @@ jobs:
417427
# GH-40558: reduce ASLR to avoid ASAN/LSAN crashes
418428
sudo sysctl -w vm.mmap_rnd_bits=28
419429
source ci/scripts/util_enable_core_dumps.sh
420-
archery docker run ubuntu-cpp-odbc
430+
archery docker run ${{ matrix.run-options || '' }} ${{ matrix.image }}
431+
- name: Upload ODBC ${{ matrix.format }} to the job
432+
if: matrix.build-type == 'release'
433+
uses: actions/upload-artifact@v7
434+
with:
435+
name: flight-sql-odbc-${{ matrix.format }}-installer
436+
path: build/cpp/ArrowFlightSQLODBC-*.${{ matrix.format }}
437+
if-no-files-found: error
421438
- name: Save Docker Volumes
422439
if: ${{ !cancelled() }}
423440
continue-on-error: true
424441
uses: apache/infrastructure-actions/stash/save@0ba14156c9f4c3cfbe4b0c9f36339ab0f8d81e53
425442
with:
426443
path: .docker
427-
key: ubuntu-cpp-odbc
444+
key: ${{ matrix.image }}-${{ matrix.format }}-${{ hashFiles('cpp/**') }}
428445
include-hidden-files: true
429446
- name: Docker Push
430447
if: >-
@@ -436,7 +453,7 @@ jobs:
436453
ARCHERY_DOCKER_USER: ${{ secrets.DOCKERHUB_USER }}
437454
ARCHERY_DOCKER_PASSWORD: ${{ secrets.DOCKERHUB_TOKEN }}
438455
continue-on-error: true
439-
run: archery docker push ubuntu-cpp-odbc
456+
run: archery docker push ${{ matrix.image }}
440457

441458
odbc-macos:
442459
needs: check-labels

‎.pre-commit-config.yaml‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -347,6 +347,7 @@ repos:
347347
?^cpp/examples/minimal_build/run\.sh$|
348348
?^cpp/examples/tutorial_examples/run\.sh$|
349349
?^cpp/src/arrow/flight/sql/odbc/install/mac/postinstall$|
350+
?^cpp/src/arrow/flight/sql/odbc/install/linux/rpm/postinstall$|
350351
?^cpp/src/arrow/flight/sql/odbc/install/unix/install_odbc\.sh$|
351352
?^cpp/src/arrow/flight/sql/odbc/install/unix/install_odbc_ini\.sh$|
352353
?^cpp/src/arrow/flight/sql/odbc/tests/dremio/set_up_dremio_instance\.sh$|

‎ci/docker/ubuntu-24.04-cpp.dockerfile‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -119,7 +119,9 @@ RUN apt-get update -y -q && \
119119
python3-venv \
120120
rados-objclass-dev \
121121
rapidjson-dev \
122+
rpm \
122123
rsync \
124+
sudo \
123125
tzdata \
124126
tzdata-legacy \
125127
unixodbc-dev \

‎ci/scripts/cpp_build.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -274,6 +274,7 @@ else
274274
-Dlz4_SOURCE=${lz4_SOURCE:-} \
275275
-Dnlohmann_json_SOURCE=${nlohmann_json_SOURCE:-} \
276276
-Dopentelemetry-cpp_SOURCE=${opentelemetry_cpp_SOURCE:-} \
277+
-DODBC_PACKAGE_FORMAT=${ODBC_PACKAGE_FORMAT:-} \
277278
-DORC_SOURCE=${ORC_SOURCE:-} \
278279
-DPARQUET_BUILD_EXAMPLES=${PARQUET_BUILD_EXAMPLES:-OFF} \
279280
-DPARQUET_BUILD_EXECUTABLES=${PARQUET_BUILD_EXECUTABLES:-OFF} \

‎compose.yaml‎

Lines changed: 5 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -534,15 +534,18 @@ services:
534534
ARROW_PARQUET: "OFF"
535535
ARROW_S3: "OFF"
536536
ARROW_SUBSTRAIT: "OFF"
537+
# Use `/arrow/build` so build artifacts are visible on the CI host
538+
# Generate ODBC installer before testing
537539
# Register ODBC before running tests
538540
depends_on:
539541
dremio:
540542
condition: service_healthy
541543
command: >
542544
/bin/bash -c "
543-
/arrow/ci/scripts/cpp_build.sh /arrow /build &&
545+
/arrow/ci/scripts/cpp_build.sh /arrow /arrow/build &&
546+
(cd /arrow/build/cpp && cpack) &&
544547
sudo /arrow/cpp/src/arrow/flight/sql/odbc/install/unix/install_odbc.sh /usr/local/lib/libarrow_flight_sql_odbc.so &&
545-
/arrow/ci/scripts/cpp_test.sh /arrow /build"
548+
/arrow/ci/scripts/cpp_test.sh /arrow /arrow/build"
546549
547550
ubuntu-cpp-minimal:
548551
# Arrow build with minimal components/dependencies

‎cpp/cmake_modules/DefineOptions.cmake‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -329,12 +329,18 @@ takes precedence over ccache if a storage backend is configured" ON)
329329
ARROW_FLIGHT)
330330

331331
define_option(ARROW_FLIGHT_SQL_ODBC
332-
"Build the Arrow Flight SQL ODBC extension"
332+
"Build the Arrow Flight SQL ODBC driver"
333333
OFF
334334
DEPENDS
335335
ARROW_FLIGHT_SQL
336336
ARROW_COMPUTE)
337337

338+
define_option(ARROW_FLIGHT_SQL_ODBC_INSTALLER
339+
"Build the installer for the Arrow Flight SQL ODBC driver"
340+
OFF
341+
DEPENDS
342+
ARROW_FLIGHT_SQL_ODBC)
343+
338344
define_option(ARROW_GANDIVA
339345
"Build the Gandiva libraries"
340346
OFF

‎cpp/src/arrow/flight/sql/odbc/CMakeLists.txt‎

Lines changed: 34 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -154,11 +154,13 @@ if(ARROW_FLIGHT_SQL_ODBC_INSTALLER)
154154

155155
set(CPACK_WIX_UI_BANNER
156156
"${CMAKE_CURRENT_SOURCE_DIR}/install/windows/arrow-wix-banner.bmp")
157+
set(ODBC_DOC_INSTALL_DIR "doc")
157158
else()
159+
set(ODBC_UNIX_FILE_NAME
160+
"ArrowFlightSQLODBC-${CPACK_PACKAGE_VERSION_MAJOR}.${ODBC_PACKAGE_VERSION_MINOR}.${ODBC_PACKAGE_VERSION_PATCH}"
161+
)
158162
if(APPLE)
159-
set(CPACK_PACKAGE_FILE_NAME
160-
"ArrowFlightSQLODBC-${CPACK_PACKAGE_VERSION_MAJOR}.${ODBC_PACKAGE_VERSION_MINOR}.${ODBC_PACKAGE_VERSION_PATCH}"
161-
)
163+
set(CPACK_PACKAGE_FILE_NAME "${ODBC_UNIX_FILE_NAME}")
162164
set(CPACK_PACKAGE_INSTALL_DIRECTORY "${CPACK_PACKAGE_NAME}")
163165

164166
set(CPACK_SET_DESTDIR ON)
@@ -174,10 +176,28 @@ if(ARROW_FLIGHT_SQL_ODBC_INSTALLER)
174176
set(ODBC_DOC_INSTALL_DIR "arrow-odbc/doc")
175177
else()
176178
# Linux
177-
# GH-49595: TODO implement DEB installer
178-
# GH-47977: TODO implement RPM installer
179-
message(STATUS "ODBC_PACKAGE_FORMAT DEB not implemented, see GH-49595")
180-
message(STATUS "ODBC_PACKAGE_FORMAT RPM not implemented, see GH-47977")
179+
set(CPACK_PACKAGE_INSTALL_DIRECTORY "${CPACK_PACKAGE_NAME}")
180+
if(${ODBC_PACKAGE_FORMAT} STREQUAL "DEB")
181+
# GH-49595 TODO: implement DEB installer
182+
message(STATUS "ODBC_PACKAGE_FORMAT DEB not implemented, see GH-49595")
183+
elseif(${ODBC_PACKAGE_FORMAT} STREQUAL "RPM")
184+
set(CPACK_RPM_PACKAGE_ARCHITECTURE x86_64)
185+
set(CPACK_GENERATOR RPM)
186+
set(CPACK_RPM_POST_INSTALL_SCRIPT_FILE
187+
"${CMAKE_CURRENT_SOURCE_DIR}/install/linux/postinst")
188+
set(CPACK_RPM_FILE_NAME "${ODBC_UNIX_FILE_NAME}.rpm")
189+
# Disable dependency check as ODBC embeds all third party dependencies
190+
set(CPACK_RPM_PACKAGE_AUTOREQPROV "no")
191+
set(CPACK_COMPONENTS_ALL_IN_ONE_PACKAGE 1)
192+
set(CPACK_RPM_COMPONENT_INSTALL ON)
193+
else()
194+
message(FATAL_ERROR "ODBC_PACKAGE_FORMAT '${ODBC_PACKAGE_FORMAT}' must be DEB or RPM for Linux installer."
195+
)
196+
endif()
197+
198+
# By default, Linux installs under /usr
199+
set(ODBC_INSTALL_DIR "lib64/arrow-odbc/lib")
200+
set(ODBC_DOC_INSTALL_DIR "lib64/arrow-odbc/doc")
181201
endif()
182202

183203
# Install ODBC
@@ -206,15 +226,15 @@ if(ARROW_FLIGHT_SQL_ODBC_INSTALLER)
206226
GROUP_READ
207227
WORLD_EXECUTE
208228
WORLD_READ)
209-
210-
# Install documentation files
211-
install(FILES "${CMAKE_CURRENT_SOURCE_DIR}/../../../../../../LICENSE.txt"
212-
"${CMAKE_CURRENT_SOURCE_DIR}/../../../../../../NOTICE.txt"
213-
"${CMAKE_CURRENT_SOURCE_DIR}/connection-options.md"
214-
DESTINATION "${ODBC_DOC_INSTALL_DIR}"
215-
COMPONENT Docs)
216229
endif()
217230

231+
# Install documentation files
232+
install(FILES "${CMAKE_CURRENT_SOURCE_DIR}/../../../../../../LICENSE.txt"
233+
"${CMAKE_CURRENT_SOURCE_DIR}/../../../../../../NOTICE.txt"
234+
"${CMAKE_CURRENT_SOURCE_DIR}/connection-options.md"
235+
DESTINATION "${ODBC_DOC_INSTALL_DIR}"
236+
COMPONENT Docs)
237+
218238
get_cmake_property(CPACK_COMPONENTS_ALL COMPONENTS)
219239
set(CPACK_COMPONENTS_ALL "ArrowFlightSQLODBC" "Docs")
220240

‎cpp/src/arrow/flight/sql/odbc/README.md‎

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,22 @@ After ODBC has been registered, you can run the ODBC tests. It is recommended to
144144
.\cpp\build\< release | debug >\< Release | Debug>\arrow-flight-sql-odbc-test.exe
145145
```
146146
147+
## Installers
148+
149+
ODBC installers are uploaded to the CI artifacts.
150+
151+
| Operating System | Package Format |
152+
|------------------|----------------|
153+
| Windows | MSI |
154+
| macOS | PKG |
155+
| Linux | DEB / RPM |
156+
157+
### Install `.RPM` on Ubuntu
158+
While installing via `.DEB` installer on Ubuntu is the recommended approach, users may install `.RPM` on Ubuntu using below command
159+
```
160+
alien -i --scripts ArrowFlightSQLODBC-<version>.rpm
161+
```
162+
147163
## Known Limitations
148164
149165
- Conversion from timestamp data type with specified time zone value to strings is not supported at the moment. This doesn't impact driver's usage of retrieving timestamp data from Power BI on Windows, and Excel on macOS and Windows. See GH-47504 for more context.

‎cpp/src/arrow/flight/sql/odbc/install/linux/postinst‎

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
#!/bin/bash
2+
#
3+
# Licensed to the Apache Software Foundation (ASF) under one
4+
# or more contributor license agreements. See the NOTICE file
5+
# distributed with this work for additional information
6+
# regarding copyright ownership. The ASF licenses this file
7+
# to you under the Apache License, Version 2.0 (the
8+
# "License"); you may not use this file except in compliance
9+
# with the License. You may obtain a copy of the License at
10+
#
11+
# http://www.apache.org/licenses/LICENSE-2.0
12+
#
13+
# Unless required by applicable law or agreed to in writing,
14+
# software distributed under the License is distributed on an
15+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
16+
# KIND, either express or implied. See the License for the
17+
# specific language governing permissions and limitations
18+
# under the License.
19+
20+
# Use temporary driver registration script to register ODBC driver in system DSN
21+
odbc_install_script="/usr/lib64/arrow-odbc/lib/install_odbc.sh"
22+
"$odbc_install_script" /usr/lib64/arrow-odbc/lib/libarrow_flight_sql_odbc.so
23+
24+
# Use temporary DSN registration script to register sample system DSN
25+
dsn_install_script="/usr/lib64/arrow-odbc/lib/install_odbc_ini.sh"
26+
"$dsn_install_script" /etc/odbc.ini
27+
28+
# clean temporary script
29+
rm -f "$odbc_install_script"
30+
rm -f "$dsn_install_script"

0 commit comments

Comments
 (0)