Make Dev Container build successful#306
Conversation
Because `jupyter` not found on Dockerfile.
Unable to `quarto convert` because the kernel name `ruby3` not `ruby`: ```console $ rake jupyter quarto convert doc/qmd/preprocess_knock_Ruby-RedAmber.qmd -o doc/notebook/preprocess_knock_Ruby-RedAmber.ipynb ERROR: Jupyter kernel 'ruby' not found. Known kernels: python3, ruby3. Run 'quarto check jupyter' with your python environment activated to check python version used. ``` Since SciRuby/iruby#317, IRuby's default kernel name has been changed to `ruby3` from `ruby`. However, we keep using `ruby` to respect Ruby's strong compatibility. Alternatively, we can modify the jupyter engine setting in qmd files from `ruby` to `ruby3`: ```yaml jupyter: ruby3 ``` However, this approach requires updating all qmd files whenever Ruby's major version changes. We don't want to pay that maintenance cost.
|
Oh... unable to
On my macOS, $ bundle exec rake test
Loaded suite /home/vscode/.rbenv/versions/3.3.0/lib/ruby/gems/3.3.0/gems/rake-13.2.1/lib/rake/rake_test_loader
Started
Finished in 20.632629204 seconds.
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
873 tests, 2183 assertions, 0 failures, 0 errors, 0 pendings, 0 omissions, 0 notifications
100% passed
-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
42.31 tests/s, 105.80 assertions/s
Coverage report generated for Unit Tests to /workspaces/red_amber/coverage.
Line Coverage: 100.0% (5638 / 5638)
Coverage report generated for Unit Tests to /workspaces/red_amber/coverage/coverage.json. 5638 / 5638 LOC (100.0%) covered.Let me investigate. |
We can get the source line from this. See also: https://www.clear-code.com/blog/2024/12/25/addr2line-and-separated-debug-information.html We can use |
|
Great Post! I would like to investigate it. |
|
We can install Steps to reproduce (debian:bookworm)We can use FROM debian:bookworm
RUN apt-get update && \
apt-get install -y -V ca-certificates lsb-release wget
RUN wget https://packages.apache.org/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb && \
apt-get install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb && \
apt-get update
RUN apt-get install -y libarrow2000-dbgsym
# To use readelf
RUN apt-get install -y binutils$ docker build -t arrow-dbgsym-bookworm .
$ docker run --rm -it arrow-dbgsym-bookworm bash
root@2f665328cbd9:/# readelf --notes /usr/lib/x86_64-linux-gnu/libarrow.so.2000
Displaying notes found in: .note.gnu.build-id
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: 10799a3aa97dd9625614cb8aaee714127366210f
root@2f665328cbd9:/# addr2line --exe=/usr/lib/debug/.build-id/10/799a3aa97dd9625614cb8aaee714127366210f.debug +0x10
??:0Steps to reproduce (ubuntu:noble)Occurred FROM ubuntu:noble
RUN apt-get update && \
apt-get install -y -V ca-certificates lsb-release wget
RUN wget https://packages.apache.org/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb && \
apt-get install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb && \
apt-get update
RUN apt-get install -y libarrow2000-dbgsym
# To use readelf
RUN apt-get install -y binutils$ docker build -t arrow-dbgsym-noble .
[+] Building 24.9s (7/8) docker:desktop-linux
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 573B 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:noble 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> CACHED [1/5] FROM docker.io/library/ubuntu:noble 0.0s
=> [2/5] RUN apt-get update && apt-get install -y -V ca-certificates lsb-release wget 13.1s
=> [3/5] RUN wget https://packages.apache.org/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_rele 10.1s
=> ERROR [4/5] RUN apt-get install -y libarrow2000-dbgsym 1.6s
------
> [4/5] RUN apt-get install -y libarrow2000-dbgsym:
0.190 Reading package lists...
1.347 Building dependency tree...
1.547 Reading state information...
1.575 E: Unable to locate package libarrow2000-dbgsym
------
Dockerfile:10
--------------------
8 | apt-get update
9 |
10 | >>> RUN apt-get install -y libarrow2000-dbgsym
11 |
12 | # To use readelf
--------------------
ERROR: failed to solve: process "/bin/sh -c apt-get install -y libarrow2000-dbgsym" did not complete successfully: exit code: 100
View build details: docker-desktop://dashboard/build/desktop-linux/desktop-linux/ubyck92ubstfd7rs7zoh49ecq |
|
Ah, Could you open it in apache/arrow? Could you install it directly for now? wget https://packages.apache.org/artifactory/arrow/ubuntu/pool/noble/main/a/apache-arrow/libarrow2000-dbgsym_20.0.0-1_amd64.ddeb
apt install -y ./libarrow2000-dbgsym_20.0.0-1_amd64.ddeb
|
|
Thanks for your polite advise!
Sure!
Yes, we can install it directly. But Steps to reproduce: FROM ubuntu:noble
RUN apt-get update && \
apt-get install -y -V ca-certificates lsb-release wget
RUN wget https://packages.apache.org/artifactory/arrow/$(lsb_release --id --short | tr 'A-Z' 'a-z')/apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb && \
apt-get install -y -V ./apache-arrow-apt-source-latest-$(lsb_release --codename --short).deb && \
apt-get update
RUN wget https://packages.apache.org/artifactory/arrow/ubuntu/pool/noble/main/a/apache-arrow/libarrow2000-dbgsym_20.0.0-1_amd64.ddeb && \
apt-get install -y ./libarrow2000-dbgsym_20.0.0-1_amd64.ddeb
# To use readelf
RUN apt-get install -y binutils$ docker build -t arrow-dbgsym-noble .
$ docker run --rm -it arrow-dbgsym-noble bash
root@2e4db0036917:/# readelf --notes /usr/lib/x86_64-linux-gnu/libarrow.so.2000
Displaying notes found in: .note.gnu.build-id
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: ebdc8688b2d2c59f4a5aabbcc170deb844dbf936
root@2e4db0036917:/# addr2line --exe=/usr/lib/debug/.build-id/eb/dc8688b2d2c59f4a5aabbcc170deb844dbf936.debug +0x10
??:0 |
|
I reported in apache/arrow#46284. |
|
Could you try |
|
I tried it. But $ docker run --rm -it arrow-dbgsym-noble bash
root@a23ed7fc5466:/# readelf --notes /usr/lib/x86_64-linux-gnu/libarrow.so.2000
Displaying notes found in: .note.gnu.build-id
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: ebdc8688b2d2c59f4a5aabbcc170deb844dbf936
root@a23ed7fc5466:/# addr2line --exe=/usr/lib/debug/.build-id/eb/dc8688b2d2c59f4a5aabbcc170deb844dbf936.debug _Z31garrow_function_options_new_rawPKN5arrow7compute15FunctionOptionsE+0x10
??:0I also tried /lib/x86_64-linux-gnu/libarrow-glib.so.2000(_Z31garrow_function_options_new_rawPKN5arrow7compute15FunctionOptionsE+0x10) [0x7f96d29aa320]root@a23ed7fc5466:/# wget https://packages.apache.org/artifactory/arrow/ubuntu/pool/noble/main/a/apache-arrow/libarrow-glib2000-dbgsym_20.0.0-1_amd64.ddeb
(snip)
root@a23ed7fc5466:/# apt install -y -V ./libarrow-glib2000-dbgsym_20.0.0-1_amd64.ddeb
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Note, selecting 'libarrow-glib2000-dbgsym' instead of './libarrow-glib2000-dbgsym_20.0.0-1_amd64.ddeb'
The following additional packages will be installed:
libarrow-acero2000 (20.0.0-1)
libarrow-glib2000 (20.0.0-1)
libglib2.0-0t64 (2.80.0-6ubuntu3.2)
libglib2.0-data (2.80.0-6ubuntu3.2)
shared-mime-info (2.4-4)
xdg-user-dirs (0.18-1build1)
Suggested packages:
low-memory-monitor (2.1-2build2)
The following NEW packages will be installed:
libarrow-acero2000 (20.0.0-1)
libarrow-glib2000 (20.0.0-1)
libarrow-glib2000-dbgsym (20.0.0-1)
libglib2.0-0t64 (2.80.0-6ubuntu3.2)
libglib2.0-data (2.80.0-6ubuntu3.2)
shared-mime-info (2.4-4)
xdg-user-dirs (0.18-1build1)
0 upgraded, 7 newly installed, 0 to remove and 1 not upgraded.
Need to get 2881 kB/9078 kB of archives.
After this operation, 17.1 MB of additional disk space will be used.
Get:1 /libarrow-glib2000-dbgsym_20.0.0-1_amd64.ddeb libarrow-glib2000-dbgsym amd64 20.0.0-1 [6198 kB]
Get:4 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libglib2.0-0t64 amd64 2.80.0-6ubuntu3.2 [1546 kB]
Get:2 https://packages.apache.org/artifactory/arrow/ubuntu noble/main amd64 libarrow-acero2000 amd64 20.0.0-1 [451 kB]
Get:3 https://packages.apache.org/artifactory/arrow/ubuntu noble/main amd64 libarrow-glib2000 amd64 20.0.0-1 [343 kB]
Get:5 http://archive.ubuntu.com/ubuntu noble-updates/main amd64 libglib2.0-data all 2.80.0-6ubuntu3.2 [48.5 kB]
Get:6 http://archive.ubuntu.com/ubuntu noble/main amd64 shared-mime-info amd64 2.4-4 [474 kB]
Get:7 http://archive.ubuntu.com/ubuntu noble/main amd64 xdg-user-dirs amd64 0.18-1build1 [18.4 kB]
Fetched 2881 kB in 5s (565 kB/s)
debconf: delaying package configuration, since apt-utils is not installed
Selecting previously unselected package libglib2.0-0t64:amd64.
(Reading database ... 5765 files and directories currently installed.)
Preparing to unpack .../0-libglib2.0-0t64_2.80.0-6ubuntu3.2_amd64.deb ...
Unpacking libglib2.0-0t64:amd64 (2.80.0-6ubuntu3.2) ...
Selecting previously unselected package libglib2.0-data.
Preparing to unpack .../1-libglib2.0-data_2.80.0-6ubuntu3.2_all.deb ...
Unpacking libglib2.0-data (2.80.0-6ubuntu3.2) ...
Selecting previously unselected package shared-mime-info.
Preparing to unpack .../2-shared-mime-info_2.4-4_amd64.deb ...
Unpacking shared-mime-info (2.4-4) ...
Selecting previously unselected package xdg-user-dirs.
Preparing to unpack .../3-xdg-user-dirs_0.18-1build1_amd64.deb ...
Unpacking xdg-user-dirs (0.18-1build1) ...
Selecting previously unselected package libarrow-acero2000:amd64.
Preparing to unpack .../4-libarrow-acero2000_20.0.0-1_amd64.deb ...
Unpacking libarrow-acero2000:amd64 (20.0.0-1) ...
Selecting previously unselected package libarrow-glib2000:amd64.
Preparing to unpack .../5-libarrow-glib2000_20.0.0-1_amd64.deb ...
Unpacking libarrow-glib2000:amd64 (20.0.0-1) ...
Selecting previously unselected package libarrow-glib2000-dbgsym:amd64.
Preparing to unpack .../6-libarrow-glib2000-dbgsym_20.0.0-1_amd64.ddeb.deb ...
Unpacking libarrow-glib2000-dbgsym:amd64 (20.0.0-1) ...
Setting up libarrow-acero2000:amd64 (20.0.0-1) ...
Setting up xdg-user-dirs (0.18-1build1) ...
Setting up libglib2.0-0t64:amd64 (2.80.0-6ubuntu3.2) ...
No schema files found: doing nothing.
Setting up libglib2.0-data (2.80.0-6ubuntu3.2) ...
Setting up shared-mime-info (2.4-4) ...
Setting up libarrow-glib2000:amd64 (20.0.0-1) ...
Setting up libarrow-glib2000-dbgsym:amd64 (20.0.0-1) ...
Processing triggers for libc-bin (2.39-0ubuntu8.4) ...
root@a23ed7fc5466:/#
root@a23ed7fc5466:/# readelf --notes /usr/lib/x86_64-linux-gnu/libarrow-glib.so.2000
Displaying notes found in: .note.gnu.property
Owner Data size Description
GNU 0x00000010 NT_GNU_PROPERTY_TYPE_0
Properties: x86 feature: IBT, SHSTK
Displaying notes found in: .note.gnu.build-id
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: 793ff56e8d52369e4e6ade6031ce9d3e5ee49e18
root@a23ed7fc5466:/# addr2line --exe=/usr/lib/debug/.build-id/79/3ff56e8d52369e4e6ade6031ce9d3e5ee49e18.debug _Z31garrow_function_options_new_rawPKN5arrow7compute15FunctionOptionsE+0x10
??:? |
|
I also tried it and it didn't work... BTW, how about using Ubuntu 24.04 instead 22.04? diff --git a/.devcontainer/Dockerfile b/.devcontainer/Dockerfile
index 04ae0f1..38057e6 100644
--- a/.devcontainer/Dockerfile
+++ b/.devcontainer/Dockerfile
@@ -1,5 +1,4 @@
-# [Choice] debian-11, debian-10, ubuntu-22.04, ubuntu-20.04, ubuntu-18.04
-ARG VARIANT=ubuntu-22.04
+ARG VARIANT=ubuntu-24.04
FROM mcr.microsoft.com/devcontainers/base:${VARIANT}
|
|
I can reproduce this by |
|
This returns // g++ -g3 -O0 a.cc $(pkgconf --cflags --libs arrow) && ./a.out
#include <stdio.h>
#include <arrow/compute/api.h>
int
main(void)
{
auto func = *(arrow::compute::GetFunctionRegistry()->GetFunction("rank"));
printf("%p\n", func->default_options()->options_type());
return 0;
}It seems that the Apache Arrow C++ package for Ubuntu has a problem. |
Because `apache-arrow-apt-source_15.0.0.1_all.deb` is unavailable in Ubuntu 24.04.
Since e558326, we use Apache Arrow latest version.
|
Thanks for your investigate! I can also reproduce this by
OK. I committed the following improvements:
|
Yes, please. Note that this is a C++ related problem not GLib nor Ruby because #306 (comment) reproduces this. |
|
The following line has changed slightly, but the /lib/x86_64-linux-gnu/libarrow-glib.so.2000(_Z31garrow_function_options_new_rawPKN5arrow7compute15FunctionOptionsE+0x18) [0x7f789c15fa58]-_Z31garrow_function_options_new_rawPKN5arrow7compute15FunctionOptionsE+0x10
+_Z31garrow_function_options_new_rawPKN5arrow7compute15FunctionOptionsE+0x18$ docker run --rm -it arrow-dbgsym-noble bash
root@b6151ebccc55:/# readelf --notes /usr/lib/x86_64-linux-gnu/libarrow.so.2000
Displaying notes found in: .note.gnu.build-id
Owner Data size Description
GNU 0x00000014 NT_GNU_BUILD_ID (unique build ID bitstring)
Build ID: ebdc8688b2d2c59f4a5aabbcc170deb844dbf936
root@b6151ebccc55:/# addr2line --exe=/usr/lib/debug/.build-id/eb/dc8688b2d2c59f4a5aabbcc170deb844dbf936.debug _Z31garrow_function_options_new_rawPKN5arrow7compute15FunctionOptionsE+0x18
??:0 |
|
|
I reported in apache/arrow#46299.
Thanks for your investigate. Should we open it in apache/arrow? If yes, Could I open it in apache/arrow? |
|
Yes, please. |
|
I reported in apache/arrow#46304. |
|
Thanks for your work in apache/arrow. All reported issues have been resolved. It seems that this segmentation fault will be resolved in the Apache Arrow 21.0.0 release. |
|
Are there any remaining issues before we merge this? |
|
No, there are no remaining issues. |
2 participants
This PR will fix the build container / build failure: