Skip to content

Commit 961e621

Browse files
AndrewyxLmh-javawilliamckhapre-commit-ci-lite[bot]Thunderbots
authored
Andrewyx/Upgrade Bazel 8 (UBC-Thunderbots#3459)
* init * partial migration * new bazel upgrade * partial convert * libs * workaround on eigen * changes * deduplicate proto lib * instanity * chaos * WIP: fix protobuf headers * diffs * working * logging change * use eigen from bcr * estop fixed * caching working + offline usage * minor cleanup * toolchain protoc * g3log repo rules * changes * use bazelisk * cleanup * use bazelisk & cleanup * refactored * rework gen * removed * building * protos compile * Add grpcio-tools to requirements * almost working * building * pathing issues * hacky * fix g3log * fix cross compile on aarch64 * revise util.sh * fix nanopb pb.h * [may revert] fix powerloop tar package * rename generated nanopb files to .nanopb.c and .nanopb.h * got nanopb to work * fix aarch64 absolute path issue * fix ci * cleanup comments * deps fix * remove sensor_msg from src * fixes protolibs * missing deps * debug removed * missing deps * missing deps * deps fixes * linking fixes * updated crc * proto fixes * hermit fixes * fix uart test case failure * fixes * [pre-commit.ci lite] apply automatic fixes * fix nanopb constraints * generalize nanopb options * [pre-commit.ci lite] apply automatic fixes * remove hacky copts * [pre-commit.ci lite] apply automatic fixes * rework buildifier * integrate new buildifier to ci * [pre-commit.ci lite] apply automatic fixes * fixes and changed platformio build upstream * working on robot * address comments * pass ci * [pre-commit.ci lite] apply automatic fixes * Add nanopb requirements to compile_pip_requirements.sh * cleanup * removed trinamic hacks * working on robot, tar broken * [pre-commit.ci lite] apply automatic fixes * tar fixes * partial fix to warnings --------- Co-authored-by: Minghao Li <mli131@student.ubc.ca> Co-authored-by: williamckha <williamha@outlook.com> Co-authored-by: pre-commit-ci-lite[bot] <117423508+pre-commit-ci-lite[bot]@users.noreply.github.com> Co-authored-by: Thunderbots <ubcrobocup@gmail.com>
1 parent 52820a7 commit 961e621

141 files changed

Lines changed: 2439 additions & 1305 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

.github/workflows/main.yml

Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -35,17 +35,17 @@ jobs:
3535
-//software/ai/hl/... \
3636
-//software/field_tests/... \
3737
-//software/embedded/... \
38-
-//cc_toolchain/...
38+
-//toolchains/cc/...
3939
4040
- name: Jetson Nano Build Test
4141
run: |
4242
cd src
43-
bazel build //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=NANO --platforms=//cc_toolchain:robot
43+
bazel build //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=NANO --platforms=//toolchains/cc:robot
4444
4545
- name: Raspberry Pi Build Test
4646
run: |
4747
cd src
48-
bazel build //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=PI --platforms=//cc_toolchain:robot
48+
bazel build //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=PI --platforms=//toolchains/cc:robot
4949
5050
5151
software-tests:
@@ -68,7 +68,7 @@ jobs:
6868
-//software/ai/hl/... \
6969
-//software/field_tests/... \
7070
-//software/ai/navigator/... \
71-
-//cc_toolchain/...
71+
-//toolchains/cc/...
7272
7373
robot-tests:
7474
name: Robot Software Tests
@@ -97,13 +97,13 @@ jobs:
9797
- name: Jetson Nano Build
9898
run: |
9999
cd src
100-
bazel build //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=NANO --platforms=//cc_toolchain:robot
100+
bazel build //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=NANO --platforms=//toolchains/cc:robot
101101
102102
103103
- name: Raspberry Pi Build
104104
run: |
105105
cd src
106-
bazel build //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=PI --platforms=//cc_toolchain:robot
106+
bazel build //software/embedded:thunderloop_main --copt=-O3 --//software/embedded:host_platform=PI --platforms=//toolchains/cc:robot
107107
108108
109109
simulated-gameplay-tests:
@@ -112,7 +112,7 @@ jobs:
112112
steps:
113113
# checks-out the repository under $GITHUB_WORKSPACE
114114
- uses: actions/checkout@v4
115-
115+
116116
- name: Environment Setup
117117
run: |
118118
"${GITHUB_WORKSPACE}"/environment_setup/setup_software.sh

environment_setup/ubuntu20_requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -10,3 +10,4 @@ PyOpenGL==3.1.6
1010
ruff==0.5.5
1111
pyqt-toast-notification==1.3.2
1212
md-toc==9.0.0
13+
grpcio-tools==1.71.0

environment_setup/ubuntu22_requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -11,3 +11,4 @@ numpy==1.26.4
1111
ruff==0.5.5
1212
pyqt-toast-notification==1.3.2
1313
md-toc==9.0.0
14+
grpcio-tools==1.71.0

environment_setup/ubuntu24_requirements.txt

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -8,3 +8,4 @@ PyOpenGL==3.1.6
88
ruff==0.5.5
99
pyqt-toast-notification==1.3.2
1010
md-toc==9.0.0
11+
grpcio-tools==1.71.0

environment_setup/util.sh

Lines changed: 23 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -9,12 +9,11 @@ install_autoref() {
99
}
1010

1111
install_bazel() {
12-
download=https://github.com/bazelbuild/bazel/releases/download/5.4.0/bazel-5.4.0-linux-arm64
12+
download=https://github.com/bazelbuild/bazelisk/releases/download/v1.26.0/bazelisk-linux-arm64
1313

1414
if is_x86 $1; then
15-
download=https://github.com/bazelbuild/bazel/releases/download/5.4.0/bazel-5.4.0-linux-x86_64
15+
download=https://github.com/bazelbuild/bazelisk/releases/download/v1.26.0/bazelisk-linux-amd64
1616
fi
17-
1817
wget -nc $download -O /tmp/tbots_download_cache/bazel
1918
sudo mv /tmp/tbots_download_cache/bazel /usr/bin/bazel
2019
sudo chmod +x /usr/bin/bazel
@@ -96,28 +95,35 @@ install_java () {
9695
}
9796

9897
install_python_dev_cross_compile_headers() {
98+
mkdir -p /opt/tbotspython/cross_compile_headers
99+
wget -N https://www.python.org/ftp/python/3.12.0/Python-3.12.0.tar.xz -O /tmp/tbots_download_cache/python-3.12.0.tar.xz
100+
tar -xf /tmp/tbots_download_cache/python-3.12.0.tar.xz -C /tmp/tbots_download_cache/
101+
cd /tmp/tbots_download_cache/Python-3.12.0
102+
103+
# The configuration is taken from the examples provided in https://docs.python.org/3.12/using/configure.html
104+
echo ac_cv_buggy_getaddrinfo=no > config.site-aarch64
105+
echo ac_cv_file__dev_ptmx=yes >> config.site-aarch64
106+
echo ac_cv_file__dev_ptc=no >> config.site-aarch64
99107
if is_x86 $1; then
100-
mkdir -p /opt/tbotspython/cross_compile_headers
101-
wget -N https://www.python.org/ftp/python/3.12.0/Python-3.12.0.tar.xz -O /tmp/tbots_download_cache/python-3.12.0.tar.xz
102-
tar -xf /tmp/tbots_download_cache/python-3.12.0.tar.xz -C /tmp/tbots_download_cache/
103-
cd /tmp/tbots_download_cache/Python-3.12.0
104-
105-
# The configuration is taken from the examples provided in https://docs.python.org/3.12/using/configure.html
106-
echo ac_cv_buggy_getaddrinfo=no > config.site-aarch64
107-
echo ac_cv_file__dev_ptmx=yes >> config.site-aarch64
108-
echo ac_cv_file__dev_ptc=no >> config.site-aarch64
109108
CONFIG_SITE=config.site-aarch64 ./configure \
110109
--build=x86_64-pc-linux-gnu \
111110
--host=aarch64-unknown-linux-gnu \
112111
-with-build-python=/usr/bin/python3.12 \
113112
--enable-optimizations \
114113
--prefix=/opt/tbotspython/cross_compile_headers > /dev/null
115-
make inclinstall -j$(nproc) > /dev/null
116-
117-
cd -
118-
rm -rf /tmp/tbots_download_cache/Python-3.12.0
119-
rm -rf /tmp/tbots_download_cache/python-3.12.0.tar.xz
114+
else
115+
CONFIG_SITE=config.site-aarch64 ./configure \
116+
--build=aarch64-pc-linux-gnu \
117+
--host=aarch64-unknown-linux-gnu \
118+
-with-build-python=/usr/bin/python3.12 \
119+
--enable-optimizations \
120+
--prefix=/opt/tbotspython/cross_compile_headers > /dev/null
120121
fi
122+
make inclinstall -j$(nproc) > /dev/null
123+
124+
cd -
125+
rm -rf /tmp/tbots_download_cache/Python-3.12.0
126+
rm -rf /tmp/tbots_download_cache/python-3.12.0.tar.xz
121127
}
122128

123129
is_x86() {

scripts/compile_pip_requirements.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,8 @@ CURR_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )"
1515
BAZEL_ROOT_DIR="$CURR_DIR/../src"
1616

1717
cd $BAZEL_ROOT_DIR
18-
bazel run //extlibs:nanopb_requirements.update
1918
bazel run //software/thunderscope:requirements.update
2019
bazel run //software/embedded/ansible:requirements.update
2120
bazel run //software/simulated_tests:requirements.update
2221
bazel run //software/embedded/robot_diagnostics_cli:requirements.update
22+
bazel run //starlark/nanopb:requirements.update

scripts/lint_and_format.sh

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -48,7 +48,7 @@ function run_clang_format () {
4848
function run_bazel_formatting () {
4949
printf "Running bazel buildifier to format all bazel BUILD files...\n\n"
5050
cd $BAZEL_ROOT_DIR
51-
bazel run @com_github_bazelbuild_buildtools//:buildifier
51+
bazel run //starlark/buildifier:buildifier.fix
5252

5353
if [[ "$?" != 0 ]]; then
5454
printf "\n***Failed to format bazel BUILD files!***\n\n"

src/.bazelrc

Lines changed: 62 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,28 +1,20 @@
1-
2-
# We rely on all symbols being included when linking for our implementation of
3-
# the "factory" design pattern.
4-
build --incompatible_remove_legacy_whole_archive=False
1+
################## Base Bazel Configs ##################
2+
common --enable_bzlmod
3+
common --lockfile_mode=update
4+
common --noincompatible_disallow_empty_glob # Allow globs to be empty without failing
5+
common --incompatible_disallow_struct_provider_syntax=false
56

67
# We want bazel to output the results of our gtests
78
test --test_output=all
89

9-
# Use Our Custom Toolchain
10-
build --crosstool_top=//cc_toolchain:toolchain
11-
12-
# Add warnings to Thunderbots code only
13-
build --per_file_copt=//proto/.*,//proto/message_translation/.*,//proto/primitive/.*,//software/.*,//shared/.*@-Wall,-Wextra,-Wconversion,-Wno-unused-parameter,-Wno-deprecated,-Werror,-Wno-deprecated-declarations
10+
################## Bazel Debug Configs ##################
1411

15-
# Warn variable length arrays only when compiling cpp
16-
build --per_file_copt=.*\.cpp@-Wvla
17-
18-
# Automatically set the CPU environment based on the `--cpu` flag as per our
19-
# defined CPU environments
20-
build --auto_cpu_environment_group=//cc_toolchain:cpus
12+
# Enable the following commands to debug Bazel. Note that the additional overhead from these commands will
13+
# cause CI to fail
14+
# common --verbose_failures
15+
# common --sandbox_debug
2116

22-
# Use a local directory as a remote cache. This shares artifacts when switching
23-
# between git branches or workspaces for the same project.
24-
# See https://docs.bazel.build/versions/master/remote-caching.html#disk-cache
25-
build --disk_cache=~/.cache/thunderbots_bazel_disk_cache
17+
################ Machine Configurations ################
2618

2719
# The XDR_RUNTIME_DIR and DISPLAY environment variables must be passed to the tests
2820
# so that our GUIs can be enabled for simulated tests. This is needed for Qt
@@ -34,6 +26,54 @@ build --test_env=DISPLAY
3426
build --action_env=PYTHON_BIN_PATH=/opt/tbotspython/bin/python3.12
3527
build --action_env=PYTHON_LIB_PATH=/opt/tbotspython/lib/python3.12
3628

37-
# Enable the new cc_toolchain resolution strategy
38-
# #TODO(#3382): Delete when we upgrade to Bazel 7.0
39-
build --incompatible_enable_cc_toolchain_resolution
29+
# Automatically set the CPU environment based on the `--cpu` flag as per our
30+
# defined CPU environments
31+
build --auto_cpu_environment_group=//toolchains/cc:cpus
32+
33+
# Use Our Custom Toolchain
34+
build --crosstool_top=//toolchains/cc:toolchain
35+
common --features=external_include_paths
36+
37+
################# Build Optimizations #################
38+
39+
# Use a local directory as a remote cache. This shares artifacts when switching
40+
# between git branches or workspaces for the same project.
41+
# See https://docs.bazel.build/versions/master/remote-caching.html#disk-cache
42+
build --disk_cache=~/.cache/thunderbots_bazel_disk_cache
43+
44+
build --experimental_remote_merkle_tree_cache # partial repo caching optimization
45+
build --incompatible_enable_proto_toolchain_resolution # protoc caching
46+
47+
# Ensure that we don't accidentally build gRPC
48+
common --per_file_copt=external/.*grpc.*@--GRPC_WAS_NOT_SUPPOSED_TO_BE_BUILT
49+
common --host_per_file_copt=external/.*grpc.*@--GRPC_WAS_NOT_SUPPOSED_TO_BE_BUILT
50+
51+
################# Build Configurations #################
52+
# Lock C++ version
53+
build --cxxopt="-std=c++2a"
54+
# Warn variable length arrays only when compiling cpp
55+
build --per_file_copt=.*\.cpp@-Wvla
56+
57+
# Enable runfiles symlink tree
58+
build --enable_runfiles
59+
60+
# Allow usage of platform specific .bazelrc configs (e.g. build:linux, common:windows, etc)
61+
common --enable_platform_specific_config
62+
63+
# We rely on all symbols being included when linking for our implementation of
64+
# the "factory" design pattern.
65+
build --incompatible_remove_legacy_whole_archive=False
66+
67+
# Escalate Warnings to fail Compile for Thunderbots code
68+
build --features=external_include_paths
69+
build --per_file_copt=proto/.*,proto/message_translation/.*,proto/primitive/.*,software/.*,shared/.*,-external/.*@-Wall,-Wextra,-Wno-unused-parameter,-Wno-deprecated,-Werror,-Wno-deprecated-declarations
70+
# TODO: #3492
71+
# build --per_file_copt=software/.*,shared/.*,-external/.*@-Wconversion
72+
73+
build --per_file_copt=.*\.pb\.(h|cc|cpp)$@-w
74+
build --per_file_copt=./external/.*,./bazel-out/.*@-w
75+
76+
############## External Deps Compatibility ##############
77+
build --noincompatible_disallow_ctx_resolve_tools
78+
build --check_direct_dependencies=off
79+

src/.bazelversion

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1 +1 @@
1-
5.4.0
1+
8.2.1

src/BUILD

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,5 @@
1-
load("@rules_python//python:defs.bzl", "py_runtime")
2-
load("@rules_python//python:defs.bzl", "py_runtime_pair")
31
load("@hedron_compile_commands//:refresh_compile_commands.bzl", "refresh_compile_commands")
2+
load("@rules_python//python:defs.bzl", "py_runtime", "py_runtime_pair")
43

54
# Generates compile_commands.json
65
refresh_compile_commands(

0 commit comments

Comments
 (0)