Skip to content

feat(Makefile): Add kernel build integration and local APT repo support #325

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
abickett wants to merge 2 commits into
base: main
Choose a base branch
from
+91 −13
Open

feat(Makefile): Add kernel build integration and local APT repo support #325

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9ef6923
fix(build-linux-deb): use linux_dir variable instead of hardcoded 'li…
abickett Mar 6, 2026
4a8aae2
feat(Makefile): Add kernel build integration and local APT repo support
abickett Mar 1, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 88 additions & 10 deletions Makefile
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,13 +1,15 @@
# DEBOS_OPTS can be overridden with:
# make DEBOS_OPTS=... all
# USE_CONTAINER can be set to yes/no/auto (default: auto)
# make USE_CONTAINER=yes all # Force container use
# make USE_CONTAINER=no all # Force native debos
# Key variables (override on the command line):
# DEBOS_OPTS="..." - override debos options
# USE_CONTAINER=yes/no/auto - control container use (default: auto)
# DEBOS_ARGS="..." - pass extra arguments to debos
# KERNEL_REPO/KERNEL_REF - kernel source for 'make kernel-deb'
# KERNEL_PACKAGE - kernel package name (auto-detected from local-apt-repo/)

# To build large images, the debos resource defaults are not sufficient. These
# provide defaults that work for us as universally as we can manage.
FAKEMACHINE_BACKEND = $(shell [ -c /dev/kvm ] && echo kvm || echo qemu)
DEBOS_OPTS := --fakemachine-backend $(FAKEMACHINE_BACKEND) --memory 1GiB --scratchsize 6GiB
DEBOS_ARGS ?=
Copy link
Copy Markdown
Contributor

@lool lool Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shoot, that will likely conflict with #319


# Container support: auto-detect if debos is available, otherwise use container
USE_CONTAINER ?= auto
Expand Down Expand Up @@ -50,13 +52,87 @@ export http_proxy
all: disk-ufs.img disk-sdcard.img

rootfs.tar: debos-recipes/qualcomm-linux-debian-rootfs.yaml
$(DEBOS_CMD) $<
$(DEBOS_CMD) $(DEBOS_ARGS) $<

disk-ufs.img: debos-recipes/qualcomm-linux-debian-image.yaml rootfs.tar
$(DEBOS_CMD) $<
$(DEBOS_CMD) $(DEBOS_ARGS) $<

disk-sdcard.img: debos-recipes/qualcomm-linux-debian-image.yaml rootfs.tar
$(DEBOS_CMD) -t imagetype:sdcard $<
$(DEBOS_CMD) $(DEBOS_ARGS) -t imagetype:sdcard $<

# Kernel build variables - override to build a specific kernel
KERNEL_REPO ?= https://github.com/torvalds/linux
Copy link
Copy Markdown
Contributor

@lool lool Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I don't think we should duplicate the list of kernel git URLs, default branch etc. between the kernel building script and the Makefile. We can argue where it should live, but let's keep only one.

The kernel build script is fairly dumb, except for picking the latest tag. For qcom-next, I specifically received guidance that I should be using the highest qcom-next-date tag as some kind of release of qcom-next, even if other changes are landing daily in tip. I wouldn't want to do that from shell or – worse – from a Makefile :)

Personally, I think it makes sense to keep a short list of "well-known" kernels in build-linux-deb.py, but it's probably not scalable to mirror every script option with a Makefile option. Perhaps we do like debos, and just have a LINUX_OPTS (can't think of a good name) that lets one pass --qcom-next etc.?

The last tricky thing is the .config fragments. With this script, we always want to use some upstream defconfig, sometimes some tree-specific configs (in qcom-next: prune.config, qcom.config), qcom-deb-images wants to pass a number of config fragments, and I can guess users might want some extra ones too. The defaults could be set in the Makefile instead of github workflows, and perhaps we provide a way to override and/or extend, like with DEBOS.

KERNEL_REF ?= master
KERNEL_DIR ?=
# Set to 'yes' to use qcom-next defaults (auto-finds latest dated tag)
USE_QCOM_NEXT ?= no
# Set to 'yes' to use linux-next defaults (auto-finds latest dated tag)
USE_LINUX_NEXT ?= no
KERNEL_DEB_EXTRA_ARGS ?=

# Local APT repo directory - mirrors what CI sets up via aptlocalrepo
LOCAL_APT_REPO ?= local-apt-repo

# KERNEL_PACKAGE: auto-detected from $(LOCAL_APT_REPO) after 'make kernel-deb'.
# Override to select a specific package, or pass KERNEL_PACKAGE= to disable.
KERNEL_PACKAGE ?= $(shell find $(LOCAL_APT_REPO) -type f -name 'linux-image-*' \
-not -name '*-dbg_*' 2>/dev/null | xargs -n1 basename 2>/dev/null | \
cut -f1 -d_ | sort | tail -1)
ifneq ($(KERNEL_PACKAGE),)
override DEBOS_ARGS += -t aptlocalrepo:$(DEBOS_WORKDIR)/$(LOCAL_APT_REPO) \
-t kernelpackage:$(KERNEL_PACKAGE)
endif

# Rebuild the APT repo index in $(LOCAL_APT_REPO).
# Requires: apt-utils (apt-ftparchive)
.PHONY: setup-apt-repo
setup-apt-repo:
cd $(LOCAL_APT_REPO) && apt-ftparchive packages . > Packages && apt-ftparchive release . > Release
@echo ""
@echo "Local APT repo ready at: $(LOCAL_APT_REPO)/"
@echo ""
@echo "KERNEL_PACKAGE will be auto-detected; to build rootfs and images:"
@echo " make rootfs.tar && make disk-ufs.img disk-sdcard.img"

# Build a kernel deb package and add it to the local APT repo.
# Examples:
# make kernel-deb USE_QCOM_NEXT=yes
# make kernel-deb KERNEL_REPO=https://github.com/qualcomm-linux/kernel KERNEL_REF=qcom-next
# make kernel-deb KERNEL_DIR=/path/to/linux # use existing source
.PHONY: kernel-deb
kernel-deb:
@if [ "$(USE_QCOM_NEXT)" = "yes" ] && [ "$(USE_LINUX_NEXT)" = "yes" ]; then \
echo "Error: Cannot use both USE_QCOM_NEXT=yes and USE_LINUX_NEXT=yes"; \
exit 1; \
elif [ "$(USE_QCOM_NEXT)" = "yes" ]; then \
scripts/build-linux-deb.py \
--qcom-next \
$(if $(KERNEL_DIR),--local-dir $(KERNEL_DIR)) \
$(KERNEL_DEB_EXTRA_ARGS) \
$(sort $(wildcard kernel-configs/*.config)); \
elif [ "$(USE_LINUX_NEXT)" = "yes" ]; then \
scripts/build-linux-deb.py \
--linux-next \
$(if $(KERNEL_DIR),--local-dir $(KERNEL_DIR)) \
$(KERNEL_DEB_EXTRA_ARGS) \
$(sort $(wildcard kernel-configs/*.config)); \
else \
scripts/build-linux-deb.py \
$(if $(KERNEL_DIR),--local-dir $(KERNEL_DIR)) \
--repo $(KERNEL_REPO) \
--ref $(KERNEL_REF) \
$(KERNEL_DEB_EXTRA_ARGS) \
$(sort $(wildcard kernel-configs/*.config)); \
fi
mkdir -p $(LOCAL_APT_REPO)/kernel
@# Kernel debs are created in parent dir of kernel source
@if [ -n "$(KERNEL_DIR)" ]; then \
mv -v $(dir $(abspath $(KERNEL_DIR)))*.deb $(LOCAL_APT_REPO)/kernel/ 2>/dev/null || \
mv -v $(abspath $(KERNEL_DIR))/../*.deb $(LOCAL_APT_REPO)/kernel/; \
else \
mv -v *.deb $(LOCAL_APT_REPO)/kernel/; \
fi
$(MAKE) setup-apt-repo

.PHONY: test
test: disk-ufs.img
Expand All @@ -65,7 +141,9 @@ test: disk-ufs.img

.PHONY: clean
clean:
rm -f disk-ufs.img1 disk-ufs.img2 disk-ufs.img
rm -f disk-sdcard.img1 disk-sdcard.img2 disk-sdcard.img
rm -f rootfs.tar
rm -f dtbs.tar.gz
rm -f disk-*.img disk-*.img.gz disk-*.img[0-9]
Copy link
Copy Markdown
Contributor

@lool lool Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure how, but I think you're reverting changes from tip?

rm -rf rootfs/
Copy link
Copy Markdown
Contributor

@lool lool Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah do we need this?

rm -rf linux/
rm -rf $(LOCAL_APT_REPO)/
Copy link
Copy Markdown
Contributor

@lool lool Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason, the / makes me uncomfortable; I kind of fear this expands to rm -rf /; perhaps leave the trailing / out on this one.

2 changes: 1 addition & 1 deletion debos-recipes/qualcomm-linux-debian-rootfs.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -44,7 +44,7 @@ actions:
cat >/etc/apt/sources.list.d/apt-local-repo.sources <<EOF
Types: deb
URIs: file:///media/apt-local-repo
Suites: /
Suites: ./
Copy link
Copy Markdown
Contributor

@lool lool Mar 13, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(Thanks for the fix!) That should be its own commit; I'm curious what problem you ran into (since apt install works in our CI)!

Trusted: true
EOF
{{- end }}
Expand Down
4 changes: 2 additions & 2 deletions scripts/build-linux-deb.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -285,15 +285,15 @@ def main():
subprocess.run(
merge_command,
check=True,
cwd="linux",
cwd=str(linux_dir),
env={"ARCH": "arm64", **subprocess.os.environ}
)

# Finalize config with olddefconfig
subprocess.run(
make_base_command + ["olddefconfig"],
check=True,
cwd="linux"
cwd=str(linux_dir)
)

log_i("Building Linux deb")
Expand Down
Loading