Skip to content

feat: install most components to /opt/bin instead of /usr/local/bin#7125

Merged
cameronmeissner merged 7 commits intomainfrom
chewi/opt-bin
Jan 20, 2026
Merged

feat: install most components to /opt/bin instead of /usr/local/bin#7125
cameronmeissner merged 7 commits intomainfrom
chewi/opt-bin

Conversation

@chewi
Copy link
Copy Markdown
Contributor

@chewi chewi commented Oct 3, 2025

What type of PR is this?

/kind feature

What this PR does / why we need it:

/usr/local/bin is generally read-only on immutable distributions, so binaries cannot be installed there. This PR installs them to /opt/bin instead.

The packages published to PMC and the system extensions built from those install their binaries to /usr/bin. These were previously moved to /usr/local/bin, but since this is no longer possible, symlinks are now created in /opt/bin instead.

Symlinks are not used when installing from a tarball because the versioned binaries (e.g. kubelet-*) are removed, leading to dangling links.

The install command has been used to write these binaries because it removes the existing file first (rather than following a symlink) and takes care of making the new file executable.

As an aside, this also fixes a failing test that was introduced in #6995.

Requirements:

  • uses conventional commit messages
  • includes documentation
  • adds unit tests
  • tested upgrade from previous version
  • commits are GPG signed and Github marks them as verified

Special notes for your reviewer:

This change is part of a wider effort to install binaries to Flatcar from systemd system extensions (sysexts) rather than binary tarballs, which are being phased out. There are too many changes to submit everything in this one PR, so another will follow this one.

I could not find any relevant documentation to update.

Regarding upgrades, I am not experienced enough with AKS to know how to test that. I don't believe the changes would cause any issue for upgrades because they only change how and where things are installed. There are practically no configuration changes besides some paths in the systemd units.

Release note:

Is one needed here? I don't know whether end users will see or care about these changes.

@chewi
Copy link
Copy Markdown
Contributor Author

chewi commented Oct 6, 2025

I can't seem to get the E2E tests to pass here, even though they did pass internally against the same commit. I have also successfully rerun Test_AzureLinux3_PMC_Install manually using the same build. I'm putting this down to flakiness.

@chewi
Copy link
Copy Markdown
Contributor Author

chewi commented Oct 8, 2025

I hadn't realised you could grab the scenario logs from these runs. Having a closer look now.

@chewi
Copy link
Copy Markdown
Contributor Author

chewi commented Jan 15, 2026

I've fixed the major issue I was having. I didn't understand that systemd units referencing the new path were being applied to the older image with binaries at the old path. I have worked around this with a cloud-config bootcmd that creates symlinks. Flatcar doesn't use cloud-config, so I have made it translate bootcmd into a systemd unit deployed by Ignition.

I haven't been able to get an entirely clean pass here or with RP, but the few failures I've seen are due to availability or other unrelated issues.

Comment thread parts/common/components.json
Comment thread parts/linux/cloud-init/artifacts/azlosguard/cse_install_osguard.sh
Comment thread vhdbuilder/packer/imagecustomizer/azlosguard/scripts/azlosguard-postinstall.sh Outdated
Comment thread parts/linux/cloud-init/nodecustomdata.yml
Comment thread parts/linux/cloud-init/artifacts/profile-d-path.sh
Comment thread parts/linux/cloud-init/nodecustomdata.yml
Comment thread pkg/agent/baker.go
chewi added 7 commits January 20, 2026 17:00
It was prefixing lines with the wrong function name, which had me
stumped for hours.
It is installed using dnf, so drop downloadLocation to avoid confusion.
Ignition runs in the initrd, whereas cloud-init typically doesn't. As
such, there is no specific Ignition feature to execute commands after
switching root, but this can still be done by adding a systemd unit.
This change collects the cloud-config bootcmds into a single shell
script and executes them via a unit. Failure is ignored because that's
what cloud-init does.

Signed-off-by: James Le Cuirot <jlecuirot@microsoft.com>
Namely, logrotate.sh, ci-syslog-watcher.sh, teleportd, and
aks-secure-tls-bootstrap-client. These are referenced by absolute path
in their systemd units and should therefore be safe to move.

/usr/local/bin is generally read-only on immutable distributions.

The bootcmd in nodecustomdata.yml provides compatibility with existing
images by creating symlinks in /opt/bin if necessary.
/usr/local/bin is generally read-only on immutable distributions. A
profile.d script has been added to ensure /opt/bin is in the PATH.
/usr/local/bin is generally read-only on immutable distributions, so
these binaries cannot be installed there.

The packages published to PMC and the system extensions built from those
install their binaries to /usr/bin. These were previously moved to
/usr/local/bin, but since this is no longer possible, symlinks are now
created in /opt/bin instead.

Symlinks are not used when installing from a tarball because the
versioned binaries (e.g. kubelet-*) are removed, leading to dangling
links.

The `install` command has been used to write these binaries because it
removes the existing file first (rather than following a symlink) and
takes care of making the new file executable.
Copy link
Copy Markdown
Contributor

@hbeberman hbeberman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good to me from an AZL/OSGuard perspective.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

components This pull request updates cached components on Linux or Windows VHDs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

7 participants