Skip to content

Disable systemd service on CoreOS live filesystems#1088

Merged
cgwalters merged 1 commit into
coreos:mainfrom
nikita-dubrovskii:issue-2136
Apr 29, 2026
Merged

Disable systemd service on CoreOS live filesystems#1088
cgwalters merged 1 commit into
coreos:mainfrom
nikita-dubrovskii:issue-2136

Conversation

@nikita-dubrovskii
Copy link
Copy Markdown
Contributor

@nikita-dubrovskii nikita-dubrovskii commented Apr 21, 2026

When running from live-iso bootupctl fails with:

    error: Querying adoptable state: creation time is not available for the filesystem

That happens because filesystem (erofs|squashfs) doesn't support statx.stx_btime field:

    $ stat /sysroot/.coreos-aleph-version.json
    Access: 2022-08-01 23:42:11.000000000 +0000
    Modify: 2022-08-01 23:42:11.000000000 +0000
    Change: 2022-08-01 23:42:11.000000000 +0000
     Birth: -

Issue: coreos/fedora-coreos-tracker#2136

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the bootloader-update.service to prevent it from running on live ISO environments by adding a ConditionKernelCommandLine=!coreos.liveiso check. This addresses a failure caused by the lack of statx.stx_btime support on ISO9660 filesystems. I have no further feedback to provide.

Copy link
Copy Markdown
Member

@dustymabe dustymabe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM - some suggestions.

Comment thread systemd/bootloader-update.service
Comment thread systemd/bootloader-update.service Outdated
Type=oneshot
# bootupd is enabled by default in CoreOS (F43+ / CentOS9+).
# On live ISOs, it fails to read metadata from
# "sysroot/.coreos-aleph-version.json" because the filesystem
Copy link
Copy Markdown
Member

@dustymabe dustymabe Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I realize here that the problem we hit was specific (i.e. timestamp on sysroot/.coreos-aleph-version.json), but I'm thinking the reasoning for this condition shouldn't be that speicific.

i.e. I was thinking along the same lines as @travier when he said:

Where are we running bootupd from the LiveISO? I don't think we should support that.

So I was thinking what we should do is detect when it wouldn't make sense for bootupd to run to try to update the bootloader. That's why my suggestion over in this comment](coreos/fedora-coreos-config#4121 (comment)) was related to the filesystem on /boot (i.e. boot.mount) and not /sysroot.

All of this is to say.. should we change the check below to findmnt on /boot instead and change this comment to something more about "it doesn't make sense to update bootloader in in "Live" environments. Rather than specifically mentioning the aleph file timestamp thing.

Copy link
Copy Markdown
Contributor Author

@nikita-dubrovskii nikita-dubrovskii Apr 22, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Problem with /boot is it's not mounted in fcos:

core@localhost:~$ findmnt /boot                                                                                                                                                                                                                                                  
core@localhost:~$ echo $?                                                                                                                                                                                                                                                                 
1

But checking /sysroot always works.

Update: i speak here about Live env.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm. I guess maybe a behavior difference between FCOS and RHCOS. I had a RHCOS 10.2 ISO laying around and this is what I see:

Red Hat Enterprise Linux CoreOS 10.2.20260325-0 (Coughlan)
Kernel 6.12.0-211.5.1.el10_2.x86_64 on x86_64

SSH host key: SHA256:4u+zqJiPzdjNR5/nOGCzDDjAoc9lFZw6NYm3YIfsMRM (ED25519)
SSH host key: SHA256:z1K8u0Gjyz/XcwGvchYgILnlQ6HfgduV8K3TjAa7dqg (ECDSA)
SSH host key: SHA256:M9ythKHnv+LZajdYWb3MYs8m+11M6lOws4Jb67khk9w (RSA)
Ignition: ran on 2026/04/22 13:40:28 UTC (this boot)
Ignition: no config provided by user
localhost login: core (automatic login)

Last login: Wed Apr 22 13:40:41 on tty1

###########################################################################
Welcome to the CoreOS live environment. This system is running completely
from memory, making it a good candidate for hardware discovery and
installing persistently to disk. Here is an example of running an install
to disk via coreos-installer:

sudo coreos-installer install /dev/sda \
    --ignition-url https://example.com/example.ign

You may configure networking via 'sudo nmcli' or 'sudo nmtui' and have
that configuration persist into the installed system by passing the
'--copy-network' argument to 'coreos-installer install'. Please run
'coreos-installer install --help' for more information on the possible
install options.
###########################################################################

[core@localhost ~]
[core@localhost ~]$ findmnt /boot
TARGET SOURCE            FSTYPE OPTIONS
/boot  /dev/loop1[/boot] erofs  ro,relatime,seclabel,user_xattr,acl,cache_strategy=readaround

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, on rhcos /boot is mounted.

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok so /boot is mounted in the ISO in RHCOS, but not FCOS?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes.

@dustymabe dustymabe changed the title Disable systemd service on coreos.live-iso Disable systemd service on CoreOS live filesystems Apr 22, 2026
@dustymabe
Copy link
Copy Markdown
Member

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds an ExecCondition to the bootloader-update.service to prevent it from running on live ISO environments where the filesystem (erofs or squashfs) lacks support for stx_btime, which causes metadata read failures. The reviewer suggested replacing the bash-based regex check with a native findmnt command using the --types flag to improve efficiency and ensure the service is correctly skipped if the mount point cannot be found.

Comment thread systemd/bootloader-update.service
@nikita-dubrovskii
Copy link
Copy Markdown
Contributor Author

/gemini review

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request adds an ExecCondition to the bootloader-update.service to prevent bootloader updates in live environments by checking the filesystem type of /sysroot. A review comment identifies a syntax error in the findmnt command, noting that excluding multiple filesystem types requires a single 'no' prefix followed by a comma-separated list (e.g., 'noerofs,squashfs') rather than prefixing each type with 'no'.

Comment thread systemd/bootloader-update.service Outdated
@nikita-dubrovskii nikita-dubrovskii force-pushed the issue-2136 branch 2 times, most recently from 5306ae4 to ca489ba Compare April 22, 2026 14:21
Copy link
Copy Markdown
Member

@dustymabe dustymabe left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@cgwalters
Copy link
Copy Markdown
Member

I believe #1072 would fix this too, and I think that approach is cleaner. WDYT?

@dustymabe
Copy link
Copy Markdown
Member

I would fix this too, and I think that approach is cleaner. WDYT?

@nikita-dubrovskii mind evaluating #1072 ?

@nikita-dubrovskii
Copy link
Copy Markdown
Contributor Author

I believe #1072 would fix this too

Nope, added some traces:

core@localhost:~$ journalctl -u bootloader-update.service 
Apr 23 09:18:51 localhost systemd[1]: Starting bootloader-update.service - Update bootloader on boot...
Apr 23 09:18:52 localhost bootupctl[1137]: TRACE: Calling get_devices from prep_before_update()
Apr 23 09:18:52 localhost bootupctl[1137]: TRACE: get_devices(target_root=/)
Apr 23 09:18:52 localhost bootupctl[1137]: TRACE: inspect_filesystem(path=.)
Apr 23 09:18:52 localhost bootupctl[1137]: TRACE: inspect_filesystem(path=.)
Apr 23 09:18:52 localhost bootupctl[1137]: TRACE: inspect_filesystem returned: source=/dev/loop1, fstype=erofs
Apr 23 09:18:52 localhost bootupctl[1137]: TRACE: get_devices returned: Some([])
Apr 23 09:18:52 localhost bootupctl[1137]: TRACE: Calling query_adopt_state from status() for component: BIOS
Apr 23 09:18:52 localhost bootupctl[1137]: TRACE: query_adopt_state()
Apr 23 09:18:52 localhost.localdomain bootupctl[1137]: error: Querying adoptable state: creation time is not available for the filesystem
Apr 23 09:18:52 localhost.localdomain systemd[1]: bootloader-update.service: Main process exited, code=exited, status=1/FAILURE
Apr 23 09:18:52 localhost.localdomain systemd[1]: bootloader-update.service: Failed with result 'exit-code'.
Apr 23 09:18:52 localhost.localdomain systemd[1]: Failed to start bootloader-update.service - Update bootloader on boot.

and I think that approach is cleaner. WDYT?

Question is should we skip run in Live-env or run and exit gracefully. I'm for skipping run completely

@nikita-dubrovskii
Copy link
Copy Markdown
Contributor Author

nikita-dubrovskii commented Apr 23, 2026

@cgwalters , i've attached a patch to your PR, with it everything works:

core@localhost:~$ journalctl -u bootloader-update.service 
Apr 23 11:48:29 localhost systemd[1]: Starting bootloader-update.service - Update bootloader on boot...
Apr 23 11:48:30 localhost bootupctl[1069]: No block-backed boot filesystem found; bootloader update is not applicable, skipping.
Apr 23 11:48:30 localhost systemd[1]: Finished bootloader-update.service - Update bootloader on boot.

@nikita-dubrovskii
Copy link
Copy Markdown
Contributor Author

@cgwalters saw your rebased PR. Still doesn't work:

Apr 24 07:43:05 localhost bootupctl[1107]: client_run_update: rootcxt=Device { name: "loop1", serial: None, model: None, partlabel: None, parttype: None, partuuid: None, partn: None, children: None, size: 904198144, maj_min: Some("7:1"), start: None, label: None, fstype: Some("ero"), uuid: Some("cb36fafd-58a7-4728-a485-6dba0af008c8"), path: Some("/dev/loop1"), pttype: None }
Apr 24 07:43:06 localhost.localdomain bootupctl[1107]: error: Querying adoptable state: creation time is not available for the filesystem
Apr 24 07:43:06 localhost.localdomain systemd[1]: bootloader-update.service: Main process exited, code=exited, status=1/FAILURE

Here is a branch with fix, mind checking it?

Comment thread systemd/bootloader-update.service Outdated
@dustymabe
Copy link
Copy Markdown
Member

I think it makes sense to do this AND #1072

This still LGTM to merge if you like Nikita.

When running from live-iso `bootupctl` fails with:

    error: Querying adoptable state: creation time is not available for the filesystem

That happens because filesystem (erofs|squashfs) doesn't support statx.stc_btime field:

    $ stat /sysroot/.coreos-aleph-version.json
    Access: 2022-08-01 23:42:11.000000000 +0000
    Modify: 2022-08-01 23:42:11.000000000 +0000
    Change: 2022-08-01 23:42:11.000000000 +0000
     Birth: -
@cgwalters cgwalters merged commit 02a2a87 into coreos:main Apr 29, 2026
10 of 12 checks passed
@nikita-dubrovskii nikita-dubrovskii deleted the issue-2136 branch April 29, 2026 12:17
@Johan-Liebert1 Johan-Liebert1 mentioned this pull request Apr 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants